Abnormal node detection method in hierarchical network

Abstract

The invention discloses an abnormal node detection method based on tomography reasoning in a hierarchical network. According to the method, a plurality of groups of intersecting paths are selected from all possible transmission paths, and a solvable equation set taking a node packet loss rate as a variable is constructed, so that the network node anomaly is evaluated. According to the invention, a non-intrusive network anomaly detection framework is established, and a detection path construction strategy and a fault positioning algorithm are provided. According to the method, code logic of forwarding nodes in the hierarchical network does not need to be modified, node state information can be obtained only by depending on end-to-end path detection, so that the problems of feasibility and invasiveness of a traditional method in actual deployment are solved, and the method has high universality and engineering practical value.

Description

Technical Field

[0001] This invention belongs to the field of network management and network security technology, specifically relating to an abnormal node detection method suitable for hierarchical networks. More specifically, this invention proposes a non-intrusive detection mechanism based on multi-path cross-validation, which can be used to evaluate the stability and potential security risks of each node in a multi-layered network structure, and is particularly suitable for heterogeneous network environments that lack centralized security control capabilities. Background Technology

[0002] As network structures become increasingly complex, layered network architectures are widely used in data center networks, software-defined networks (SDN), multi-region service meshes, VPN systems, and some anonymous communication networks. These networks typically consist of multiple functionally layered nodes, with data being forwarded layer by layer to complete end-to-end communication. The overall network performance and security highly depend on the availability and trustworthiness of intermediate nodes. However, in actual operation, these intermediate nodes may experience data transmission interruptions, increased latency, or leakage of sensitive information due to hardware or software failures, configuration errors, performance bottlenecks, or even malicious control. Especially in network environments lacking centralized control and trusted auditing, such as volunteer-driven anonymous networks, decentralized communication platforms, or cross-regional distributed VPN networks, monitoring node status and detecting anomalies become critical challenges for network security operations and maintenance.

[0003] Traditional methods for monitoring node health status primarily include: reputation score systems, which establish a scoring system based on a node's performance in historical communications (such as packet loss rate and response latency) for path selection and risk assessment; and node log collection, which involves deploying log reporting modules at the node end to record its operational status and processing behavior, and then centrally uploading the logs to a security server for subsequent analysis. These methods have several significant limitations: subjectivity and instability. Reputation score systems rely on specific scoring algorithms and time windows, making their evaluation results susceptible to short-term fluctuations and difficult to accurately distinguish between "temporarily abnormal" and "persistently risky" nodes. Furthermore, malicious nodes can mislead the system through "reputation manipulation attacks," leading to misjudgments of benign nodes. Deployment complexity is high; log collection methods often require code-level modifications to nodes and the introduction of new services and support. This is difficult to implement in volunteer-participated networks (such as Mixnet or Tor) or hardware-centric networks (such as switch data centers), and the high intrusiveness may increase security concerns for node operators. In addition, not all data anomalies can be collected by the log system, such as hardware and firmware failures.

[0004] Therefore, there is an urgent need for a non-intrusive detection mechanism that does not require modification of existing nodes, is easy to deploy, and has high evaluation accuracy, and can effectively identify abnormal or potentially malicious intermediate nodes in the network in untrusted or low-cooperation environments. Summary of the Invention

[0005] To overcome the problems of complex deployment, high invasiveness, and insufficient evaluation accuracy in existing technologies, this invention proposes a hierarchical reasoning-based abnormal node detection method suitable for layered network architectures. This method is non-intrusive and has low deployment costs, requiring no code modification to existing network nodes or relying on the deployment of new services to evaluate the stability and abnormal behavior of relay nodes in the network.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An anomaly node detection method for hierarchical networks is characterized by employing a strategy based on path intersection and hierarchical reasoning. This involves constructing multiple cross-layer detection paths and analyzing the detection results to achieve accurate judgment of the state of each node. The method includes the following steps:

[0008] S1: Obtain the structural information of the target hierarchical network and identify the set of available nodes in each layer.

[0009] S2: Select several paths to be probed based on the minimum identifiable strategy. The minimum identifiable strategy is the theoretical basis for ensuring that this method can accurately identify abnormal nodes. Its purpose is to select the fewest probe paths to reduce the probe cost while ensuring the accuracy of node evaluation.

[0010] S3: Inject probe packets into the network from the ingress node and perform statistics and recording at the egress node. Each path must pass through l nodes, where l is the network layer number, and only one node is selected from each layer. Based on the path control capabilities of the hierarchical network, a control mechanism is constructed to explicitly specify the packet transmission path.

[0011] S4: Based on node behavior analysis algorithms, abnormal nodes are located. The analysis algorithm is divided into node-level anomaly detection and port-level anomaly detection, ensuring that this method is applicable to almost all hierarchical network anomaly detection tasks.

[0012] S5: Repeat steps S1 to S4 until all abnormal nodes are detected.

[0013] Furthermore, the feature is that the minimum identifiable strategy described in step S2 requires first modeling all paths in the network as a matrix, and then selecting a specific number of valid paths from it. In the path matrix, each row represents a path, and each column represents whether a node is selected, where 0 represents that the node is not selected and 1 represents that it is selected.

[0014] Furthermore, the key feature is that the selection of effective paths involves choosing the smallest submatrix with a stable and unique solution from the constructed matrix. The paths contained within this submatrix are the effective detection paths. A submatrix with a stable and unique solution must satisfy the following condition: based on the constructed system of equations... The transmission success rate of each node {s1,s2,…,s} N It has a stable and unique solution.

[0015] Furthermore, the process of selecting the submatrix is ​​characterized by modeling: constructing an equation using the sending records and receiving results of path i. The success rate of a path is the product of the success rates of multiple transmission nodes. Where S... i M represents the success rate of path i detection. ij Let s be the i-th row of the submatrix, representing the sequence of nodes traversed by path i. j Let be the transmission success rate of node j. The equations corresponding to all paths in M ​​form a system of equations.

[0016] Furthermore, the control mechanism described in step S3 refers to a method that uses technologies such as software-defined networking, programmable switches, or multi-layer encapsulation structures to perform complete or partial path control on the probe packet path.

[0017] Furthermore, the node behavior analysis algorithm described in step S4 can be achieved by solving a system of equations. Obtain the transmission success rate S of all nodes. j Then, based on the calculation results, the transmission success rate for each path is calculated. And compared with the actual detection success rate S i A comparison was performed to verify the consistency of the results.

[0018] Furthermore, the feature is that when the calculation result is close to the actual detection result, the calculation result is used as the node packet loss rate; otherwise, a more detailed port packet loss rate calculation process is performed, because a small number of ports may be abnormal while a large number of ports are normal.

[0019] Furthermore, the key feature is that, during the calculation of port packet loss rate, it is first necessary to identify the path causing the inconsistency. This path is then broken down into several links. After decomposition, if the success rate of a certain link cannot be directly determined, it needs to be combined with other links with known high success rates to form a new probe path and then used for auxiliary probes according to the aforementioned method to indirectly calculate the success rate of all links.

[0020] Furthermore, the method is characterized in that, in step S4, a system of equations including error terms is employed for solving the equations. During the solution process, the method should minimize the differences between all calculated values ​​and actual measured values ​​while avoiding excessive focus on the influence of factors such as measurement errors and network noise.

[0021] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0022] (1) Non-intrusive: This method combines equation solving theory with gradient descent algorithm to construct a layered multi-way cross calculation algorithm, which can infer node performance from path performance, avoids modification of multi-layer network node code, and eliminates the need for additional deployment.

[0023] (2) Accuracy: Using packet loss rate as a node stability indicator effectively avoids the problems of unclear monitoring intervals and indicators. Furthermore, the theoretical boundaries of effective detection paths are constrained to reduce invalid detections and improve computational efficiency.

[0024] (3) Robustness: The underlying logic of complex attack behavior was analyzed and the corresponding detection logic was implemented to prevent the attacker from exploiting it and causing adverse effects, thus expanding its scope of application; and finally, it achieved universal support for layered networks. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the abnormal node detection method based on hierarchical reasoning in an embodiment of the present invention.

[0027] Figure 2 This is a simplified schematic diagram of the detection path matrix in an embodiment of the present invention.

[0028] Figure 3 This is a schematic diagram illustrating the principle of root cause partitioning based on weighted least squares in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This invention aims to provide an abnormal node detection method suitable for hierarchical network architectures. This method infers the operational status and potential abnormal behavior of each node in the network by injecting probe data packets and analyzing their transmission success rate.

[0031] Layered networks are a common network organization form, offering good scalability and manageability. By flexibly expanding nodes within each layer or employing domain-based transmission mechanisms, network size can be rapidly adjusted to meet business needs. However, the complexity of its multi-layered structure and the limitations of some intermediate nodes' computing power or functionality make faults or abnormal behaviors more easily masked, thus impacting overall Quality of Service (QoS) and network security. In recent years, layered analysis, as a non-intrusive network state reasoning method, has provided new possibilities for state awareness and anomaly detection in multi-layered networks. This technology constructs path observation models to indirectly infer the behavior of invisible nodes, avoiding modification and intervention of the nodes themselves.

[0032] The proposed anomaly node detection method constructs a technical framework for efficiently applying hierarchical inference in hierarchical networks, encompassing a minimum identifiable strategy and a multi-granularity behavior analysis algorithm. This method ensures controllable transmission costs of probe data packets while guaranteeing the coverage and discriminative power of the collected information. Furthermore, by introducing port-level behavior modeling, it can effectively identify fine-grained anomalies in nodes, enhancing adaptability and detection accuracy in various complex scenarios.

[0033] Example:

[0034] To further illustrate the technical solution and implementation of this invention, the following detailed description is provided using a typical application scenario—a layered hybrid network (Mixnet). This embodiment is merely an example and does not limit the application of this invention to other layered network architectures, such as data center networks, VPN systems, or multi-domain service meshes.

[0035] A layered hybrid network is a privacy-preserving communication network architecture. Its core objective is to anonymize user communication paths through the mixing and forwarding operations of multiple layers of nodes. In this architecture, the network is divided into multiple logical layers, each consisting of multiple mixing nodes. Data packets must be processed and forwarded sequentially through one node in each layer to complete communication from source to destination.

[0036] This network structure typically has the following characteristics: a clear hierarchical structure, with each communication path starting from the first layer, selecting a node at each layer for mixing processing until the last layer node sends out the data packet; strong node autonomy, with most Mix nodes maintained by independent entities or volunteers, lacking a unified control and management mechanism; path invisibility, as nodes do not share global path information for privacy purposes, making it difficult to use traditional centralized monitoring methods; and random and unpredictable forwarding behavior, which also makes it easy for malicious nodes to exploit for packet loss, tampering, or analysis attacks. Given these characteristics, the layered Mixnet is a typical multi-hop, unobservable, and non-cooperative layered network structure, making it highly suitable for the application scenarios of the abnormal node detection method of this invention. Through hierarchical reasoning of the construction and results of the detection path under this structure, the evaluation of node operating status and anomaly detection can be effectively achieved without modifying node code or requiring a security server.

[0037] like Figure 1 As shown, the method described in this embodiment includes the following steps:

[0038] This method operates on an epoch-based basis. An epoch represents several probe and localization cycles (S1-S4). After each epoch, the calculation results, including reliable and unreliable nodes, are reported to the user.

[0039] S1, the sender first selects a large probe range G, for example, a subset of nodes that contains 1 / 10 of the total number of nodes in the entire hybrid network.

[0040] S2, transform all possible paths into a K×N matrix P, where K is the count of all paths and N is the count of nodes in G. Each column of matrix P corresponds to a node. If a node is contained in path i, then P... i =1.

[0041] Simplify matrix P to form submatrix M. Select A paths from the N nodes of matrix P, such that the matrix M formed by these A paths has a rank of N-l+1, where l is the number of layers in the hierarchical hybrid network.

[0042] The minimum identifiable strategy refers to selecting the smallest set of equations that can provide a stable solution. This requires modeling the detection process.

[0043]

[0044] Treat M as an unsimplified detection matrix, i.e., M = P, M ij This indicates whether node i is included in path j. (Based on the success rate calculation formula...) All paths existing in P can be represented as a system of equations:

[0045] The system of equations is then solved, but it cannot be solved directly; therefore, a logarithmic operation is required on both sides. Considering M... ij The value could be 0, so after transforming the above equation and taking its logarithm, we obtain a system of equations:

[0046] At this point, it is necessary to ensure that log(s) j A stable solution exists. According to the principle of unique matrix solutions, M must be a full-rank matrix for s to have a stable solution. j A stable solution exists. However, due to the path selection logic, M cannot be a full-rank matrix. Therefore, it is necessary to find secondary mathematical constraints. For example...

[0047]

[0048] Figure 2 The diagram shows the simplified constraint logic for M. A hybrid network guarantees at least one normal node in each layer; in this example, we assume nodes A, F, and H. This can be determined from the vector s = {s...} A ,s B ,...,s I} T Remove the corresponding unknowns, since log(1) = 0, removing nodes with a 100% success rate does not affect the solution of the above system of equations. After removing the above points, matrix M becomes the new matrix M'. At the same time, both M and the measured values ​​need to be removed according to the corresponding node removal operation. The matrix M' after removing nodes is a matrix containing Nl nodes, and its full rank is also Nl, which corresponds to the rank of the original matrix M being N-l+1. Therefore, it can be determined that the theoretical constraint for the equation to have a unique solution is that the rank of the original matrix M should be N-l+1. Therefore, it is only necessary to delete paths that do not affect the rank and retain the minimum number of N-l+1 paths to form M.

[0049] S3, the sender sends k probe packets along all paths described in the probe matrix M. The receiver counts the probe packets and reports the results to the sender via a SURB structure, thus ensuring the non-linkability between the sender and receiver. Each probe packet has different identifier bits containing the round number, path ID, and sequence number. Several probe packets corresponding to the same path can use different path IDs, which improves security. The receiver counts the packets based on the round number, path ID, and sequence number after unpacking. SURB is a secure encapsulation method. The sender includes the required round number, path ID, and sequence number in the first half of the SURB. The receiver can use the second half of the SURB as the header, add new content after it to form a new packet, and then automatically forward the packet to the destination described in the SURB by placing it in the hybrid network.

[0050] The detection frequency k for each path is determined by the maximum detection frequency of its constituent nodes. All nodes have the same initial detection frequency, but the detection results in each round trigger a detection frequency modification function. By adjusting the detection frequency, it is possible to detect as many abnormal behaviors as possible within a certain detection capability. The adjustment logic is to apply c←c+0.1·c to nodes with packet loss rates near a threshold. 1.8 For nodes significantly less than the threshold, apply c←c-0.05·c 1.2 For nodes significantly larger than the threshold, apply c←c+0.1·c 1.2 At the same time, nodes with similar detection frequencies are combined to form the next round of detection.

[0051] S4, the sending node constructs an equation based on the sending records and counting results of path i. Where S i M is the path-related probe success rate, which is the ratio of the number of probe packets arriving at the receiving node to the number of packets sent. ij This is the path selection matrix, representing all nodes contained in path i, s j The transmission success rate of each node is the term to be solved. The equations corresponding to all paths are formed into a system of equations, which are then solved using the least squares method to obtain the transmission success rate of all nodes.

[0052] The calculation results are verified to match the detection results. If they do not match, weighted least squares is used to infer the link causing the inconsistency. The weight of each path is the square of the success rate of the route; that is, the higher the success rate of a path, the more likely its nodes are to be considered non-abnormal. The success probability of each node can be calculated to deduce the success rate of each transmission path. Figure 3 As shown, packet loss on link CE results in a Transmission Success Rate (TSR) of 0.63 for node C calculated using the least squares method. After using the weighted least squares method, the TSR of node C is 0.96, which is closer to the true value.

[0053] At this point, it can be discovered Figure 3 The measured and calculated values ​​for path BCD are close, so no further investigation is needed. However, path ACE shows a significant discrepancy between the measured and calculated values, requiring it to be broken down into two links, AC and CE. Two links with known low packet loss rates should be selected and combined, such as AC with CG and BC with CE. Then, probing is performed on these two paths, ACG and BCE, and the success rate is calculated. This identifies the problematic link as CE.

[0054] S5, Repeat steps (S1-S4) until all unstable nodes are discovered. Each repetition constitutes one round, and each round excludes some unstable nodes. To improve accuracy and handle nodes that recover quickly, only nodes that are counted as packet loss for three consecutive rounds should be excluded. After several rounds, if there are no nodes in the network that need to be dispatched, it means an epoch is complete. Probe nodes can continue to execute new epochs for 24 / 7 monitoring. Excluded unstable nodes are not removed from the network and can continue to be probed in the next epoch to restore functionality.

[0055] This document uses examples of methods and network environments to illustrate the principles and implementation methods of the present invention. The descriptions of these embodiments are merely for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for anomaly node detection for a layered network, the method comprising: The method comprises the following steps: S1: Obtain the structure information of the target layered network, and identify the available node set in each layer. S2: Select a plurality of to-be-probed paths based on a minimum identifiable strategy. The minimum identifiable strategy is a theoretical basis for ensuring that the method can accurately identify abnormal nodes, and the purpose is to select the least number of probing paths to reduce the probing cost while ensuring the accuracy of node evaluation. S3: Inject a probing data packet from an entry node into the network, and perform statistics and recording at an exit node. Each path needs to pass through l nodes, where l is the number of network layers, and only one node is selected in each layer. Based on the path control ability of the layered network, a control mechanism is constructed to realize the explicit designation of the data packet transmission path. S4: Based on a multi-granularity behavior analysis algorithm, the positioning of abnormal nodes is realized. The analysis algorithm includes node-level anomaly detection and port-level anomaly detection, which ensures that the method is applicable to almost all layered network anomaly detection tasks. S5: Repeat steps S1 to S4 until all abnormal nodes are detected.

2. The method of claim 1, wherein, The minimum identifiable strategy in step S2 needs to first model all paths in the network as a matrix, and then select a specific number of effective paths from the matrix. In the path matrix, each row represents a path, and each column represents whether a node is selected, where 0 represents that the node is not selected and 1 represents that the node is selected.

3. The method of claim 2, wherein, The selection of the effective path is the process of selecting the minimum sub-matrix with stable and unique solution from the constructed matrix, and the path contained in the sub-matrix is the effective detection path. The sub-matrix with stable and unique solution needs to satisfy the condition: based on the constructed equation group The transmission success rate {s1, s2, …, s N} of each node has stable and unique solution.

4. The method of claim 3, wherein, The process of selecting the sub-matrix is based on modeling: using the sending record and receiving result of path i to build equations The product of the success rates of multiple transmission nodes is the transmission success rate of the path. Among them, S i represents the detection success rate of path i, M ij is the i-th row in the sub-matrix, represents the node sequence passed through by path i, s j is the transmission success rate of node j. The equations corresponding to all paths in M constitute an equation group 5. The method of claim 1, wherein, The control mechanism in step S3 refers to a method of completely or partially controlling the path of the probing packet by using software-defined network, programmable switch, or multi-layer encapsulation structure technology.

6. The method of claim 1, wherein, The node behavior analysis algorithm in step S4 can be solved by solving the equation set Obtain the transmission success rate s of all nodes j . Then, based on the calculation results, the calculated transmission success rate of each path is deduced and compared with the actual detection success rate S i to verify the consistency of the results.

7. The method of claim 6, wherein, When the calculation result is similar to the actual probing result, the calculation result is taken as the node packet loss rate, otherwise a more detailed port packet loss rate calculation process is performed, because the node may have a small number of port abnormalities and a large number of normal ports.

8. The method of claim 7, wherein, In the port packet loss rate calculation process, the path causing the inconsistency needs to be determined first. Then it is disassembled into several links. After disassembly, if the success rate of a certain link cannot be directly determined, it needs to be combined with other links with high known success rates to form a new probing path and perform auxiliary detection according to the method of claim 5 to indirectly calculate the success rate of all links.

9. The method of claim 1, wherein, In step S4, an equation system containing error terms is used to solve the method. In the solving process, the method should minimize the difference between all calculated values and actual measured values while avoiding excessive attention to measurement errors and network noise and other factors.

Images