Distributed large-scale anti-traceability elastic network intelligent routing method and system

Abstract

The invention provides a distributed large-scale anti-traceability elastic network intelligent routing method and system, belongs to the field of network communication and network security, and is suitable for intelligent routing decision optimization in a dynamic network environment. The method is based on a multi-agent reinforcement learning framework, network nodes are mapped into independent agents, and path traceability risks are blocked through local information constraints; dynamic feature aggregation of a neighbor link state is realized by adopting a lightweight graph attention network, and the local sensing efficiency of a large-scale network is improved; a QMIX algorithm is introduced, and network parameters are optimized by nonlinear fusion of a local Q value and global graph state representation through a hybrid network; and in combination with a self-adaptive exploration mechanism driven by action entropy, the sudden change scene strategy response capability is enhanced. According to the system, in military anonymous communication, dark network data transmission and cross-border sensitive services, the anti-traceability and transmission efficiency balance can be remarkably improved, the characteristics of high concealment, high elasticity and low resource consumption are achieved, and a systematic routing solution is provided for a dynamic network environment.

Description

Technical Field

[0001] The present invention belongs to the field of network communication technology. Specifically, the present invention relates to a distributed large-scale reverse tracing elastic network intelligent routing method and system. Background Art

[0002] With the rapid development of the internet, network scale continues to expand, the number of nodes increases dramatically, and network topologies become increasingly complex. Traditional centralized routing mechanisms, when used in large-scale networks, suffer from problems such as high routing latency, low bandwidth utilization, and a high risk of single points of failure. Furthermore, while existing anonymous communication networks utilize dynamic circuit mechanisms, paths remain fixed within a single session, and entry node selection strategies suffer from long-term immutability. This leads to the accumulation of traffic fingerprints and the risk of cross-circuit timing correlation attacks, making true dynamic reverse tracing difficult.

[0003] Existing distributed routing algorithms mitigate the scalability issues of centralized control through local decision-making. However, in dynamic topologies and large-scale node scenarios, they still face challenges such as frequent routing oscillations and a lack of global coordination in path optimization. Distributed Q-learning routing methods rely on nodes to independently learn forwarding strategies, leading to the accumulation of local optimal solutions and difficulty balancing network load and end-to-end latency. In the field of anonymous communication, the mainstream Tor network uses multi-layer relay encryption to hide communication endpoints. However, its fixed entry nodes and periodic rerouting mechanism still reveal user identities due to traffic timing characteristics. Attackers can use long-term traffic fingerprints to correlate the entry nodes of different sessions and conduct cross-circuit tracing attacks. Although existing dynamic routing anonymity technologies use randomized path selection, path generation relies on a centralized coordinator, which poses a single point of failure risk and does not address the dynamic coordinated optimization of multi-hop paths in large-scale networks. Furthermore, routing methods based on graph neural networks (GNNs) improve routing decision efficiency by modeling network topology relationships. However, traditional GNNs require message passing across the entire graph topology, resulting in computational complexity that increases exponentially with the number of nodes, making them unsuitable for ultra-large-scale networks.

[0004] Currently, distributed, large-scale, anti-traceable, elastic network routing technology faces the following challenges: First, scalability bottlenecks in dynamic topologies and ultra-large-scale networks. Traditional centralized / semi-centralized architectures cannot efficiently support dynamic node scaling. Furthermore, GNN routing methods based on global graph computations face exponential computational complexity, making them difficult to adapt to the real-time changes in ultra-large-scale network topologies. Second, anonymity flaws stemming from centralized reliance and cross-session associations. Existing anonymous routing mechanisms, due to fixed entry node selection, centralized path coordination, and exposed traffic timing characteristics, face the risk of cross-circuit traceability attacks and single points of failure, making it difficult to achieve dynamic anti-traceability goals. Third, the lack of global optimization of the coordination mechanism between distributed agents makes it difficult to coordinate local routing strategies with global network goals, leading to unbalanced resource allocation and suboptimal path optimization. Therefore, existing technologies cannot effectively support the combined requirements of efficient routing, elastic adaptation, and anti-traceability in dynamic, anonymous networks. An innovative distributed routing approach is urgently needed. Therefore, the multi-agent reinforcement learning framework combined with QMIX and the Neighborhood Sampling Graph Attention Network (NSGAT) were introduced into distributed routing research. The complexity of large-scale topological calculations was reduced through local subgraph sampling to adapt to dynamic scaling scenarios. A distributed agent architecture and local information decision-making mechanism were designed to block path associations and achieve lightweight anti-traceability. The local Q value and the global graph state were dynamically integrated through the hybrid network to collaboratively optimize end-to-end latency and throughput, breaking through the global collaboration bottleneck of distributed agents. Summary of the Invention

[0005] In view of this, the present invention proposes a distributed large-scale anti-traceability elastic network intelligent routing method and system, aiming to solve the problems of low transmission efficiency, insufficient anonymity and unbalanced resource allocation caused by node scale expansion, path correlation attack threats and suboptimal local decision-making in a dynamic topology network environment.

[0006] To achieve the above object, the present invention provides the following technical solutions: A distributed large-scale anti-traceability elastic network intelligent routing method and system, through the design of distributed elastic architecture, maps network nodes into independent agents, makes decisions based on local state, and blocks path-related traceability attacks; combines a lightweight graph attention mechanism to dynamically aggregate neighbor link states to achieve local topology awareness; adopts the QMIX multi-agent collaborative learning framework, dynamically generates weight matrices through hybrid networks, and nonlinearly integrates local Q values ​​into global optimization goals to ensure that routing strategies take into account low latency, high throughput, and load balancing; introduces an action entropy-driven adaptive exploration mechanism to support rapid model fine-tuning when topology changes drastically, ensuring service continuity. The routing method includes the following steps: Step 1: Build a multi-agent distributed anti-traceability elastic network model. Set up an agent for each network node. Agents communicate with each other through directly adjacent agents and exchange network status information. Distributed agents only know the previous and next hop information of the data and have anti-traceability capabilities. Step 2: Message communication between adjacent agents is implemented using a graph attention network. Through neighbor sampling, full-graph sampling is optimized to small-batch neighbor sampling centered on some nodes. Using the attention mechanism to distinguish the importance of neighbors, GAT is adaptable to large-scale network scenarios. Step 3: Introduce the QMIX algorithm to implement collaborative learning of agent intelligent routing. QMIX improves on the Deep Q-Network (DQN) and processes the joint action value function in a multi-agent environment through value function decomposition. Each network node acts as an independent agent, calculating a local Q value based on its own state and neighbor messages obtained by GAT through a local Q network to represent its local utility for routing decisions. QMIX's hybrid network dynamically integrates the local Q values ​​of all nodes through a supernetwork. Combined with the global graph state representation constructed by GAT, it nonlinearly generates a global Q value to optimize network objectives such as end-to-end latency and throughput. Step 4: Training is divided into three steps: distributed experience collection, centralized training, and distributed execution. Each node selects actions based on the improved entropy-driven adaptive exploration mechanism, executes routing decisions, and observes the utilization of local reward links. It periodically exchanges messages through GAT to build local states. and global state The hybrid network calculates the global Q value based on the global state, and optimizes the parameters of the local Q network and the hybrid network through the TD error, with the goal of minimizing the prediction error of the global Q value. After training, each node only relies on the local Q value and the neighbor information obtained by GAT to select an action, without the participation of the hybrid network. The distributed reverse-source elastic network intelligent routing gives the best next hop for the network node, and after combining them, the routing sequence of the service flow is obtained. Furthermore, the distributed elastic network model constructed in step 1 is specifically: A one-to-one mapping tightly coupled agent architecture is proposed to model the distributed large-scale reverse traceability elastic network environment as a graph. , Indicates the Anti-Traceability device set and represents the terminal link set, Represents the agent set, and the device set One-to-one mapping, the agent can only obtain local status through the bound device, avoiding global topology exposure and blocking tracing attacks based on path association. and Represent the global state and action space respectively. Assume that there is agents, each agent The local observation state is expressed as , then the global state space can be defined as the Cartesian product of these local states. Specifically, if each agent The local observation state From their respective local state spaces , define the global state space is the Cartesian product of the local state spaces of each agent ( ), the local observation state Contains device nodes queue depth, link bandwidth utilization, and global action space Local actions for each agent The Cartesian product of ), the local action Select the next routing path; Furthermore, the message communication between adjacent agents in step 2 realizes cross-agent collaborative decision-making through the GAT functional component, and each agent The internally deployed GAT module includes the following core functional units: Dynamic neighbor sampling and feature extraction: Each agent extracts local information based on its observations. Each agent then shares these features with neighboring agents in the graph and combines the received neighbor features with its own observations to enhance decision-making capabilities. A dynamic neighbor random sampling strategy is proposed to avoid the traditional GAT requiring all nodes in the graph to participate in the calculation. Combining the link delay changes, bandwidth utilization, and load balancing of the network topology, only some high-value neighbors are aggregated. Each agent only extracts some high-value neighbor nodes from the distributed large-scale reverse traceability elastic network instead of processing the entire graph data, reducing the computational load of distributed nodes and adapting to the dynamic nature of large-scale networks. Through the above-mentioned lightweight feature extraction unit, the agent encodes the queue load, adjacency table, bandwidth, and delay into feature vectors as the input basis of the GAT. By combining the attention mechanism with a distributed, large-scale, and resilient back-tracing network through attention weight aggregation, the agent distinguishes the importance of different neighbors. GAT assigns higher attention weights to neighboring nodes with high link quality and low load. This process does not require preset rules, but is dynamically adjusted through learnable parameters. The agent aggregates the feature vectors of neighboring nodes according to the attention weights. The aggregated information contains the context required for global collaboration. The above design enables the GAT component to extract network topology features while reducing communication overhead. The attention mechanism enables the system to flexibly respond to the dynamic changes of distributed large-scale back-traceable elastic networks, while the modular functional units facilitate distributed deployment and expansion, ultimately supporting an efficient, hidden, and highly available elastic network.

[0007] Furthermore, the collaborative learning mechanism of distributed elastic routing in step 3 uses the QMIX algorithm to implement multi-agent value function decomposition and global strategy optimization, which specifically includes the following technical features: Distributed local Q network architecture, for each agent Deploy a local Q network , whose input is the hidden state generated by the GAT module containing aggregated neighbor messages and local status representation of link indicators With optional actions ∈ , the output is the local Q value of the corresponding action, which represents the expected future reward of forwarding to the next hop in the current state of the local network; Traditional MARL uses a simple value decomposition method to add the Q values ​​of local agents to obtain the global Q value, but ignores the different contributions of different agents to the global Q value. QMIX assigns different contribution ratios to different agents, integrates the hybrid network with each local agent network, and the hybrid network Generate dynamic weight matrix With the bias vector , the local Q value of each agent The nonlinear combination is the global Q value, and the local Q value of each agent Satisfy the monotonicity constraint ( ), the specific form is ,in is the Exponential Linear Unit (ELU) activation function, and The global graph state representation constructed based on GAT Generate, where is the node-level aggregate feature; Distributed training mechanism, using the Centralized Training and Decentralized Execution (CTDE) framework, each agent in the training phase Regularly upload local experience data To the regional coordinator, the coordinator is based on the global reward , calculate the temporal difference (TD) error ,in represents the end-to-end delay, represents the throughput, Represents the packet loss rate, and finally updates the local Q network parameters synchronously through back propagation and hybrid network parameters , while freezing the target network parameters Training with stability; Furthermore, the distributed experience collection described in step 4 includes the following steps: The traditional epsilon-greedy strategy only uses linear or exponential decay rates and cannot be dynamically adjusted according to the actual exploration effect of the node. To solve this problem, an action entropy-driven adaptive annealing mechanism is proposed. v , calculate the information entropy of its historical action distribution , each agent Based on the exploration rate Dynamic annealing adjustment based on node historical action entropy If the agent v The action exploration is sufficient, the entropy is high, and it is quickly reduced. , give priority to high reward paths; if the action exploration is insufficient and the entropy is low, slow down Decay speed, encouraging diverse exploration; acting Perform the selected action After that, the state is transferred to , by collecting local immediate rewards ,in Indicates link utilization, Indicates the queue occupancy rate, Represents the link delay, using the GAT message passing mechanism to is a cycle, exchanging hidden state differences with neighbor agents Message, build local state , each agent Computing global graph feature vectors based on aggregated neighbor messages ,Traditional empirical data cannot explicitly reflect the real-time state changes such as the sudden drop of the bandwidth of neighboring nodes, the joining / exit of proxy nodes, etc., ,so it is considered to add the hidden state differences of neighboring nodes in the GAT multi-layer propagation ,into the empirical tuple. To perceive the sudden changes in bandwidth utilization and delay jitter caused by link quality fluctuations, dynamic changes in topology such as neighbor nodes going offline or new proxy nodes being added, and changes in neighbor queue load trends caused by traffic pattern changes, the regional coordinator adopts an asynchronous parallel gradient update mechanism to receive the experience tuples uploaded by each proxy. , build a global experience pool , where the global reward , Representation Agent The local immediate reward, represents the global throughput, represents the variance of end-to-end delay; Furthermore, the centralized training described in step 4 includes the following steps: The traditional MARL local network weights are fixed, and it only relies on the linear monotonic combination of local Q values. The local action value functions of each agent are summed up to obtain the joint action value function, ignoring the differences between different agents. The QMIX hybrid network is proposed to nonlinearly merge the local value functions of single agents. The hybrid weights of QMIX are dynamically generated by the super network, rather than the fixed scalar superposition of the traditional method. In addition, global state information is added to assist in the training and learning process. The hybrid network uses the global state to As a condition, a dynamic weight matrix is ​​generated through the hypernetwork With the bias vector , the QMIX algorithm converts the local Q value The nonlinear combination is the global Q value , and calculate the timing difference target ,in is the target hybrid network parameter; Minimize the loss function through a dual-delayed deep deterministic policy gradient framework ,in is the regularization coefficient, which suppresses the local Q value deviation and adopts the parameter delay synchronization strategy. Step synchronization of local Q network parameters To all agents; Furthermore, the distributed execution described in step 4 includes the following steps: Agents Deploy local Q networks and GAT modules with frozen parameters, and rely only on local observations during execution Neighbor messages aggregated with GAT, through greedy strategy Generate routing decisions. The routing sequence of the business flow is generated by an iterative path construction algorithm. The source node agent selects the optimal next hop node based on the Q value. The relay node agent recursively executes the same decision until it reaches the destination node, forming a complete routing path. ,The traditional method needs to be retrained globally when the performance degrades due to topology changes. ,This method propagates the routing delay and packet loss rate indicators back ,to the path nodes through the path feedback mechanism, triggering the ,incremental fine-tuning of the local Q network, and ,the model is updated without interrupting the service, and supports ,online adaptive updates. ,When it is detected that the number of new / removed nodes exceeds ,the number of new / removed nodes exceeds ,the number of new / removed nodes exceeds ,the number of new / removed nodes exceeding ,the number of new / removed nodes exceeding ,the number When the topology changes drastically, the federated learning protocol is started and each agent calculates the gradient based on local experience. , update the global model after multi-party aggregation; Furthermore, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the distributed large-scale reverse tracing elastic network intelligent routing method and system as described in any one of the above.

[0008] Compared with the prior art, the present invention has the following beneficial effects: First, this paper designs a dynamic topology perception and anti-tracing mechanism based on a lightweight graph attention network. It uses a dynamic neighbor sampling strategy to select highly stable and anonymous links, combined with multi-head attention feature aggregation, to perceive network state changes in real time and block traceability attacks based on path association.

[0009] Secondly, the present invention proposes a multi-agent collaborative decision-making architecture based on QMIX, which dynamically fuses local Q values ​​through a hybrid network, constructs a global state, and models routing decisions as a multi-agent collaborative optimization problem, while ensuring end-to-end delay and high throughput, and achieving optimization of load balancing and link congestion rate.

[0010] Furthermore, the present invention proposes a dynamic empirical collaboration mechanism based on hidden state differential perception. By recording the hidden state differences of neighboring nodes in multi-layer GAT propagation, it constructs a multidimensional feature encoding that encompasses link quality fluctuations, topology changes, and traffic pattern changes. Combined with a dual-delay deep deterministic policy gradient framework, it minimizes the temporal difference loss function, suppresses local Q-value deviations, and ensures consistency between distributed decision-making and global policy.

[0011] Finally, based on the practical needs of large-scale dynamic networks, this paper proposes an online adaptive update and elastic collaboration mechanism. By deeply integrating local agents with global federated learning, it supports dynamic optimization of routing decisions and real-time incremental updates of models. When network topology is restructured on a large scale, distributed federated collaboration enables secure aggregation of multi-node experience and model synchronization, endowing the network with the dual capabilities of dynamic self-healing and covert collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. In the drawings: Figure 1 The figure is a schematic diagram of the steps of a distributed large-scale reverse tracing elastic network intelligent routing method in one embodiment of the present invention.

[0013] Figure 2 Schematic diagram of the various component units of the GAT module of the network intelligent body in one embodiment of the present invention.

[0014] Figure 3 Schematic diagram of the network structure of the QMIX model in one embodiment of the present invention.

[0015] Figure 4 Schematic diagram of a training scenario under the CTDE framework in one embodiment of the present invention.

[0016] Figure 5 This is a diagram of the model architecture of centralized training in one embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0018] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0019] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0020] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0021] In order to solve the problems of low transmission efficiency, insufficient anonymity and imbalanced resource allocation caused by node scale expansion, path-related attack threats and suboptimal local decision-making in dynamic topology network environments, the present invention provides a distributed large-scale reverse tracing elastic network intelligent routing method and system, such as Figure 1 As shown, the method includes the following steps S101 to S104: Step S101: Construct a distributed elastic network model based on a tightly coupled proxy architecture to achieve information isolation and dynamic expansion.

[0022] Step S102: Construct a message communication mechanism based on GAT, integrate multi-dimensional feature encoding and dynamic neighbor sampling strategy, and achieve covert collaborative decision-making with low communication overhead between agents through attention weight aggregation.

[0023] Step S103: Introduce the QMIX algorithm to realize the decomposition of multi-agent value function, and perform nonlinear combination and global optimization of local Q values.

[0024] Step S104: Design a global optimization mechanism for the CTDE framework, and jointly train local and hybrid network parameters through TD error to achieve distributed back-tracing elastic routing decisions.

[0025] In step S101, in order to build a distributed network model based on a tightly coupled agent architecture, the network environment is first modeled as a graph structure. ,in Represents the set of anti-traceability terminal devices. is the set of communication links between devices, It is a set of proxies that are strictly mapped one-to-one with terminal devices.

[0026] Furthermore, each proxy only obtains local observation state through its bound terminal devices and independently selects the next-hop routing path within its local action space. This design fundamentally blocks path-association-based tracing attacks through information isolation and topology hiding, while also supporting network elastic expansion and dynamic adaptation.

[0027] Specifically, each agent Only the corresponding device can be accessed Real-time status information, including the queue depth of data to be forwarded by the device, the bandwidth utilization of adjacent links, etc. Agents do not share global topology or routing path information, and only interact with each other through local observation status and neighbor messages.

[0028] Global state space It is composed of the Cartesian product of all agent local state spaces ( ), but the agent cannot directly observe the global state and can only observe the state based on local information And the implicit features of neighboring nodes make decisions. Global action space It is defined as the Cartesian product of each agent's local action space ( ), where each agent’s local action Indicates the next-hop path selection starting from the current device.

[0029] By constructing this network model, a dual isolation mechanism is implemented: on the one hand, the agent only relies on local information to generate actions, avoiding the exposure of the global topology. Even if some nodes are controlled by attackers, the data source cannot be inferred by path backtracking; on the other hand, the Cartesian product form of the global state and action space disperses the decision-making process to each agent for independent execution, supporting the dynamic joining or exit of nodes.

[0030] In step S102, a lightweight message communication mechanism based on graph attention network is designed to achieve efficient collaborative decision-making between adjacent agents. The internally deployed GAT module consists of three functional units: lightweight feature encoding, attention weight aggregation, and dynamic neighbor sampling. Its core goal is to extract network topology features with low communication overhead and high computing efficiency in a large-scale dynamic network environment, supporting the flexibility and confidentiality of distributed routing decisions.

[0031] like Figure 2 As shown, it is an agent Schematic diagram of the various component units of the GAT module.

[0032] The lightweight feature encoding unit is responsible for converting the local network state into a vector form that can be processed by GAT. Three types of indicators are collected: queue load reflects the queue depth of the device's data packets to be forwarded, directly representing the instantaneous congestion level of the node; the adjacency table records the basic attributes of the neighboring nodes directly connected to the device and the corresponding links, used to build local topological relationships; the real-time link status dynamically captures network indicators such as bandwidth utilization, latency, and packet loss rate, characterizing the real-time communication quality of the link. These raw indicators are normalized and nonlinearly mapped through the embedding layer, and finally encoded into a feature vector of fixed dimension d. , as the input basis of GAT.

[0033] The attention weight aggregation unit is the core of the GAT module to achieve intelligent collaboration. Its design goal is to dynamically perceive the importance of neighboring nodes and fuse local states with global context to support efficient and adaptive routing decisions. In specific implementation, each agent Generate its own feature vector through lightweight feature encoding unit , and obtain a set of high-value neighbors from the dynamic neighbor sampling module The eigenvector of After normalization to eliminate dimensional differences, the agent's own and its neighbors' feature vectors are input into the shared weight matrix , projecting it into a unified latent space: Extract high-level features related to routing decisions and reduce model complexity through parameter sharing.

[0034] Then, for each pair of nodes , splicing the projected features of the agent itself and its neighbors , and use the attention parameter vector Calculate the raw attention coefficient: , where LeakyReLU is used to introduce nonlinearity to avoid gradient disappearance. Attention coefficient Reflecting on neighbors For Agents The potential importance of the decision.

[0035] Then, the neighbor set is processed by the Softmax function. The attention coefficient is normalized to obtain an interpretable attention weight: .

[0036] Finally, the agent Aggregation features Generated by weighted summation: .in, It is the ELU activation function, which is used to enhance nonlinear expression capabilities.

[0037] This process enables the agent to adaptively focus on high-value neighbors without relying on preset rules. When the bandwidth utilization of a link suddenly increases, the attention weight of its corresponding neighbors will automatically decrease, thus guiding routing decisions to avoid congested areas. It contains both local observation information and implicit global collaborative context, providing input for subsequent Q-value calculation.

[0038] The dynamic neighbor sampling unit is the key to the GAT module's adaptation to the dynamics of the elastic network. In specific implementation, each agent dynamically selects some high-value neighbor nodes (nodes connected by links with low latency, sufficient bandwidth, and light load) based on the real-time status of its bound terminal devices, and only aggregates features of these nodes. In specific implementation, the agent periodically calculates the comprehensive score of the adjacent links. ( is an adjustable weight coefficient), where Indicates bandwidth utilization. Indicates delay, Indicates the load balancing degree. The highest score is selected based on the comprehensive score. This strategy significantly reduces the resource consumption of distributed nodes while ensuring that the features of key links can be captured first.

[0039] In step S103, collaborative learning is completed through the QMIX algorithm to achieve multi-agent value function decomposition and global strategy optimization. Figure 3 Figure 2 shows a schematic diagram of the network structure of the QMIX model.

[0040] During execution, each agent Deploy an independent local Q network , the input consists of three parts: one is the hidden state generated by the GAT module , through the multi-layer graph attention network, the dynamic features of neighbor nodes are aggregated to form a contextual representation containing cross-agent collaboration information; secondly, the local observation state , local network indicators collected in real time by the terminal device bound to the agent, including device nodes The queue depth, link bandwidth utilization, current link load rate, etc.; the third is optional action , the set of next-hop routing path actions that the agent can choose. The output of the local Q network is each optional action in the current state The local Q value represents the expected value of the future cumulative reward that can be obtained by the next hop forwarding after selecting this action in the current local network state.

[0041] Furthermore, to achieve global collaborative optimization, QMIX uses a hybrid network The local Q value The nonlinear combination is the global Q value. Introducing the hypernetwork to generate a dynamic weight matrix and the bias vector , and based on the global graph state representation constructed by GAT Adjust the portfolio strategy.

[0042] Among them, the calculation form of the hybrid network is: , the weight matrix and bias Dynamically generated through the global state s. This design satisfies the monotonicity constraint ( ), that is, the global Q value With any local Q value Thus, it ensures that the local optimization direction is consistent with the global goal.

[0043] In step S104, the training is divided into three steps: distributed experience collection, centralized training, and distributed execution. The TD error is used to jointly train local and hybrid network parameters to achieve distributed back-tracing elastic routing decisions. Figure 4 The following is a schematic diagram of the training scenario under the CTDE framework.

[0044] The distributed experience collection described above breaks through the static limitations of traditional experience sampling through an adaptive annealing strategy driven by action entropy, and provides global collaborative data with high information density for subsequent centralized training.

[0045] Specifically, each agent Calculate the information entropy of its historical action distribution in real time ,quantifying the diversity of action selection.

[0046] In some embodiments, if the agent chooses the same link to forward data packets for a long time, its action distribution approaches a unimodal distribution, and the entropy value Significantly reduced, at this time the system dynamically adjusts the exploration rate , slow down If the rate of decay is too low, the agent is forced to try other links to discover potential low-latency paths. Conversely, if the agent's actions are evenly distributed, the exploration rate is rapidly reduced, prioritizing the use of verified high-Q-value paths. This mechanism, through an entropy feedback loop, achieves an autonomous balance between exploration and exploitation, making it particularly suitable for dynamic network environments with frequently fluctuating link states.

[0047] acting Execute an action Then, through the local immediate reward function Quantify the decision effect, where is the link utilization, is the queue occupancy, is the link delay. Exchange hidden state differences with neighbors through GAT's message passing mechanism , constructing enhanced local state , calculate the global graph feature vector based on aggregated neighbor messages .in, For passing The node hidden state of the layer graph attention aggregation encodes the dynamic characteristics of the neighbor link load, delay, etc. Through weighted delay information , explicitly captures the impact of link quality mutations on state representation.

[0048] The experience tuple additionally records the hidden state differences of neighbor nodes in each layer of GAT , used to perceive bandwidth utilization mutations and delay jitter caused by link quality fluctuations, dynamic changes in topology such as neighbor nodes going offline or new proxy nodes being added, and changes in neighbor queue load trends caused by traffic pattern changes.

[0049] Furthermore, the regional coordinator adopts an asynchronous parallel gradient update mechanism to receive the extended experience tuples uploaded by the agent. , build a global experience pool . Global Rewards , comprehensive local rewards , global throughput and end-to-end delay variance , forcing the policy model to prioritize path stability during training rather than simply pursuing local low latency.

[0050] In the specific implementation process of the centralized training, each agent Periodically collect local experience data, including current status , perform actions , instant rewards and the new status after transfer , and the quadruple Upload to the regional coordinator. The coordinator calculates the comprehensive reward based on the global performance indicators ,in is the end-to-end delay, is the network throughput, is the packet loss rate, weight coefficient Used to balance the priorities of different optimization goals. Figure 5The following figure shows the model architecture diagram for centralized training.

[0051] After the global reward calculation is completed, the coordinator updates the policy model through the TD error. The TD error δ is defined as: ,in is the discount factor, are the target network parameters, Based on the global state of the current hybrid network and joint actions The predicted global Q value, and The next state of the target network The maximum expected Q value of . By minimizing the mean square loss of TD error , the coordinator uses the back propagation algorithm to synchronously update the local Q network parameters of each agent and hybrid network parameters .

[0052] Furthermore, the loss function is minimized through a double-delayed deep deterministic policy gradient framework: , is the temporal difference target, is the regularization coefficient, which is used to suppress the deviation of local Q value. When the Q value of an agent is abnormally high due to local observation deviation, the regularization term By penalizing the squared magnitude of the Q-value, the model is forced to reduce its reliance on unreliable local estimates, thereby preventing the global policy from being misled by noisy data from individual agents.

[0053] Then, a parameter delay synchronization strategy is adopted, The updated local Q network parameters are Synchronizing all agents reduces the overhead of frequent communication while ensuring consistency across all agents during distributed execution. In dynamic topology scenarios, the delayed synchronization mechanism allows for multiple iterations of fine-tuning locally before synchronizing globally once parameters stabilize, thus balancing convergence speed with model stability.

[0054] Finally, to improve training stability, the target network parameters Keep frozen during training, only every Step 2: From the current hybrid network parameters Soft Update ( This mechanism effectively alleviates the moving target problem and prevents policy oscillation. Furthermore, the agent's uploaded experience data is prioritized within the global experience pool (Prioritized Experience Replay), focusing on samples with high TD error to accelerate model convergence and improve the model's adaptability to dynamic scenarios.

[0055] The distributed execution, during the implementation process, each agent Deploy local Q networks and GAT modules with frozen parameters, and rely only on local observations during execution Neighbor messages aggregated with GAT, through greedy strategy Generate routing decisions. The routing sequence of the business flow is generated by an iterative path construction algorithm. The source node agent selects the optimal next hop node based on the Q value. The relay node agent recursively executes the same decision until it reaches the destination node, forming a complete routing path. , building a low-latency, high-reliability routing sequence hop by hop.

[0056] Furthermore, the system introduces a path feedback mechanism to cope with network dynamics. When the performance indicators of the routing path exceed the preset threshold, the relevant indicators are back-propagated to the path nodes, triggering incremental fine-tuning of the local Q network. Its agent updates the local Q network parameters online based on the feedback indicator data. , adjusting the Q value of the node selected as the next hop without interrupting service or retraining the global model. This mechanism achieves local policy adaptation through lightweight gradient adjustment, significantly improving the system's responsiveness in dynamic environments.

[0057] In some embodiments, when the network topology changes dramatically, for example, when the number of nodes added or removed exceeds a preset threshold ΔN, the system starts the federated learning protocol to adapt to the new network environment. In the specific implementation steps, all proxy nodes participating in the federated learning initialize the local model parameters , ensuring the consistency of initial parameters.

[0058] Then, each agent node calculates the gradient based on local experience data During the calculation process, differential privacy technology is used to ensure data privacy by injecting noise into the gradient information. Subsequently, the gradient information is encrypted to prevent malicious nodes from stealing or tampering.

[0059] Encrypted gradient The data is transmitted to a trusted aggregation node through a secure channel. The aggregation node uses secure multi-party computing (MPC) technology to complete the gradient addition operation without decryption, thereby obtaining the global gradient. Subsequently, the aggregation node decrypts the global gradient and adjusts the global model parameters according to the preset learning rate .

[0060] Updated global model parameters The data is transmitted to all proxy nodes through a secure channel to ensure that each proxy node can obtain the latest global model parameters. After receiving the global model parameters, each proxy node updates the local model parameters. , ensuring the consistency of model parameters of all nodes.

[0061] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a distributed large-scale reverse tracing elastic network intelligent routing method.

[0062] In summary, the present invention provides a distributed large-scale anti-traceability elastic network intelligent routing method and system, including: constructing a multi-agent distributed anti-traceability elastic network model according to the node scale and security requirements in a dynamic topology network environment, mapping nodes into independent agents and limiting local information visibility to block path-related traceability risks; designing a neighbor sampling and message aggregation module based on a lightweight graph attention mechanism, dynamically screening key link status information through local topology perception, and adapting to large-scale network scenarios; introducing the QMIX multi-agent collaborative learning framework, combining local Q networks with global hybrid networks, realizing nonlinear optimization of distributed routing decisions, and simultaneously improving latency, throughput and load balancing performance; designing an action entropy-driven adaptive exploration mechanism to support rapid policy fine-tuning when the topology changes drastically, and ensuring routing service continuity; constructing a CTDE collaborative architecture to jointly optimize local and global policy parameters through TD errors to generate highly concealed and highly flexible routing sequences. The method provided by the present invention deeply integrates distributed agent decision-making, graph attention mechanism and collaborative reinforcement learning to solve the problems of path-related attack threats and suboptimal local decision-making in large-scale dynamic networks, effectively improve transmission efficiency, anonymity and resource allocation balance, and ensure the reliable operation of intelligent routing in complex anti-tracing scenarios.

[0063] It should be understood by those skilled in the art that the various exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether to implement the system in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave.

[0064] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0065] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0066] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A distributed large-scale reverse tracing elastic network intelligent routing method and system, characterized by: The method comprises the following steps: The distributed large-scale anti-provenance elastic network is a transmission network with a large number of nodes, a wide node distribution span, and a complex network topology, and it has elastic scaling and anti-provenance capabilities. The distributed large-scale anti-provenance elastic network intelligent routing method is a distributed large-scale anti-provenance elastic network intelligent routing method based on a neighbor sampling graph attention network (NSGAT) and Q-value mixed multi-agent reinforcement learning (QMIX Multi-Agent Reinforcement Learning, QMARL). It uses graph neural networks to learn hidden topological information and is suitable for elastic network environments. Neighbor sampling is used to reduce the computational burden of large-scale networks. Nodes determine the next hop forwarding of data based only on local information, without exposing the source of the data. The specific steps are as follows: Step 1: Build a multi-agent distributed anti-traceability elastic network model. Set up an agent for each network node. Agents communicate with each other through directly adjacent agents and exchange network status information. Distributed agents only know the previous and next hop information of the data and have anti-traceability capabilities. Step 2: Message communication between adjacent agents is implemented using a Graph Attention Network (GAT). This optimizes full-graph sampling to small-batch neighbor sampling centered around certain nodes through neighbor sampling. The attention mechanism is used to distinguish the importance of neighbors. GAT is adaptable to large-scale network scenarios. Step 3: Introduce the QMIX algorithm to implement collaborative learning of agent intelligent routing. QMIX improves on the Deep Q-Network (DQN) and processes the joint action value function in a multi-agent environment through value function decomposition. Each network node acts as an independent agent, calculating a local Q value based on its own state and neighbor messages obtained by GAT through a local Q network to represent its local utility for routing decisions. QMIX's hybrid network dynamically integrates the local Q values ​​of all nodes through a supernetwork. Combined with the global graph state representation constructed by GAT, it nonlinearly generates a global Q value to optimize network objectives such as end-to-end latency and throughput. Step 4: Training is divided into three steps: distributed experience collection, centralized training, and distributed execution. Each node selects actions based on an improved entropy-driven adaptive exploration mechanism, executes routing decisions, and observes the utilization of local reward links. Messages are periodically exchanged through GAT to construct an approximation of the local state s and the global state S. The hybrid network calculates the global Q value based on the global state, and optimizes the parameters of the local Q network and the hybrid network through TD error. The goal is to minimize the prediction error of the global Q value. After training is completed, each node only relies on the local Q value and neighbor messages obtained by GAT to select actions, without the participation of the hybrid network. The distributed reverse-source elastic network intelligent routing gives the best next hop for the network node, and after combination, the routing sequence of the service flow is obtained.

2. The distributed large-scale reverse tracing elastic network intelligent routing method and system according to claim 1 is characterized in that: The distributed elastic network model is constructed in step 1, specifically: A one-to-one mapping tightly coupled agent architecture is proposed to model the distributed large-scale reverse traceability elastic network environment as a graph. , Indicates the Anti-Traceability device set and represents the terminal link set, Represents the agent set, and the device set One-to-one mapping, the agent can only obtain local status through the bound device, avoiding global topology exposure and blocking tracing attacks based on path association. and Represent the global state and action space respectively. Assume that there is agents, each agent The local observation state is expressed as , then the global state space can be defined as the Cartesian product of these local states. Specifically, if each agent The local observation state From their respective local state spaces , define the global state space is the Cartesian product of the local state spaces of each agent ( ), the local observation state Contains device nodes queue depth, link bandwidth utilization, and global action space Local actions for each agent The Cartesian product of , the local action Select the next routing path.

3. The distributed large-scale reverse tracing elastic network intelligent routing method and system according to claim 1 is characterized in that: The message communication between adjacent agents in step 2 realizes cross-agent collaborative decision-making through the GAT functional component. Each agent The internally deployed GAT module includes the following core functional units: Dynamic neighbor sampling and feature extraction: Each agent extracts local information based on its observations. Each agent then shares these features with neighboring agents in the graph and combines the received neighbor features with its own observations to enhance decision-making capabilities. A dynamic neighbor random sampling strategy is proposed to avoid the traditional GAT requiring all nodes in the graph to participate in the calculation. Combining the link delay changes, bandwidth utilization, and load balancing of the network topology, only some high-value neighbors are aggregated. Each agent only extracts some high-value neighbor nodes from the distributed large-scale reverse traceability elastic network instead of processing the entire graph data, reducing the computational load of distributed nodes and adapting to the dynamic nature of large-scale networks. Through the above-mentioned lightweight feature extraction unit, the agent encodes the queue load, adjacency table, bandwidth, and delay into feature vectors as the input basis of the GAT. By combining the attention mechanism with a distributed, large-scale, and resilient back-tracing network through attention weight aggregation, the agent distinguishes the importance of different neighbors. GAT assigns higher attention weights to neighboring nodes with high link quality and low load. This process does not require preset rules, but is dynamically adjusted through learnable parameters. The agent aggregates the feature vectors of neighboring nodes according to the attention weights. The aggregated information contains the context required for global collaboration. The above design enables the GAT component to extract network topology features while reducing communication overhead. The attention mechanism enables the system to flexibly respond to the dynamic changes of distributed large-scale back-traceable elastic networks, while the modular functional units facilitate distributed deployment and expansion, ultimately supporting an efficient, hidden, and highly available elastic network.

4. The distributed large-scale reverse tracing elastic network intelligent routing method and system according to claim 1 is characterized in that: The collaborative learning mechanism of distributed elastic routing in step 3 uses the QMIX algorithm to implement multi-agent value function decomposition and global strategy optimization, specifically including the following technical features: Distributed local Q network architecture, for each agent Deploy a local Q network , whose input is the hidden state generated by the GAT module containing aggregated neighbor messages and local status representation of link indicators With optional actions ∈ , the output is the local Q value of the corresponding action, which represents the expected future reward of forwarding to the next hop in the current state of the local network; Traditional MARL uses a simple value decomposition method to add the Q values ​​of local agents to obtain the global Q value, but ignores the different contributions of different agents to the global Q value. QMIX assigns different contribution ratios to different agents, integrates the hybrid network with each local agent network, and the hybrid network Generate dynamic weight matrix With the bias vector , the local Q value of each agent The nonlinear combination is the global Q value, and the local Q value of each agent Satisfy the monotonicity constraint ( ), the specific form is ,in is the Exponential Linear Unit (ELU) activation function, and The global graph state representation constructed based on GAT Generate, where is the node-level aggregate feature; Distributed training mechanism, using centralized training-distributed execution framework, each agent in the training phase Regularly upload local experience data To the regional coordinator, the coordinator is based on the global reward , calculate the timing difference error ,in represents the end-to-end delay, represents the throughput, Represents the packet loss rate, and finally updates the local Q network parameters synchronously through back propagation and hybrid network parameters , while freezing the target network parameters To train for stability.

5. The distributed large-scale reverse tracing elastic network intelligent routing method and system according to claim 1 is characterized in that: The distributed experience collection described in step 4 includes the following steps: The traditional epsilon-greedy strategy only uses linear or exponential decay rates and cannot be dynamically adjusted according to the actual exploration effect of the node. To solve this problem, an action entropy-driven adaptive annealing mechanism is proposed. v , calculate the information entropy of its historical action distribution , each agent Based on the exploration rate Dynamic annealing adjustment based on node historical action entropy If the agent v The action exploration is sufficiently entropy is high, then it is quickly reduced , give priority to high reward paths; if the action exploration is insufficient and the entropy is low, slow down Decay speed, encouraging diverse exploration; acting Perform the selected action After that, the state is transferred to , by collecting local immediate rewards ,in Indicates link utilization, Indicates the queue occupancy rate, Represents the link delay, using the GAT message passing mechanism to is a cycle, exchanging hidden state differences with neighbor agents Message, build local state , each agent Computing global graph feature vectors based on aggregated neighbor messages ,Traditional empirical data cannot explicitly reflect the real-time state changes such as the sudden drop of the bandwidth of neighboring nodes, the joining / exit of proxy nodes, etc., ,so it is considered to add the hidden state differences of neighboring nodes in the GAT multi-layer propagation ,into the empirical tuple. To perceive the sudden changes in bandwidth utilization and delay jitter caused by link quality fluctuations, dynamic changes in topology such as neighbor nodes going offline or new proxy nodes being added, and changes in neighbor queue load trends caused by traffic pattern changes, the regional coordinator adopts an asynchronous parallel gradient update mechanism to receive the experience tuples uploaded by each proxy. , build a global experience pool , where the global reward , Representation Agent The local immediate reward, represents the global throughput, Represents the variance of end-to-end delay.

6. The distributed large-scale reverse tracing elastic network intelligent routing method and system according to claim 1 is characterized in that: The centralized training described in step 4 includes the following steps: The traditional MARL local network weights are fixed, and it only relies on the linear monotonic combination of local Q values. The local action value functions of each agent are summed up to obtain the joint action value function, ignoring the differences between different agents. The QMIX hybrid network is proposed to nonlinearly merge the local value functions of single agents. The hybrid weights of QMIX are dynamically generated by the super network, rather than the fixed scalar superposition of the traditional method. In addition, global state information is added to assist in the training and learning process. The hybrid network uses the global state to As a condition, a dynamic weight matrix is ​​generated through the hypernetwork With the bias vector , the QMIX algorithm converts the local Q value The nonlinear combination is the global Q value , and calculate the timing difference target ,in is the target hybrid network parameter; Minimizing the loss function via a dual-delayed deep deterministic policy gradient framework ,in is the regularization coefficient, which suppresses the local Q value deviation and adopts the parameter delay synchronization strategy. Step synchronization of local Q network parameters To all agents.

7. The distributed large-scale reverse tracing elastic network intelligent routing method and system according to claim 1 is characterized in that: The distributed execution in step 4 includes the following steps: Agents Deploy local Q networks and GAT modules with frozen parameters, and rely only on local observations during execution Neighbor messages aggregated with GAT, through greedy strategy Generate routing decisions. The routing sequence of the business flow is generated by an iterative path construction algorithm. The source node agent selects the optimal next hop node based on the Q value. The relay node agent recursively executes the same decision until it reaches the destination node, forming a complete routing path. ,The traditional method needs to be retrained globally when the performance degrades due to topology changes. ,This method propagates the routing delay and packet loss rate indicators back ,to the path nodes through the path feedback mechanism, triggering the ,incremental fine-tuning of the local Q network, and ,the model is updated without interrupting the service, and supports ,online adaptive updates. ,When it is detected that the number of new / removed nodes exceeds ,the number of new / removed nodes exceeds ,the number of new / removed nodes exceeds ,the number of new / removed nodes exceeding ,the number of new / removed nodes exceeding ,the number When the topology changes drastically, the federated learning protocol is started and each agent calculates the gradient based on local experience. , and update the global model after multi-party aggregation.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images