Distributed intelligent routing methods, systems, electronic devices, and storage media

Abstract

This invention relates to the field of communication technology, providing a distributed intelligent routing method, system, electronic device, and storage medium. The method includes: determining a first feature vector for each node; inputting the first feature vector of each node and the second feature vectors of its neighboring nodes into a graph neural network corresponding to each node to obtain a third feature vector for each node; wherein the nodes are connected via a communication link; inputting the third feature vector of each node, queue information, and node information of the destination node into a routing decision network corresponding to each node to obtain a forwarding interface corresponding to the action with the minimum cost; and forwarding data packets using the forwarding interface. This invention alleviates the communication pressure of consensus among numerous neural network parameters by extending graph neural networks to a distributed architecture based on communication links, and combines it with a multi-agent reinforcement learning architecture to achieve adaptive, globally optimized autonomous routing decision-making and scheduling with low communication overhead.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a distributed intelligent routing method, system, electronic device, and storage medium. Background Technology

[0002] Autonomous and intelligent network control and management are long-term goals of network protocol design. Improvements in the computing power of packet processing devices and the ability to measure network conditions such as performance and behavior have driven a new data-driven network paradigm and a transformation in protocol design. This new paradigm aims to learn from measurements and data and apply this knowledge to protocol design. In this context, machine learning, especially various deep learning algorithms, has been applied to multiple network application areas, such as routing algorithms, resource management, and traffic scheduling. Routing algorithms are a fundamental problem in networks, guiding each router to determine which interface should forward packets destined for a node. Machine learning-based routing can predict future traffic patterns by measuring past traffic, thereby optimizing route configuration.

[0003] Currently, intelligent routing mainly includes two approaches: first, intelligent routing algorithms and protocols based on reinforcement learning; and second, intelligent routing algorithms and protocols based on supervised learning. However, the first approach suffers from problems such as local and asymmetric information, which can easily lead to limited global optimization of routing decisions and routing loops. The second approach incurs high communication overhead, relies on learning and training on large amounts of labeled offline datasets, lacks real-time interaction with the environment and adaptive tuning capabilities, and is difficult to adapt to dynamic network management scenarios with large state spaces. Therefore, how to achieve intelligent routing has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a distributed intelligent routing method, system, electronic device, and storage medium to solve the problem of how to achieve intelligent routing. By extending graph neural networks to a distributed architecture based on communication links, the communication pressure of consensus of a large number of neural network parameters is alleviated. Furthermore, by combining it with a multi-agent reinforcement learning architecture, adaptive and globally optimized autonomous routing decisions and scheduling are achieved with low communication overhead.

[0005] This invention provides a distributed intelligent routing method, comprising:

[0006] A first feature vector of each node is determined, and the first feature vector of each node and the second feature vector of its neighboring nodes are input into the graph neural network corresponding to each node to obtain the third feature vector of each node; wherein, each node is connected through a communication link.

[0007] The third feature vector, queue information, and node information of each node are input into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the lowest cost.

[0008] The forwarding interface is used to forward data packets.

[0009] In one embodiment, the step of inputting the first feature vector of each node and the second feature vectors of its neighboring nodes into the graph neural network corresponding to each node to obtain the third feature vector of each node includes:

[0010] The first feature vector of each node and the second feature vector of its neighboring nodes are input into the graph neural network corresponding to each node, so that the graph neural network iteratively updates the first feature vector based on the second feature vector, and transforms the iteratively updated first feature vector into a third feature vector.

[0011] Obtain the third feature vector of each node from the graph neural network output corresponding to each node.

[0012] In one embodiment, the process of inputting the first feature vector of each node and the second feature vectors of its neighboring nodes into the graph neural network corresponding to each node to obtain the third feature vector of each node is as follows:

[0013]

[0014]

[0015] In the formula, This represents the first eigenvector of node x at time t. N represents the function used to update the feature vector of node x based on the second feature vector of its neighbor node y. x Let x represent the set of one-hop neighbor nodes of node x. This represents a function that transforms the first eigenvector after iteration into the third eigenvector, o x This represents the third eigenvector of node x.

[0016] In one embodiment, the process of inputting the third feature vector, queue information, and destination node information of each node into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the minimum cost is as follows:

[0017]

[0018] In the formula, A n Indicates the action space, a t Indicates from action space A nThe action with the lowest cost is selected, and 'a' represents the action space A. n In the action, σ represents the greed coefficient, θ n S represents the network parameters for routing decisions. n S represents the state space of a node. n Including the third feature vector, queue information, and node information of the destination node, Q n (S n ,a;θ n ) represents the state and action value function of the routing decision network.

[0019] In one embodiment, after forwarding the data packet using the forwarding interface, the method further includes:

[0020] Determine the mean square error between the target value and the output value of the routing decision network;

[0021] The parameters of the graph neural network and the routing decision network are updated based on the mean square error.

[0022] In one embodiment, the process of determining the mean square error between the target value and the output value of the routing decision network is as follows:

[0023] L t =(y i -Q n (S n ,a n |θ n ,g n )) 2

[0024] In the formula, L t y represents the mean square error. i Q represents the target value to be minimized. n (S n ,a n |θ n ,g n ) represents the output of the node's current decision action, θ n G represents the network parameters for routing decisions. n This represents the parameters of the graph neural network.

[0025] In one embodiment, the calculation process for minimizing the target value is as follows:

[0026]

[0027] In the formula, y i c represents the target value to be minimized. i Let γ represent the cost of forwarding data from node i to the peer node, and γ represent the discount factor. This represents the estimated cost of subsequent forwarding returned by the peer node to node i.

[0028] The present invention also provides a distributed intelligent routing system, comprising:

[0029] A graph neural network is used to iteratively update the first feature vector of each node based on the second feature vector of the neighboring nodes to obtain the third feature vector of each node, and send the third feature vector of each node to the environment module.

[0030] The environment module is used to receive the third feature vectors of each node sent by the graph neural network and the queue information sent by the group processing module; perform feature fusion on the third feature vectors of each node and the queue information to obtain fused information, and send the fused information to the routing decision module;

[0031] The routing decision module is used to receive the fusion information sent by the environment module, select a forwarding interface based on the fusion information, and send the decision information of selecting the forwarding interface to the packet processing module.

[0032] The packet processing module is used to receive decision information sent by the routing decision module and update the queue information based on the decision information.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the distributed intelligent routing method as described above.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the distributed intelligent routing method as described above.

[0035] The distributed intelligent routing method, system, electronic device, and storage medium provided by this invention determine the first feature vector of each node, and input the first feature vector of each node and the second feature vector of its neighboring nodes into the graph neural network corresponding to each node to obtain the third feature vector of each node; wherein, each node is connected through a communication link; the third feature vector of each node, queue information, and the node information of the destination node are input into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the minimum cost; and the forwarding interface is used to forward data packets. This invention alleviates the communication pressure of consensus of a large number of neural network parameters by extending the graph neural network to a distributed architecture based on communication links, and combines it with a multi-agent reinforcement learning architecture to achieve adaptive, globally optimized autonomous routing decision-making and scheduling with low communication overhead. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating the distributed intelligent routing method provided by the present invention;

[0038] Figure 2 This is a schematic diagram of the network topology provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the structure of the distributed intelligent routing system provided by the present invention;

[0040] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0042] The following is combined Figures 1-4 The present invention describes a distributed intelligent routing method, system, electronic device, and storage medium.

[0043] Specifically, this invention provides a distributed intelligent routing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the distributed intelligent routing method provided by the present invention.

[0044] The distributed intelligent routing method provided in this embodiment of the invention includes:

[0045] Step 100: Determine the first feature vector of each node, and input the first feature vector of each node and the second feature vector of its neighboring nodes into the graph neural network corresponding to each node to obtain the third feature vector of each node; wherein, each node is connected through a communication link.

[0046] It should be noted that the main function of GNN (Graph Neural Network) is to fully explore and model the global features of the entire network, thereby bridging the performance gap between distributed local decision-making and global decision-making.

[0047] The network topology can be mapped to a graph, where each node corresponds to a router in the network, and each edge corresponds to a communication link between routers. Each node x initializes its characteristics to i. x Understandably, for traditional routing protocols, each router in the topology is assigned an integer identifier R. x In this embodiment of the invention, this identifier is used as the initial feature i x .

[0048] GNN is an iterative process of passing and fusing messages between neighboring nodes. Each node iteratively interacts with its neighbors and updates its feature representation to obtain the final third feature vector o. x Specifically, the first feature vector of each node and the second feature vector of its neighboring nodes are input into the graph neural network corresponding to each node, so that the graph neural network iteratively updates the first feature vector based on the second feature vector, and transforms the iteratively updated first feature vector into a third feature vector. Then, the third feature vector of each node is obtained from the output of the graph neural network corresponding to each node.

[0049] For example, each node x sets its initial feature i x The mapping is done using a hidden representation, which can be understood as a feature representation, i.e., a feature vector. Each node x updates its own hidden representation (first feature vector) based on the hidden representations (second feature vectors) of its single-hop neighbors during the iteration process, until a stable equilibrium state is reached. A stable equilibrium state is defined as the difference between two consecutive updated hidden representations being less than a certain threshold. Finally, the hidden representation (first feature vector) updated through multiple iterations is transformed into a third feature vector o. x The third feature vector is an abstract representation of the global characteristics of the network, which is used for subsequent routing decisions.

[0050] The mathematical model represents the above process as follows:

[0051]

[0052] In the above formula, This represents the first eigenvector of node x at time t. N represents the function used to update the feature vector of node x based on the second feature vector of its neighbor node y. x Let x represent the set of one-hop neighbor nodes of node x. This represents a function that transforms the first eigenvector after iteration into the third eigenvector, o x Represents the third feature vector of node x, init(i x ) represents a function that initializes the hidden representation based on the input characteristics.

[0053] It should be noted that, in the embodiments of the present invention, init(i x It is implemented using one-hot encoding. and It is implemented through a neural network, and the neuron parameters of each node are different.

[0054] In this embodiment of the invention, the GNN is spatially distributed across every node in the entire network, and a portion of node x is represented by G-netx, such as... Figure 3 As shown, the various G-netx nodes are connected via communication links, and all the G-netx nodes together form a unified GNN. The G-netx nodes... and Both are implemented using feedforward neural networks, which have the same structure, but the parameters differ for different nodes.

[0055] Step 200: Input the third feature vector, queue information and destination node information of each node into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the minimum cost.

[0056] It should be noted that in multi-agent reinforcement learning, all nodes are equivalent and have the same neural network structure. Each node can be viewed as an independent agent that learns and makes decisions by observing the state of its local network and communicating with its neighbors. Therefore, this embodiment of the invention uses any node n in the network as an example to describe the definition of each element in reinforcement learning.

[0057] State space: Assume the message currently being processed is message p. The state space of agent n is defined as follows:

[0058] S n :{d p ,o n ,K n}

[0059] In the formula, d p Indicates the destination address of message p, o n K represents the information observed by the node itself. n It's a message sent by a neighbor. Among them, o n It is information related to the time it takes to forward to each neighbor node, K nThis refers to the information possessed by agent n that relates to the characteristics of the global network. As the network topology and traffic change, the state of the nodes also changes continuously.

[0060] Action Space: In the message routing problem, the action space of agent n is mainly determined by the set of neighboring nodes. Let A be the action space of agent n. n :W n Among them, W n Let S represent the set of neighboring nodes of node n. The action space of each agent n has the same dimension as its set of neighboring nodes. For each message to be processed, agent n first obtains its current state S. n Then from action space A n Choose the optimal action and forward the message to the corresponding neighbor node.

[0061] Cost: In this embodiment of the invention, time cost is used to guide action selection, with the average delivery time being the minimum cost. The overhead of forwarding packet p is defined as the sum of queuing time and transmission delay, c = q + h, where q represents the queuing time of agent n and h represents the transmission delay from the next-hop node to the destination node.

[0062] Q-net consists of three fully connected layers, with the output of G-net serving as its input. The input to Q-net mainly comprises three parts:

[0063] (1) Current destination: The destination node is used to determine the action space of the current message.

[0064] (2) Queue length: The queue length of each interface is used to characterize the congestion level of each neighboring node.

[0065] (3) Feature vector: Use its own feature vector to capture and characterize global network features, and update it periodically by G-net based on neighbor feature vectors.

[0066] It should be noted that the above information needs to be one-hot encoded before it can be input into Q-net. For example, if the destination node of the current data packet is node 2, the one-hot encoded result of the first part of the input information is [0010000]. The length of a one-hot vector is usually fixed at an upper limit to ensure compatibility.

[0067] To avoid local optima, this embodiment of the invention introduces a greedy strategy. At each decision time t, if a data packet p arrives at the head of the queue of agent n, agent n first obtains the state information S. n This includes its own queue information. n The feature vector K obtained from communicating with neighboring nodes nThe data packet p has two sources: one is sent by the application layer of agent n, and the other is forwarded by agent n's neighbor nodes. Then, agent n follows the σ-greedy policy from action space A. n Select action a t As shown below:

[0068]

[0069] In the formula, A n Indicates the action space, a t Indicates from action space A n The action with the lowest cost is selected, and 'a' represents the action space A. n In the action, σ represents the greed coefficient, θ n S represents the network parameters for routing decisions. n S represents the state space of a node. n Including the third feature vector, queue information, and node information of the destination node, Q n (S n ,a;θ n ) represents the state and action value function of the routing decision network.

[0070] It is understandable that agent n in its action space A n In this case, an action is randomly selected with probability σ, or the action that minimizes the value of q is selected with probability 1-σ.

[0071] Step 300: Forward the data packet using the forwarding interface.

[0072] After determining the forwarding interface, use that forwarding interface to forward data packets.

[0073] The distributed intelligent routing method provided in this invention determines the first feature vector of each node, and inputs the first feature vector of each node and the second feature vectors of its neighboring nodes into the graph neural network corresponding to each node to obtain the third feature vector of each node; wherein, each node is connected through a communication link; the third feature vector of each node, queue information, and node information of the destination node are input into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the minimum cost; and the forwarding interface is used to forward data packets. This invention alleviates the communication pressure of consensus of a large number of neural network parameters by extending the graph neural network to a distributed architecture based on communication links, and combines it with a multi-agent reinforcement learning architecture to achieve adaptive, globally optimized autonomous routing decision and scheduling with low communication overhead.

[0074] Based on the above embodiments, after forwarding data packets using the forwarding interface, the method further includes: determining the mean square error between the target value and the output value of the routing decision network; and updating the parameters of the graph neural network and the routing decision network based on the mean square error.

[0075] It should be noted that, based on Q-routing, this embodiment of the invention integrates multi-agent reinforcement learning and GNN, and further designs a learning algorithm for distributed intelligent routing decision-making.

[0076] The objective function of the learning algorithm is as follows:

[0077]

[0078] In the formula, y i c represents the target value to be minimized. i Let γ represent the cost of forwarding data from node i to the peer node, and γ represent the discount factor. This represents the estimated cost of subsequent forwarding returned by the peer node to node i.

[0079] By minimizing the target value y i The mean squared error between the current Q-net output and the current Q-net output is used to update the parameters of the neural network. The error function is as follows:

[0080] L t =(y i -Q n (S n ,a n |θ n ,g n )) 2

[0081] In the formula, L t y represents the mean square error. i Q represents the target value to be minimized. n (S n ,a n |θ n ,g n ) represents the output of the node's current decision action, θ n G represents the network parameters for routing decisions. n This represents the parameters of the graph neural network.

[0082] After receiving the estimated subsequent forwarding time from the next-hop node, node n records the decision made in this instance. Save to experience replay pool D n Among them After each decision-making C times, node n randomly selects X samples from its own experience replay pool, calculates the mean square error based on the aforementioned error function, and uses the gradient descent algorithm to determine the routing decision network parameters θ. n And graph neural network parameters g n Adjustments.

[0083] This invention improves routing performance by determining the mean squared error between the target value and the output value of the routing decision network, and then updating the parameters of the graph neural network and the routing decision network based on this mean squared error. Updating the parameters of the graph neural network and the routing decision network improves the effectiveness of the network output, thereby enhancing routing performance.

[0084] Based on the above embodiments, this embodiment of the invention will analyze and explain the distributed intelligent routing method using a specific example.

[0085] refer to Figure 2 , Figure 2 This is a network topology diagram provided by the present invention, including 23 nodes and 37 links.

[0086] Adopting based on Figure 2 The network topology is monitored and statistically analyzed using a publicly available traffic matrix dataset. This dataset provides the communication rate between any two nodes at the sampling time. In this embodiment of the invention, the communication rate is proportionally converted into the number of data packets. The steps for implementing distributed intelligent routing are as follows:

[0087] 1. Initialize the network topology, including initializing the neural network model, feature vectors, cost function, and loss function.

[0088] 2. Loop through Epochs, switching traffic data files sequentially and updating the initial packet list and generation time for each node. An Epoch is defined as the process of a complete dataset passing through the neural network once and returning once. Essentially, an Epoch represents the entire dataset undergoing one computation within the neural network.

[0089] 3. During the loop time t, each node performs the following operations sequentially:

[0090] (1) Interact with the neighboring nodes using the feature vector of the user, and use the received feature vector of the neighboring nodes and the user's own feature vector as the input of the graph neural network. Use the output of the graph neural network as the updated feature vector of the user. Iterate the above process K times.

[0091] (2) Traverse the data packet dictionary, which contains all data packet sequence numbers and their arrival times. If there is a data packet with a corresponding time, determine its forwarding interface and add it to the corresponding forwarding queue.

[0092] (3) For each forwarding interface, forward the head data packet in each interface queue;

[0093] (4) Check if there are any data packets forwarded by neighboring nodes. If so, perform a decision operation and add them to the corresponding forwarding queue.

[0094] The forwarding interface decision and learning process in step 3 is as follows:

[0095] (1) Each node obtains its own feature vector and encodes it together with the destination node number as the input to the routing decision network;

[0096] (2) The routing decision network outputs the cost value corresponding to each action and selects the action with the smallest cost value as the forwarding interface.

[0097] (3) Return the cost corresponding to the decided forwarding interface to the previous node;

[0098] (4) The previous node stores the state vector, destination node number, instantaneous cost and cost value used for its decision into the memory bank;

[0099] (5) After each node makes a decision C times, it performs a learning process to update the parameters of the graph neural network and the routing decision network.

[0100] 4. When a data packet arrives at its destination node, the forwarding time used is statistically accumulated; after all data packets have arrived at their destination, the average forwarding time used is calculated.

[0101] 5. Every E epochs, the average forwarding times within that segment are summed up as a reference point.

[0102] After completing the above operations, if the average forwarding time decreases as the Epoch increases, it indicates that the expected result is met.

[0103] This invention extends graph neural networks to a distributed architecture based on communication links, alleviating the communication pressure of consensus among a large number of neural network parameters. It also combines graph neural networks with a multi-agent reinforcement learning architecture to achieve adaptive, globally optimized autonomous routing decisions and scheduling with low communication overhead.

[0104] Based on the above embodiments, this invention proposes a distributed intelligent routing system, with reference to... Figure 3 , Figure 3 This is a schematic diagram of the structure of the distributed intelligent routing system provided by the present invention.

[0105] The distributed intelligent routing system of this invention includes:

[0106] A graph neural network is used to iteratively update the first feature vector of each node based on the second feature vector of the neighboring nodes to obtain the third feature vector of each node, and send the third feature vector of each node to the environment module.

[0107] The environment module is used to receive the third feature vectors of each node sent by the graph neural network and the queue information sent by the group processing module; perform feature fusion on the third feature vectors of each node and the queue information to obtain fused information, and send the fused information to the routing decision module;

[0108] The routing decision module is used to receive the fusion information sent by the environment module, select a forwarding interface based on the fusion information, and send the decision information of selecting the forwarding interface to the packet processing module.

[0109] The packet processing module is used to receive decision information sent by the routing decision module and update the queue information based on the decision information.

[0110] It should be noted that, Figure 3 Solid lines represent logical connections between modules, while dashed lines represent communication links between nodes.

[0111] like Figure 3 As shown, this embodiment of the invention uses node 0 and node 1 as examples to describe a distributed intelligent routing system. Figure 3 In this diagram, a portion of node 0 is represented by G-net 0, and a portion of node 1 is represented by G-net 1. Each node corresponds to a graph neural network (G-net), and the G-nets of different nodes are connected via communication links. All the G-nets of all nodes together constitute a complete graph neural network (GNN). Environment modules are represented by Env, such as Env 0 and Env 1; routing decision modules are represented by Q-net, such as Q-net 0 and Q-net 1; and packet processing modules are represented by Queue modules, such as Queue module0 and Queue module 1.

[0112] The following is a breakdown of each component in the distributed intelligent routing system:

[0113] The graph neural network G-net is used to iteratively update the first feature vector of each node based on the second feature vector of the neighboring nodes, to obtain the third feature vector of each node, and then send the third feature vector of each node to the environment module.

[0114] The environment module Env receives the third feature vectors of each node sent by the graph neural network and the queue information sent by the group processing module; it performs feature fusion on the third feature vectors of each node and the queue information to obtain fused information, and sends the fused information to the routing decision module.

[0115] The routing decision module Q-net receives fusion information sent by the environment module, selects a forwarding interface based on the fusion information, and sends the decision information of the selected forwarding interface to the packet processing module.

[0116] The Queue module receives decision information from the routing decision module and updates the queue information based on this information. For example, it adds corresponding queues based on the decision information and forwards packets according to the queue order.

[0117] The distributed intelligent routing system provided in this invention alleviates the communication pressure of consensus of a large number of neural network parameters by extending graph neural networks to a distributed architecture based on communication links, and combines it with a multi-agent reinforcement learning architecture to achieve adaptive, globally optimized autonomous routing decision-making and scheduling with low communication overhead.

[0118] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a distributed intelligent routing method, which includes:

[0119] A first feature vector of each node is determined, and the first feature vector of each node and the second feature vector of its neighboring nodes are input into the graph neural network corresponding to each node to obtain the third feature vector of each node; wherein, each node is connected through a communication link.

[0120] The third feature vector, queue information, and node information of each node are input into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the lowest cost.

[0121] The forwarding interface is used to forward data packets.

[0122] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the distributed intelligent routing method provided by the methods described above, the method comprising:

[0124] A first feature vector of each node is determined, and the first feature vector of each node and the second feature vector of its neighboring nodes are input into the graph neural network corresponding to each node to obtain the third feature vector of each node; wherein, each node is connected through a communication link.

[0125] The third feature vector, queue information, and node information of each node are input into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the lowest cost.

[0126] The forwarding interface is used to forward data packets.

[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed intelligent routing method, characterized in that, include: The first feature vector of each node is determined, and the first feature vector of each node and the second feature vector of its neighboring nodes are input into the graph neural network corresponding to each node to obtain the third feature vector of each node; wherein, the graph neural networks of each node are connected through communication links, that is, the graph neural network is extended to a distributed architecture based on communication links, and all nodes together constitute a whole graph neural network. The third feature vector, queue information, and node information of each node are input into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the lowest cost. The data packets are forwarded using the aforementioned forwarding interface; The process of inputting the first feature vector of each node and the second feature vector of its neighboring nodes into the graph neural network corresponding to each node to obtain the third feature vector of each node is as follows: ； ； In the formula, express t Time Node The first eigenvector, Indicates the use of neighbor nodes Second feature vector update node A function of the eigenvectors, Represents a node The set of one-hop neighbor nodes, This represents a function that transforms the first eigenvector after iteration into the third eigenvector. Represents a node The third eigenvector; After forwarding the data packets using the forwarding interface, the method further includes: Determine the mean square error between the target value and the output value of the routing decision network; The parameters of the graph neural network and the routing decision network are updated based on the mean square error; The process of determining the mean square error between the target value and the output value of the routing decision network is as follows: ； In the formula, Indicates mean square error. This indicates minimizing the target value. This represents the output of the node's current decision action. Indicates the network parameters for routing decisions. Represents the parameters of a graph neural network; The calculation process for minimizing the target value is as follows: ； In the formula, This indicates minimizing the target value. Represents a node The cost of forwarding to the peer node. Indicates the discount factor. This indicates that the peer node returns the information to the node. The estimated cost of subsequent forwarding; After receiving the forwarding time estimate returned by the next-hop node, the node n Record of this decision Save to experience replay pool D n Among them ; Each decision to be implemented C After that, the node n Randomly draw from one's own experience replay pool X Each sample is used to calculate the mean square error based on the error function, and the gradient descent algorithm is employed to determine the network parameters for routing decisions. Graph Neural Network Parameters Adjustments.

2. The distributed intelligent routing method according to claim 1, characterized in that, The step of inputting the first feature vector of each node and the second feature vector of its neighboring nodes into the graph neural network corresponding to each node to obtain the third feature vector of each node includes: The first feature vector of each node and the second feature vector of its neighboring nodes are input into the graph neural network corresponding to each node, so that the graph neural network iteratively updates the first feature vector based on the second feature vector, and transforms the iteratively updated first feature vector into a third feature vector. Obtain the third feature vector of each node from the graph neural network output corresponding to each node.

3. The distributed intelligent routing method according to claim 1, characterized in that, The process of inputting the third feature vector, queue information, and destination node information of each node into the routing decision network corresponding to each node to obtain the forwarding interface corresponding to the action with the minimum cost is as follows: ； In the formula, Indicates the space for action. Indicates from the action space The action with the lowest cost is selected. Indicates action space The actions in Represents the greed coefficient. Indicates the network parameters for routing decisions. Represents the state space of a node. This includes the third feature vector, queue information, and node information of the destination node. This represents the state and action value function of the routing decision network.

4. A distributed intelligent routing system, characterized in that, The distributed intelligent routing method applied to any one of claims 1 to 3 includes: A graph neural network is used to iteratively update the first feature vector of each node based on the second feature vector of the neighboring nodes to obtain the third feature vector of each node, and send the third feature vector of each node to the environment module. The environment module is used to receive the third feature vectors of each node sent by the graph neural network and the queue information sent by the group processing module; perform feature fusion on the third feature vectors of each node and the queue information to obtain fused information, and send the fused information to the routing decision module; The routing decision module is used to receive the fusion information sent by the environment module, select a forwarding interface based on the fusion information, and send the decision information of selecting the forwarding interface to the packet processing module. The packet processing module is used to receive decision information sent by the routing decision module and update the queue information based on the decision information.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the distributed intelligent routing method as described in any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the distributed intelligent routing method as described in any one of claims 1 to 3.

Images