Intelligent routing method, device and equipment based on reinforcement learning

Abstract

This application discloses an intelligent routing method, apparatus, and device based on reinforcement learning. It includes: constructing a communication network model graph, which comprises multiple network nodes, each of which is an agent; at any given time, each network node acquires network state information obtained after interacting with the environment; inputting the network state information into the agent's policy network, and outputting the action with the highest probability; determining the next network node for data packet routing based on the output action information, and sending the data packet to the selected next network node. This application employs an adaptive multi-agent reinforcement learning algorithm, where each routing node makes routing decisions based on the current network state without needing to build a routing table. Compared with other algorithms, this achieves lower average data packet transmission latency and improves network routing efficiency.

Description

Technical Field

[0001] This application relates to the field of network routing technology, and more specifically, to an intelligent routing method, apparatus, and device based on reinforcement learning. Background Technology

[0002] Routing is the process of selecting transmission paths for data packets in a network, and it is a core component of network communication. The continuous expansion of network scale and the increasing complexity of network conditions have brought about challenges in data packet routing, including efficiency and reliability issues. Therefore, how to improve data packet routing efficiency and reduce transmission latency has always been a major focus of researchers.

[0003] Current dynamic routing algorithms typically calculate the optimal path to a destination based on certain optimization criteria, such as minimum hop count or minimum cost. Furthermore, to adapt to changes in network conditions and calculate the optimal route, they often need to periodically exchange local databases with surrounding or even more distant nodes, generating bandwidth pressure. Additionally, the routing strategies lack timeliness. These algorithms generally suffer from two main problems: 1) Limitations in network state awareness: In large networks, due to the large number and wide distribution of nodes, it is difficult for a single node to fully perceive the real-time state of the entire network, including path congestion; 2) Uniqueness of path selection: Since data packets select paths based on the same rules, a surge in one type of traffic may overload one path while leaving nodes on other paths idle. Summary of the Invention

[0004] This application provides an intelligent routing method, apparatus, and device based on reinforcement learning, which at least solves the technical problems of low routing efficiency and high transmission latency in related technologies.

[0005] According to one aspect of the embodiments of this application, a reinforcement learning-based intelligent routing method is provided, comprising:

[0006] Construct a communication network model diagram, which includes multiple network nodes, where each network node is an intelligent agent;

[0007] At any given time, each network node obtains the network state information obtained after interacting with the environment.

[0008] The network state information is input into the agent's policy network, and the action with the highest probability is output.

[0009] Based on the output action information, determine the next network node for packet routing and send the packet to the selected next network node.

[0010] In an optional embodiment, the network state information includes the originating node of the data packet, the destination node of the data packet, the historical actions of the current network node, and the number of data packets currently queued by adjacent network nodes.

[0011] In an optional embodiment, after each network node obtains the network state information obtained from its current interaction with the environment, the method further includes:

[0012] Get the set of all neighboring network nodes of the current agent's network node;

[0013] The action space of the agent is generated based on the set of all adjacent network nodes.

[0014] In an optional embodiment, before inputting the network state information into the agent's policy network, the method further includes:

[0015] Construct a network model of the intelligent agent;

[0016] The network model of the agent includes a policy network and an evaluation network;

[0017] The policy network consists of an input layer, a hidden layer, and an output layer, and is used to decide which action should be taken in the current network state.

[0018] The evaluation network consists of an input layer, a hidden layer, and an output layer, and is used to evaluate the value of actions under the current policy.

[0019] In an optional embodiment, after determining the next network node for packet routing based on the output action information, the method further includes:

[0020] Determine whether the next network node in the data packet routing is the target node;

[0021] If it is not the target node, record the number of hops, transmission cost, queuing delay, and state value of the data packet at that network node;

[0022] If it is the target node, calculate the reward value of each node along the entire transmission path from the starting node to the target node.

[0023] In an optional embodiment, calculating the reward value of a node includes:

[0024] The reward value of the current node is obtained by subtracting the transmission cost and the queuing delay of the current node from the reward value of the next hop node.

[0025] In an optional embodiment, calculating the state value of a network node includes:

[0026] The network state information is input into the agent's evaluation network to obtain the state value under the current policy.

[0027] In an optional embodiment, it further includes:

[0028] Based on the state value and reward value, calculate the loss value of the evaluation network and policy network for any node;

[0029] The evaluation network and policy network are updated based on their loss values.

[0030] According to another aspect of the embodiments of this application, a reinforcement learning-based intelligent routing device is also provided, comprising:

[0031] A network graph construction module is used to construct a communication network model graph, which includes multiple network nodes, wherein each network node is an agent;

[0032] The state space acquisition module is used to enable each network node to acquire the network state information obtained after interacting with the environment at any given time.

[0033] The decision-making module is used to input the network state information into the policy network of the agent and output the action with the highest probability.

[0034] The execution module is used to determine the next network node for packet routing based on the output action information and send the packet to the selected next network node.

[0035] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described reinforcement learning-based intelligent routing method through the computer program.

[0036] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0037] This application provides a reinforcement learning-based intelligent routing method that treats each network node as an independent agent and makes routing decisions through a separate model, providing the optimal action based on the current real-time network state. This enables the agent to dynamically adjust routing decisions according to the current network state, achieving load balancing, avoiding overload on a single path, and improving routing flexibility. It effectively reduces the average latency of data packets transmitted in the network, meeting the needs of applications with high real-time requirements and improving data packet routing efficiency. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0039] Figure 1 This is a flowchart of an intelligent routing method based on reinforcement learning according to an embodiment of this application;

[0040] Figure 2 This is a communication network model diagram according to an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of a reinforcement learning-based intelligent routing method according to an embodiment of this application;

[0042] Figure 4 This is a simulated network node diagram according to an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of a simulation result according to an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of a simulation result according to an embodiment of this application;

[0045] Figure 7 This is a schematic diagram of a reinforcement learning-based intelligent routing device according to an embodiment of this application;

[0046] Figure 8 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] This application proposes a low-latency routing algorithm based on multi-agent reinforcement learning, aiming to minimize transmission latency between nodes and learn the optimal routing strategy. In this scheme, each node is treated as an independent agent, and routing decisions are made through a separate model. The proposed scheme considers not only the characteristics of data packets during training but also the current state of neighboring nodes (data packet queuing, etc.). Fully considering the information of neighboring nodes enhances the agent's network awareness, enabling prediction of network congestion. Furthermore, it allows the agent to dynamically adjust routing decisions based on the current network state, achieving load balancing, avoiding overload on a single path, and improving routing flexibility.

[0050] The following is in conjunction with the appendix Figure 1 This application provides a detailed description of the reinforcement learning-based intelligent routing method according to its embodiments. It should be noted that the following application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the implementation methods of this application are not limited in any way. On the contrary, the implementation methods of this application can be applied to any applicable scenario. Figure 1 As shown, the method mainly includes the following steps:

[0051] S101 constructs a communication network model diagram, which includes multiple network nodes, each of which is an intelligent agent.

[0052] like Figure 2 As shown, construct a communication network model diagram. The communication network consists of multiple network nodes interconnected by links, and each node can forward and receive data packets. Let the undirected graph G = (V, E) represent the topology of the communication network, where V and E are the sets of router nodes and the sets of links between nodes, respectively. Let the data transmission rate of node i be C. i .

[0053] A data packet is defined as a triple (s, d, B), where s represents the originating node of the packet in the network, d represents the destination node of the packet routing, and B is the packet size in bytes. The transmission time of a data packet of size B at node i can be expressed as...

[0054] Treat each node in the network as an intelligent agent.

[0055] Depend on Figure 2As we know, the task of packet routing is to transmit each data packet from its origin node to its destination node through relay by multiple routers. It mainly consists of three parts: data packet arrival at the origin node, data packet queuing, and data packet transmission. The queues within the nodes follow a first-in, first-out (FIFO) principle. The packet routing problem is to find the optimal transmission path between the source and destination nodes. Each router continuously passes the data packet at the head of its queue to its neighboring nodes until the data packet reaches its destination.

[0056] The objective of this application is to design a routing algorithm that minimizes the average transmission delay of all data packets from the source node to the destination node. Let P be the set of all data packets to be transmitted, and t p Let be the total transmission delay of data packet p from its origin to its destination node. Based on this, the task of minimizing the average transmission delay of the data packet can be expressed as:

[0057]

[0058] Where K is the size of the data packet set P, that is, the total number of all data packets.

[0059] S102 At any given time, each network node obtains the network status information obtained after interacting with the environment.

[0060] The network status information includes the originating node of the data packet, the destination node of the data packet, the historical actions of the current network node, and the number of data packets currently queued by adjacent network nodes.

[0061] In environments where network states are constantly changing, it is crucial to ensure that agents can effectively observe the network information they need. The state space of each node is defined as a four-tuple, including the origin of the data packet, the destination node of the data packet, the three historical actions of this node, and the number of data packets currently queued by neighboring nodes.

[0062] Focusing solely on packet attributes, such as originating node, destination node, and packet size, can overlook the complexity and variability of the network environment, thus presenting limitations. Therefore, this application considers incorporating metrics reflecting the immediate congestion status of the network as part of the input, such as the number of packets currently queued at neighboring nodes, to improve the flexibility of routing decisions.

[0063] Furthermore, after each network node obtains the network state information obtained from its current interaction with the environment, it also includes: obtaining the set of all neighboring network nodes of the network node where the current agent is located; and generating the agent's action space based on the set of all neighboring network nodes.

[0064] Understandably, the action space is defined as the set of neighboring nodes of node i. Accordingly, for each agent, the size of the action space is equal to the number of its neighboring nodes.

[0065] S103 inputs the network state information into the agent's policy network and outputs the action with the highest probability.

[0066] First, an agent network model is constructed. The agent network model includes a policy network and an evaluation network. The policy network, consisting of an input layer, hidden layers, and an output layer, is used to decide which action to take in the current network state. The evaluation network, consisting of an input layer, hidden layers, and an output layer, is used to evaluate the value of actions under the current policy.

[0067] Specifically, each agent adopts the Actor-Critic framework, which consists of two networks: an Actor policy network responsible for generating actions, and a Critic evaluation network responsible for assessing the value of the current state. Both the Actor and Critic networks use the same neural network structure. The input layer takes the state obtained after interacting with the environment as input, followed by a hidden layer containing 256 neurons. The output of this hidden layer is processed by a softmax function to obtain the probability of each selectable action. The agent then makes corresponding routing decisions based on these probabilities.

[0068] Furthermore, action decisions are made based on a policy network of agents. Each node (agent) in the network learns and routes independently. At any time t, each node observes the current network state, inputs the current state into the agent's policy network based on the current state representation, and outputs the action with the highest probability, which determines the next-hop node for the data packet route. This can be represented as:

[0069] a t =argmax a Q(θ q );

[0070] Where Q is the policy neural network, θ q For the parameters of the network, a t This is the output of the policy network in the current state.

[0071] S104 determines the next network node for packet routing based on the output action information and sends the packet to the selected next network node.

[0072] In one implementation, the next-hop node for the data packet's route is determined based on the action with the highest output probability. After the next-hop node is determined according to the policy network, the data packet is sent through the selected path.

[0073] In an optional embodiment, after determining the next network node for packet routing based on the output action information, the method further includes:

[0074] Determine whether the next network node in the packet routing is the target node; if it is not the target node, record the packet's hop count, transmission cost, queuing delay, and state value at that network node; if it is the target node, calculate the reward value of each node along the entire transmission path from the starting node to the target node.

[0075] The reward is defined to encourage agents to choose paths that reduce packet transmission latency and queuing time, while minimizing the number of packet transmissions, thereby improving overall network performance. For any node i, the reward r is defined in this application. i The reward of the next hop node minus the transmission cost minus the queuing delay of that node can be specifically expressed as:

[0076] r i =γ·r i6+1 -q i+1 -cost

[0077] Where γ is the attenuation factor, which is 0.9 in an exemplary embodiment, cost is the transmission cost, and q i+1 This represents the queuing delay at the next node. i r is the reward for the current node. i+1 The reward for the next hop node is calculated. The transmission cost is set to a constant, indicating that transmitting data packets has a cost, and the agent will try to reduce the cost during training, thereby minimizing the number of data packet transmissions.

[0078] The calculation of the state value of network nodes includes: inputting network state information into the agent's evaluation network to obtain the state value under the current policy; and outputting the action value under the current policy based on the evaluation network.

[0079] Furthermore, based on the state value and reward value, the loss values ​​of the evaluation network and policy network for any node are calculated; the evaluation network and policy network are then updated based on the loss values ​​of the evaluation network and policy network.

[0080] In one embodiment of this application, the loss values ​​of the evaluation network and the policy network for any node i are calculated as follows:

[0081]

[0082] Among them, L critic L represents the loss value used to evaluate the network. actor Represents the loss value of the policy network; J is the set of all data packets passing through node i, Q(θ) qV(θ) represents the reward value at the node. π ) represents the state value of a node, π * This represents the optimal policy. The policy network loss means that when the TD error value is greater than 0, the network is trained to increase the probability of selecting that action; otherwise, the probability of selecting that action is decreased.

[0083] Furthermore, backpropagation is performed to update the model.

[0084]

[0085] Where α1 and α2 are the learning rates of the policy network and the evaluation network, respectively.

[0086] Through the above update method, the agent updates the policy network to optimize routing decisions and updates the value network to better evaluate the value of the current state, thereby obtaining better performance thereafter, and continues to iterate until convergence or the number of iterations is reached.

[0087] This application presents a routing scheme that provides a multi-agent joint routing framework based on deep reinforcement learning. By combining information from multiple agents, each agent dynamically makes routing decisions based on real-time network conditions, ensuring high reliability and low latency in data packet transmission. The signaling content between nodes in the multi-agent interaction includes the node's unique identifier and the number of data packets currently queued by neighboring nodes. This reduces signaling overhead, enabling nodes to perceive network conditions and supporting efficient routing decisions. In this application, the reinforcement learning reward is defined as the reward of the next node minus the transmission cost and the node's queuing time. This definition aims to reduce data packet transmission latency and also decrease the number of data packet transmissions.

[0088] To facilitate understanding of the solutions provided in the embodiments of this application, the following is in conjunction with the appendix. Figure 3 Further description.

[0089] like Figure 3 As shown, each node in the network is considered an agent, and for each agent, there is a policy network and an evaluation network.

[0090] First, obtain the environment state. Retrieve the attributes of the currently pending data packet and request the number of currently queued data packets from neighboring nodes. Input the acquired network state information into the policy network and output an action. Determine the next-hop node for the data packet based on the output action. Input the acquired environment state into the evaluation network to obtain the value of the current action.

[0091] Furthermore, the reward is calculated, and the TD error is calculated based on the reward and value output. The policy network and value network are then updated based on the error loss. Network optimization is performed. These steps are iterated until convergence, improving the network's routing performance.

[0092] like Figure 4 The simulation shows that a network topology with 9 nodes was randomly generated using a Python simulation environment. Figure 5 Simulation results show that, over 300 iterations, the cumulative rewards of all intelligent nodes exhibit an overall decreasing trend, indicating that the latency and number of transmission iterations are effectively reduced. The proposed solution helps improve the overall network transmission efficiency and performance.

[0093] Figure 6 Simulation results compare the proposed scheme with the cumulative distribution function (CDF) of the classic OSPF algorithm. It can be seen that the proposed scheme has a lower average packet transmission latency, with over 80% of packets arriving at the target node within 5ms. This is because the proposed scheme has a more comprehensive understanding of network conditions and avoids the limitation of single-path selection, thus achieving load balancing.

[0094] According to another aspect of the embodiments of this application, a reinforcement learning-based intelligent routing apparatus for implementing the above-described reinforcement learning-based intelligent routing method is also provided. For example... Figure 7 As shown, the device includes:

[0095] The network graph construction module 701 is used to construct a communication network model graph, which includes multiple network nodes, where each network node is an agent.

[0096] The state space acquisition module 702 is used to allow each network node to acquire the network state information obtained after interacting with the environment at any given time.

[0097] Decision module 703 is used to input network state information into the agent's policy network and output the action with the highest probability.

[0098] The execution module 704 is used to determine the next network node for data packet routing based on the output action information and send the data packet to the selected next network node.

[0099] It should be noted that the reinforcement learning-based intelligent routing device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the reinforcement learning-based intelligent routing method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the reinforcement learning-based intelligent routing device and the reinforcement learning-based intelligent routing method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0100] According to another aspect of the embodiments of this application, an electronic device corresponding to the reinforcement learning-based intelligent routing method provided in the foregoing embodiments is also provided, so as to execute the reinforcement learning-based intelligent routing method described above.

[0101] Please refer to Figure 8 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the reinforcement learning-based intelligent routing method provided in any of the foregoing embodiments of this application.

[0102] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0103] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. The memory 801 is used to store programs. After receiving execution instructions, the processor 800 executes the program. The reinforcement learning-based intelligent routing method disclosed in any of the foregoing embodiments of this application can be applied to the processor 800, or implemented by the processor 800.

[0104] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0105] The electronic device provided in this application embodiment and the intelligent routing method based on reinforcement learning provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0106] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A smart routing method based on reinforcement learning, characterized in that, include: A communication network model diagram is constructed, which includes multiple network nodes, each of which is an agent. Each agent adopts the Actor-Critic framework, and the agent consists of two networks: an Actor policy network responsible for generating actions and a Critic evaluation network responsible for evaluating the value of the current state. At any given time, each network node obtains network status information obtained after interacting with the environment; the network status information includes the originating node of the data packet, the destination node of the data packet, the historical actions of the current network node, and the number of data packets currently queued by adjacent network nodes; The network state information is input into the agent's policy network, and the action with the highest probability is output; this can be expressed as: Where Q is the policy neural network, For network parameters, This is the output of the policy network in the current state; Based on the output action information, determine the next network node for packet routing and send the packet to the selected next network node; determine whether the next network node for packet routing is the target node; if it is not the target node, record the packet's hop count, transmission cost, queuing delay, and state value at that network node; if it is the target node, calculate the reward value for each node along the entire transmission path from the starting node to the target node; calculate the node's reward value by subtracting the transmission cost and the queuing delay of the current node from the reward value of the next hop node; calculate the loss value of the evaluation network and policy network for any node based on the state value and reward value; update the evaluation network and policy network based on the loss value of the evaluation network and policy network.

2. The method according to claim 1, characterized in that, After each network node obtains the network state information obtained from its current interaction with the environment, it also includes: Get the set of all neighboring network nodes of the current agent's network node; The action space of the agent is generated based on the set of all adjacent network nodes.

3. The method according to claim 1, characterized in that, Before inputting the network state information into the agent's policy network, the method further includes: Construct a network model of the intelligent agent; The network model of the agent includes a policy network and an evaluation network; The policy network consists of an input layer, a hidden layer, and an output layer, and is used to decide which action should be taken in the current network state. The evaluation network consists of an input layer, a hidden layer, and an output layer, and is used to evaluate the value of actions under the current policy.

4. The method according to claim 1, characterized in that, Calculating the state value of a network node includes: The network state information is input into the agent's evaluation network to obtain the state value under the current policy.

5. A smart routing device based on reinforcement learning, characterized in that, include: The network graph construction module is used to construct a communication network model graph, which includes multiple network nodes, each of which is an agent. Each agent adopts the Actor-Critic framework and consists of two networks: an Actor policy network responsible for generating actions and a Critic evaluation network responsible for evaluating the value of the current state. The state space acquisition module is used to enable each network node to acquire the network state information obtained after interacting with the environment at any given time. The network state information includes the originating node of the data packet, the destination node of the data packet, the three historical actions of the current network node, and the number of data packets currently queued by adjacent network nodes. The decision-making module is used to input the network state information into the agent's policy network and output the action with the highest probability; it can be represented as: Where Q is the policy neural network, For network parameters, This is the output of the policy network in the current state; The execution module is used to determine the next network node for data packet routing based on the output action information, and send the data packet to the selected next network node; determine whether the next network node for data packet routing is the target node; if it is not the target node, record the hop count, transmission cost, queuing delay, and state value of the data packet at that network node; if it is the target node, calculate the reward value of each node on the entire transmission path from the starting node to the target node; calculate the reward value of a node, including: subtracting the transmission cost and the queuing delay of the current node from the reward value of the next hop node to obtain the reward value of the current node; calculate the loss value of the evaluation network and policy network for any node based on the state value and the reward value; update the evaluation network and policy network based on the loss value of the evaluation network and policy network.

6. An electronic device, characterized in that, It includes a processor and a memory storing program instructions, the processor being configured to execute the reinforcement learning-based intelligent routing method as described in any one of claims 1 to 4 when executing the program instructions.

Images