A method for generating an information transmission path based on reinforcement learning

Abstract

The application discloses a kind of information transmission path generation methods based on reinforcement learning, belong to wireless communication intelligent decision-making technical field.The application includes the following steps: constructing information transmission optimization environment class;Information transmission path agent class is constructed;Information transmission task description information is constructed, based on information transmission task description information, calling information transmission optimization environment class, instantiates information transmission optimization environment env;From information transmission optimization environment env, obtain the attribute of node quantity and each node, set learning rate, discount factor, initial exploration rate, exploration rate attenuation coefficient and minimum exploration rate, call information transmission path agent class, instantiates information transmission path agent agent;Information transmission path agent agent is trained;Optimal path of information transmission is obtained using trained information transmission path agent.The application has multi-constraint fusion capability, and logic framework is clear, easy to realize and maintain.

Description

Technical Field

[0001] This invention relates to the field of wireless communication intelligent decision-making technology, and in particular to an information transmission path generation method based on reinforcement learning. Background Technology

[0002] In the process of information transmission through communication networks, fields such as unmanned systems, aerospace telemetry and control, and spectrum management all rely on wireless communication networks for the stable transmission of measurement, situational awareness, and situational information. With the increasing number of distributed nodes in the system, finding suitable information transmission paths under various requirements and constraints such as transmission capacity and transmission length has become a current research hotspot.

[0003] In the patent application "A Communication Node Connectivity Analysis Method Based on Knowledge Graph" (CN119966834B), filed by the 54th Research Institute of China Electronics Technology Group Corporation, a breadth-first search method is used to obtain information transmission paths and determine the connectivity between communication nodes. In the patent application "A Communication Countermeasure Effect Evaluation Method Based on Knowledge Graph" (CN120455233B), also filed by the 54th Research Institute of China Electronics Technology Group Corporation, a shortest path acquisition method is used to obtain information transmission paths and determine the connectivity between two nodes in a complex electromagnetic environment. Traditional graph optimization methods (DFS, BFS, and Dijstra, etc.) are widely used in information transmission scenarios; however, these methods typically only address a single objective, such as shortest path or reachability constraints or objectives. To simultaneously achieve optimization for multiple objectives, the optimization method itself needs to be manually modified, and specific modifications need to be made for the actual problem, adjusting the overall processing logic of the method. This makes it difficult to integrate constraints of complex business processes, and the logical structure is unclear. In Huang Zhuozhen's research paper "Research on Path Selection for IoT Big Data Transmission Considering Low Latency," published in *Yangtze Information and Communication*, the optimal transmission path is obtained using the ant colony algorithm, aiming to minimize latency and maximize transmission success rate. However, this method does not consider factors such as node reachability and communication capacity requirements during topology construction, making it unsuitable for complex, multi-objective scenarios and constraints in engineering projects. Summary of the Invention

[0004] To address the need for optimizing information transmission paths under multiple objectives and constraints, this invention proposes an information transmission path generation method based on reinforcement learning. This invention features multi-constraint fusion capabilities, a clear logical framework, and is easy to implement and maintain.

[0005] The technical solution adopted in this invention is as follows:

[0006] A reinforcement learning-based method for generating information transmission paths includes the following steps:

[0007] Step 1: Construct an information transmission optimization environment class;

[0008] Step 2: Construct the intelligent agent class for the information transmission path;

[0009] Step 3: Construct information transmission task description information, including communication network topology diagram, information start node, information end node, information transmission prohibited node list, and information capacity required for information transmission; based on the information transmission task description information, call the information transmission optimization environment class and instantiate the information transmission optimization environment env;

[0010] Step 4: Obtain the number of nodes and the attributes of each node from the information transmission optimization environment env, set the learning rate, discount factor, initial exploration rate, exploration rate decay coefficient and minimum exploration rate, call the information transmission path agent class, and instantiate the information transmission path agent.

[0011] Step 5: Train the intelligent agent for the information transmission path;

[0012] Step 6: Use the trained information transmission path agent to obtain the optimal information transmission path.

[0013] Furthermore, the information transmission optimization environment class includes an initialization function `__init__`, a reset function `reset`, a step function `step`, a communication network topology graph acquisition function `get_current_graph`, and attributes `self.original_graph`, `self.graph`, `self.start`, `self.end`, `self.nodes`, `self.forbidden_point_list`, `self.information_width`, `self.node_to_idx`, and `self.idx_to_node`, wherein:

[0014] The initialization function __init__ is used to initialize various attributes. The input of the initialization function __init__ is the communication network topology diagram, the information start point, the information end point, the list of nodes prohibited from information transmission, and the information capacity required for information transmission. The communication network topology diagram includes nodes, the communication distance between nodes, and the remaining information transmission capacity.

[0015] `self.original_graph` and `self.graph` are used to store the communication network topology diagram for deep copying. `self.start` stores the starting node of the information, `self.end` stores the ending node of the information, and `self.nodes` stores all nodes in the network as a list. The nodes stored in `self.nodes` are obtained by extracting all keys from the `self.original_graph` dictionary. `self.forbidden_point_list` stores the list of nodes where information transmission is prohibited. `self.information_width` stores the capacity required for this information transmission. `self.node_to_idx` stores a dictionary that converts node names to node indices, where the node name is the key and the node index is the value. `self.idx_to_node` stores a dictionary that converts node indices to node names, where the node index is the key and the node name is the value.

[0016] The reset function is used to perform the following steps:

[0017] Make a deep copy of self.original_graph into self.graph, and assign self.start to self.current_node, where self.current_node is a variable used to store the name of the node containing the current information;

[0018] Using the dictionary self.node_to_idx, the index of the current node is retrieved using self.current_node as the key, and this index is used as the return value of the reset function.

[0019] The step function is used to perform the following steps:

[0020] Using the dictionary self.idx_to_node, with action_idx as the key, the node name is obtained and stored in the action variable, and self.current_node is stored in the current variable; action_idx is the input of the step function step, representing the index of the next node in the currently selected information transmission path;

[0021] Read the current key in self.graph, check if the corresponding value contains action, and determine if action is an adjacent node of current node. If it is, continue to execute the subsequent process. If not, return the index of current node as the state after the action is executed, and return the reward value reward and the termination information done=False.

[0022] Determine if the action is in self.forbidden_point_list to determine if the action node is a forbidden node; if it is a forbidden node, return the index of the current node as the state after the action is executed, and return the reward value and the abort information done=False; otherwise, continue to execute the subsequent process.

[0023] Obtain the information path capacity remaining from the current node to the action node from self.graph, save remaining to current_remaining, and determine whether the capacity required for this information transmission, self.information_width, is greater than current_remaining. If it is greater, return the index of the current node as the state after the action is executed, and return the reward value reward and the termination information done=False. Otherwise, continue to execute the subsequent process.

[0024] The difference between current_remaining and self.information_width is used as the new information transmission capacity, which is stored in the information path capacity remaining of the path from the corresponding current node to the action node in self.graph; self.current_node is stored as the action node, and the distance distance of the information path from the current node to the action node obtained from self.graph is taken as a negative value and added to the step reward reward_step to obtain the reward value reward;

[0025] If action equals self.end, meaning the information has reached the end point, then add the final reward to the reward, return the index of the current node as the state after the action is executed, and return the reward value reward + final_reward and the termination information done=True; otherwise, return the index of the current node as the state after the action is executed, and return the reward value reward and the termination information done=False.

[0026] The function `get_current_graph` retrieves the communication network topology graph and returns `self.graph`.

[0027] Furthermore, the information transmission path optimization agent class includes an initialization function `__init__`, an action selection function `act`, a learning function `learn`, and attributes `self.size`, `self.lr`, `self.gamma`, `self.epsilon`, `self.epsilon_decay`, `self.min_epsilon`, and `self.q_table`; where:

[0028] The initialization function __init__ is used to initialize various attributes. The inputs to the initialization function __init__ are the number of nodes, node list, learning rate, discount factor, initial exploration rate, exploration rate decay coefficient, and minimum exploration rate of the communication network topology graph.

[0029] `self.size` is used to store the number of nodes in the communication network topology graph; `self.lr` is used to store the learning rate; `self.gamma` is used to store the discount factor; `self.epsilon` is used to store the initial exploration rate; `self.epsilon_decay` is used to store the exploration rate decay coefficient; `self.min_epsilon` is used to store the minimum exploration rate; and `self.q_table` is used to store a two-dimensional Q-table. The rows of the two-dimensional Q-table correspond to the index of the current node, and the columns correspond to the index of the next node after the action is executed. The dimension is the number of nodes.

[0030] The action selection function `act` takes the current state as its input, i.e., the index of the node where the agent is currently located, and outputs the action, i.e., the index of the node the agent decides to go to. The action selection function `act` performs the following steps: it generates a random number between 0 and 1. If the random number is less than `self.epsilon`, it randomly selects the index of a node as the output. Otherwise, it selects the index with the largest value and the smallest index value from the row corresponding to the current node index in the two-dimensional Q-table as the action output.

[0031] The `learn` function is used to perform the following steps:

[0032] The current node index is used as the state, the node index to which the agent decides to go is used as the action, and the reward and done information after the action is executed are obtained.

[0033] Take the Q value q_value from the state and action columns of the two-dimensional Q table;

[0034] If the termination information done=True, then the value of the variable target is set to reward; otherwise, the largest value is selected in the state row of the two-dimensional Q table, multiplied by the discount factor self.gamma, and then added to reward. The result is then assigned to the variable target.

[0035] Calculate (target-q_value)×self.lr+q_value, and assign the result to the state row and action column of the two-dimensional Q table;

[0036] Choose the smaller value from self.min_epsilon×self.epsilon and self.min_epsilon×self.epsilon_decay, assign it to self.epsilon, and update the value of self.epsilon.

[0037] Furthermore, step 5 is specifically implemented as follows:

[0038] Set the maximum number of training episodes;

[0039] In each training iteration, firstly, the reset function in the information transmission optimization environment `env` is executed to obtain the initial node position `state` of the agent. Then, the training iteration optimization begins, which involves: executing the action selection function `act` in the information transmission path optimization agent `agent`, taking `state` as input, and obtaining the agent's action `action`, i.e., the communication node to be visited; executing the step function `step` in the information transmission optimization environment `env`, taking `action` as input, and obtaining the node position `next_state`, reward, and termination information `done` after the action is executed; and executing the learning function `learn` in the information transmission path optimization agent `agent`, taking `state`, reward, `next_state`, and `done` as input. If `done` = True or the number of iterations is greater than 50, the iteration optimization is terminated; otherwise, the next iteration continues.

[0040] Once the maximum number of training episodes is reached, the training of the agent for the information transmission path is complete.

[0041] Furthermore, step 6 is performed as follows:

[0042] The reset function in the environment env is used to optimize information transmission and reset the environment, converting the node names stored in the state into node indices and reassigning them to the state.

[0043] Set the number of iterations to twice the number of nodes, and perform the following steps in each iteration:

[0044] The node index is assigned to the variable action, with the position corresponding to the first maximum value in the state row of the two-dimensional Q-table in the information transmission path agent as the node index.

[0045] Call the step function in the information transmission optimization environment env, input action, and obtain the node position next_state, reward, and done information after the action is executed;

[0046] By optimizing the information transmission environment env using idx_to_node, next_state is converted into a node name and stored in the variable next_node;

[0047] Reassign next_state to state;

[0048] If done=True, exit the loop; otherwise, continue the loop.

[0049] The optimal information transmission path can be obtained by using the function get_current_graph to retrieve the communication network topology graph in the information transmission optimization environment env.

[0050] Compared with existing methods, the advantages of this invention are as follows:

[0051] 1. This invention possesses multi-constraint fusion capabilities. Traditional methods struggle to balance multiple heterogeneous optimization objectives such as path length, tolerance constraints, and node access permissions. This invention, by designing a composite reward function and integrating it with environmental design, transforms the multi-objective optimization problem into a Markov decision process, thereby achieving the generation of optimal information transmission strategies under complex constraints and requirements.

[0052] 2. This invention achieves clear decoupling of the logical framework, improving the maintainability of the system. By constructing independent information transmission optimization environment and information transmission path intelligent agent classes, this invention achieves physical isolation of functional logic. The information transmission optimization environment class focuses on constructing the information transmission path graph structure, communication capacity consumption mechanism, node position transfer rules, and reward function. The information transmission path intelligent agent class interacts through standardized interfaces, forming a closed loop for path optimization. This framework significantly improves the logical clarity of the method, reduces system coupling, and enhances code readability and maintainability. Detailed Implementation

[0053] The present invention will be further described below with reference to specific embodiments.

[0054] A reinforcement learning-based method for generating information transmission paths is proposed. This method first establishes an optimization environment based on the communication network topology, integrating objectives and constraints such as prohibited transmission nodes, shortest transmission length, and minimum number of transmissions. Then, a path optimization agent based on reinforcement learning is constructed. This agent interacts with the environment and, through training, acquires the optimal information transmission path and returns the changes in the communication network topology after information transmission.

[0055] The specific steps of this method are as follows:

[0056] Step 1: Construct the information transmission optimization environment class, which includes initialization functions, reset functions, step functions, and functions for obtaining the communication network topology diagram. The specific method is as follows:

[0057] Step 1-1: Construct the initialization function `__init__` for the `PathEnv` message transmission optimization environment class. The inputs are the communication network topology diagram (including nodes, node-time communication distances, and remaining message transmission capacity), the message start point, the message destination, the list of nodes prohibited from message transmission, and the required message capacity. The initialization function `__init__` is used to establish instance attributes:

[0058] `self.original_graph` and `self.graph` are used to store a deep copy of the communication network topology graph;

[0059] self.start is used to store the starting node of information;

[0060] self.end is used to store information about the termination node;

[0061] self.nodes is used to store all nodes in the network as a list, which is obtained by extracting all keywords from the self.original_graph dictionary;

[0062] `self.forbidden_point_list` is used to store a list of nodes where information transmission is prohibited.

[0063] The `self.information_width` parameter is the capacity required to store the information transmitted in this instance.

[0064] `self.node_to_idx` is a dictionary that stores a dictionary of transformations from node names to node indices, where node names are keys and node indices are values.

[0065] `self.idx_to_node` is a dictionary that stores a dictionary of transformations from node indexes to node names, where the node index is the key and the node name is the value.

[0066] Step 1-2: Construct the reset function `reset`. Deep copy `self.original_graph` into `self.graph`, and assign `self.start` to `self.current_node`. `self.current_node` stores the name of the node containing the current information. Use the `self.node_to_idx` dictionary with `self.current_node` as the key to find the index of the current node, and return this index as the function's return value.

[0067] Steps 1-3: Construct the step function `step`. The function input is `action_idx`, representing the index of the next node in the currently selected information transmission path. Using the dictionary `self.idx_to_node`, with `action_idx` as the key, obtain the node name, save it to the `action` variable, and save `self.current_node` to the `current` variable.

[0068] By checking if the value corresponding to the `current` keyword in `self.graph` contains the action, it determines whether the action is an adjacent node of the `current` node. If not, the `step` function terminates, returning the index of the `current` node as the state after the action's execution, a reward value of -100, and the termination message `done=False`. Otherwise, execution continues.

[0069] This function determines whether an action node is a forbidden node by checking if the action is in `self.forbidden_point_list`. If it is in the forbidden node list, it returns the index of the current node as the status after the action was executed, the reward value `reward` is -100, and the termination message `done=False`. Otherwise, execution continues.

[0070] The information path capacity (remaining) from the current node to the action node is obtained from `self.graph` and stored in `current_remaining`. It is then checked whether the capacity required for this information transmission (`self.information_width`) is greater than `current_remaining`. If it is greater, it means the current transmission path is insufficient to support information transmission. In this case, the index of the current node is returned as the status after the action execution, the reward value is -100, and the termination message `done=False` is returned. Otherwise, execution continues.

[0071] The new information transmission capacity is generated by subtracting self.information_width from current_remaining and stored in the remaining portion of the path from the current node to the action node in self.graph. self.current_node is stored as the action node. The reward is obtained by taking the negative of the distance (distance) from the current node to the action node obtained from self.graph and adding it to the reward_step, where reward_step = -1.

[0072] If `action` equals `self.end`, meaning the message has reached its destination, then `reward` is incremented by `final_reward`, where `final_reward` = 50. The index of the current node is returned as the state after the action execution, along with the reward value and the termination information `done` = `True`. Otherwise, execution continues, and the index of the current node is returned as the state after the action execution, along with the reward value and the termination information `done` = `False`.

[0073] Steps 1-4: Construct the communication network topology graph using the function get_current_graph, which returns self.graph.

[0074] Step 2: Construct an information transmission path agent class based on reinforcement learning. This class includes an initialization function, an action selection function, and a learning function. The specific method is as follows:

[0075] Step 2-1: Construct the initialization function `__init__` for the Agent class, which optimizes the information transmission path. The inputs are the number of nodes in the communication network topology, learning rate, discount factor, initial exploration rate, exploration rate decay coefficient, and minimum exploration rate. The `__init__` function is used to establish instance attributes:

[0076] self.size is used to store the number of nodes in the communication network topology graph;

[0077] self.lr is used to store the learning rate;

[0078] self.gamma is used to store the discount factor;

[0079] self.epsilon is used to store the initial exploration rate;

[0080] self.epsilon_decay is used to store the exploration rate decay coefficient;

[0081] self.min_epsilon is used to store the minimum exploration rate;

[0082] `self.q_table` is used to store a two-dimensional Q-table, where rows correspond to the index of the current node, columns correspond to the index of the next node after the action is executed, and the dimension is the number of nodes.

[0083] Step 2-2: Construct the action selection function `act`. The input is the current state, i.e., the index of the node where the agent is currently located. The output is the action, i.e., the index of the node the agent decides to go to. If a random number obtained from 0 to 1 is less than `self.epsilon`, randomly select the index of a node as the output. Otherwise, select the index with the largest value and the smallest index from the row corresponding to the current node index in the Q table as the action output.

[0084] Steps 2-3: Construct the learning function `learn`. The input is the index of the current node as the state, and the agent determines the node index to move to as the action, the reward after the action is executed, and the termination information as `done`.

[0085] Take the Q value q_value from the state and action columns of the Q table.

[0086] If the termination message "done" is True, the value of the "target" variable is "reward". Otherwise, the largest value in the "state" row of the Q table is multiplied by the discount factor "self.gamma" and then added to "reward" and assigned to "target".

[0087] The target minus q_value multiplied by the learning rate self.lr, plus q_value, is assigned to the state row and action column of table Q.

[0088] Choose the smaller value from self.min_epsilon and the product of self.epsilon and self.epsilon_decay and assign it to self.epsilon.

[0089] Step 3: Construct information transmission task description information, including a communication network topology diagram (including nodes, node-time communication distances, and remaining information transmission capacity), the information start node, the information end node, a list of prohibited information transmission nodes, and the information capacity required for information transmission; based on the information transmission task description information, call the information transmission optimization environment class and instantiate the information transmission optimization environment `env`. The specific method is as follows:

[0090] Step 3-1: Construct the information transmission task description information. Construct a communication network topology diagram in dictionary format, including the distance between each communication node and its connected nodes, and the remaining communication capacity. `start_point` stores the starting point of the information transmission, `end_point` stores the ending point, `forbidden_point_list` stores the list of nodes where information transmission is prohibited, and `information_width` stores the capacity required for this information transmission.

[0091] Step 3-2: Create a communication transmission path optimization environment. Call the PathEnv class, input the communication network topology diagram, the information start point, the information end point, the list of prohibited nodes for information transmission, and the information capacity required for transmission, and instantiate the env environment. The communication network topology diagram is as follows:

[0092] graph = {

[0093] 'A': {

[0094] 'B': {'distance': 1, 'remaining': 2},

[0095] 'C': {'distance': 4, 'remaining': 2}

[0096] },

[0097] 'B': {

[0098] 'A': {'distance': 1, 'remaining': 2},

[0099] 'C': {'distance': 2, 'remaining': 2},

[0100] 'D': {'distance': 1, 'remaining': 3}

[0101] },

[0102] 'C': {

[0103] 'D': {'distance': 1, 'remaining': 2},

[0104] 'E': {'distance': 4, 'remaining': 2}

[0105] },

[0106] 'D': {

[0107] 'E': {'distance': 1, 'remaining': 2}

[0108] },

[0109] 'E': {}

[0110] }

[0111] Information start point start_point = 'A'

[0112] Endpoint of information = 'E'

[0113] The list of nodes where information transmission is prohibited is `forbidden_point_list = ['C']`.

[0114] The information capacity required for information transmission is imformation_width = 2.

[0115] Step 4: Obtain the number of nodes and the attributes of each node from the information transmission optimization environment (env); set the learning rate, discount factor, initial exploration rate, exploration rate decay coefficient, and minimum exploration rate; call the information transmission path agent class and instantiate the information transmission path agent; the specific method is as follows:

[0116] Obtain the nodes attribute from the environment environment, calculate the list length to get the number of nodes, call the Agent class and input the number of nodes, instantiate the agent intelligent agent, initialize the learning rate to 0.9, the discount factor to 0.95, the initial exploration rate to 0.5, the exploration rate decay coefficient to 0.995, and the minimum exploration rate to 0.01.

[0117] Step 5: Train the intelligent agent for the information transmission path; the specific method is as follows:

[0118] Set the episode to the maximum number of training iterations. In each training iteration, first, execute the `reset` function in `env` to obtain the initial node position (`state`) of the agent. Then, enter the training iteration optimization, which involves: executing the `act` function in `agent` and inputting `state` to obtain the agent's action, i.e., the communication node to go to. Execute the `step` function in `agent` and inputting `action` to obtain the node position `next_state`, reward, and termination information `done` after the action is executed. Execute the `learn` function in `agent` and input `state`, `reward`, `next_state`, and `done`. If `done` is True or the iteration count is greater than 50, exit the iteration optimization. Finally, when the maximum number of training iterations (`episode`) is reached, execute step 6. Otherwise, continue training.

[0119] Step 6: Use the trained information transmission path agent to obtain the optimal information transmission path; the specific method is as follows:

[0120] To obtain the optimal information transmission path: The environment is reset using the `reset` function in `env`, converting the node names stored in `state` into node indices and assigning them to `state`. A `path` list is constructed to store path nodes, and `state` is stored within it.

[0121] The loop iteration count is set to twice the number of nodes. In each iteration, the node index corresponding to the first maximum value in the `state` row of the `Q` table in the agent is assigned to the `action`. The `step` function in the `env` environment is called, taking the `action` as input to obtain the node position `next_state`, reward, and termination information `done` after the action is executed. `next_state` is converted into a node name using the `idx_to_node` dictionary in the `env` environment and stored in `next_node`, and also added to the `path` list. `next_state` is then stored in `state`. When the value of `done` is `True`, the loop exits; otherwise, the loop continues.

[0122] The optimal transmission path obtained is:

[0123] A—B—D—E

[0124] Using the `get_current_graph` function in the `env` environment, the communication network topology after information transmission is obtained as follows:

[0125] graph = {

[0126] 'A': {

[0127] 'B': {'distance': 1, 'remaining': 0},

[0128] 'C': {'distance': 4, 'remaining': 2}

[0129] },

[0130] 'B': {

[0131] 'A': {'distance': 1, 'remaining': 2},

[0132] 'C': {'distance': 2, 'remaining': 2},

[0133] 'D': {'distance': 1, 'remaining': 1}

[0134] },

[0135] 'C': {

[0136] 'D': {'distance': 1, 'remaining': 2},

[0137] 'E': {'distance': 4, 'remaining': 2}

[0138] },

[0139] 'D': {

[0140] 'E': {'distance': 1, 'remaining': 0}

[0141] },

[0142] 'E': {} # End point

[0143] }

[0144] In summary, this invention enables the optimization and generation of information transmission paths through the interaction and training of an intelligent agent with an optimized environment. This method allows for the fusion of multiple objectives and constraints, effectively generating information transmission paths.

Claims

1. A method for generating information transmission paths based on reinforcement learning, characterized in that, Includes the following steps: Step 1: Construct an information transmission optimization environment class; the information transmission optimization environment class includes an initialization function __init__, a reset function reset, a step function step, a communication network topology graph acquisition function get_current_graph, and attributes self.original_graph, self.graph, self.start, self.end, self.forbidden_point_list, self.information_width, self.node_to_idx, and self.idx_to_node; Step 2: Construct the information transmission path intelligent agent class; the information transmission path optimization intelligent agent class includes an initialization function __init__, an action selection function act, a learning function learn, and attributes self.size, self.lr, self.gamma, self.epsilon, self.epsilon_decay, self.min_epsilon, and self.q_table; Step 3: Construct information transmission task description information, including communication network topology diagram, information start node, information end node, information transmission prohibited node list, and information capacity required for information transmission; based on the information transmission task description information, call the information transmission optimization environment class and instantiate the information transmission optimization environment env; Step 4: Obtain the number of nodes and the attributes of each node from the information transmission optimization environment env, set the learning rate, discount factor, initial exploration rate, exploration rate decay coefficient and minimum exploration rate, call the information transmission path agent class, and instantiate the information transmission path agent. Step 5: Train the intelligent agent for the information transmission path; the specific method is as follows: Set the maximum number of training episodes; In each training iteration, firstly, the reset function in the information transmission optimization environment `env` is executed to obtain the initial node position `state` of the agent. Then, the training iteration optimization begins, which involves: executing the action selection function `act` in the information transmission path optimization agent `agent`, taking `state` as input, and obtaining the agent's action `action`, i.e., the communication node to be visited; executing the step function `step` in the information transmission optimization environment `env`, taking `action` as input, and obtaining the node position `next_state`, reward, and termination information `done` after the action is executed; and executing the learning function `learn` in the information transmission path optimization agent `agent`, taking `state`, reward, `next_state`, and `done` as input. If `done` = True or the number of iterations is greater than 50, the iteration optimization is terminated; otherwise, the next iteration continues. Once the maximum number of training episodes is reached, the training of the agent for the information transmission path is complete. Step 6: Use the trained information transmission path agent to obtain the optimal information transmission path; the specific method is as follows: The environment is reset using the reset function in the information transmission optimization environment env. The environment is deep copied from self.original_graph to self.graph, and the node names stored in state are converted into node indices and reassigned to state. self.original_graph and self.graph are used to store the deep copy of the communication network topology diagram. Set the number of iterations to twice the number of nodes, and perform the following steps in each iteration: The node index is assigned to the variable action, with the position corresponding to the first maximum value in the state row of the two-dimensional Q-table in the information transmission path agent as the node index. Call the step function in the information transmission optimization environment env, input action, and obtain the node position next_state, reward, and done information after the action is executed; The `self.idx_to_node` method in the `env` environment optimizes information transmission, converts `next_state` into node names, and stores them in the variable `next_node`. `self.idx_to_node` is used to store a dictionary of conversions from node indices to node names, where node indices are keys and node names are values. Reassign next_state to state; If done=True, exit the loop; otherwise, continue the loop. The function `get_current_graph` returns `self.graph` to obtain the optimal information transmission path by utilizing the communication network topology graph obtained in the information transmission optimization environment `env`.

2. The information transmission path generation method based on reinforcement learning according to claim 1, characterized in that, In the information transmission optimization environment class: The initialization function __init__ is used to initialize various attributes. The input of the initialization function __init__ is the communication network topology diagram, the information start point, the information end point, the list of nodes prohibited from information transmission, and the information capacity required for information transmission. The communication network topology diagram includes nodes, the communication distance between nodes, and the remaining information transmission capacity. `self.start` is used to store the starting node of the information, `self.end` is used to store the ending node of the information, `self.forbidden_point_list` is used to store the list of nodes where information transmission is prohibited, `self.information_width` is used to store the capacity required for this information transmission, and `self.node_to_idx` is used to store a dictionary of conversions from node name to node index, where the node name is the key and the node index is the value. The reset function is used to perform the following steps: Make a deep copy of self.original_graph into self.graph, and assign self.start to self.current_node, where self.current_node is a variable used to store the name of the node containing the current information; Using the dictionary self.node_to_idx, the index of the current node is retrieved using self.current_node as the key, and this index is used as the return value of the reset function. The step function is used to perform the following steps: Using the dictionary self.idx_to_node, with action_idx as the key, the node name is obtained and stored in the action variable, and self.current_node is stored in the current variable; action_idx is the input of the step function step, representing the index of the next node in the currently selected information transmission path; Read the current key in self.graph, check if the corresponding value contains action, and determine if action is an adjacent node of current node. If it is, continue to execute the subsequent process. If not, return the index of current node as the state after the action is executed, and return the reward value reward and the termination information done=False. Determine if the action is in self.forbidden_point_list to determine if the action node is a forbidden node; If it is a prohibited node, return the index of the current node as the state after the action is executed, and return the reward value and the abort information done=False; otherwise, continue to execute the subsequent process. Obtain the information path capacity remaining from the current node to the action node from self.graph, save remaining to current_remaining, and determine whether the capacity required for this information transmission, self.information_width, is greater than current_remaining. If it is greater, return the index of the current node as the state after the action is executed, and return the reward value reward and the termination information done=False. Otherwise, continue to execute the subsequent process. The difference between current_remaining and self.information_width is used as the new information transmission capacity, which is stored in the information path capacity remaining of the path from the corresponding current node to the action node in self.graph; self.current_node is stored as the action node, and the distance distance of the information path from the current node to the action node obtained from self.graph is taken as a negative value and added to the step reward reward_step to obtain the reward value reward; If action equals self.end, meaning the information has reached the end point, then add the final reward to the reward, return the index of the current node as the state after the action is executed, and return the reward value reward + final_reward and the termination information done=True; otherwise, return the index of the current node as the state after the action is executed, and return the reward value reward and the termination information done=False. The function `get_current_graph` retrieves the communication network topology graph and returns `self.graph`.

3. The information transmission path generation method based on reinforcement learning according to claim 2, characterized in that, In the intelligent agent class of information transmission path: The initialization function __init__ is used to initialize various attributes. The inputs to the initialization function __init__ are the number of nodes, node list, learning rate, discount factor, initial exploration rate, exploration rate decay coefficient, and minimum exploration rate of the communication network topology graph. `self.size` is used to store the number of nodes in the communication network topology graph; `self.lr` is used to store the learning rate; `self.gamma` is used to store the discount factor; `self.epsilon` is used to store the initial exploration rate; `self.epsilon_decay` is used to store the exploration rate decay coefficient; `self.min_epsilon` is used to store the minimum exploration rate; and `self.q_table` is used to store a two-dimensional Q-table. The rows of the two-dimensional Q-table correspond to the index of the current node, and the columns correspond to the index of the next node after the action is executed. The dimension is the number of nodes. The action selection function `act` takes the current state as its input, i.e., the index of the node where the agent is currently located, and outputs the action, i.e., the index of the node the agent decides to go to. The action selection function `act` performs the following steps: it generates a random number between 0 and 1. If the random number is less than `self.epsilon`, it randomly selects the index of a node as the output. Otherwise, it selects the index with the largest value and the smallest index value from the row corresponding to the current node index in the two-dimensional Q-table as the action output. The `learn` function is used to perform the following steps: The current node index is used as the state, the node index to which the agent decides to go is used as the action, and the reward and done information after the action is executed are obtained. Take the Q value q_value from the state and action columns of the two-dimensional Q table; If the termination information done=True, then the value of the variable target is set to reward; otherwise, the largest value is selected in the state row of the two-dimensional Q table, multiplied by the discount factor self.gamma, and then added to reward. The result is then assigned to the variable target. Calculate (target-q_value)×self.lr+q_value, and assign the result to the state row and action column of the two-dimensional Q table; Choose the smaller value from self.min_epsilon×self.epsilon and self.min_epsilon×self.epsilon_decay, assign it to self.epsilon, and update the value of self.epsilon.