Internet of Things routing method based on graph neural network and reinforcement learning

Abstract

The invention discloses an Internet of Things routing method based on a graph neural network and reinforcement learning, and relates to the technical field of Internet of Things communication. The method comprises the following steps: firstly, constructing a network logic topology and a state model containing parameters such as bandwidth and time delay; based on the time random walk sequence, using an LSTM auto-encoder to extract node time evolution characteristics, migrating a weight initialization Node2vec model to extract static structure characteristics, and generating space-time fused node representation; then, neighbor message updating features are aggregated through a graph neural network, and global states are spliced to construct reinforcement learning input; and finally, designing a differentiated reward function for different service types sensitive to time delay or throughput, and optimizing next hop selection by using a deep reinforcement learning agent to maximize network utility as a target. According to the method, the topology dynamic change can be accurately sensed, multi-service differentiated scheduling is realized, the network throughput and robustness are remarkably improved, and the transmission delay is reduced.

Description

Technical Field

[0001] This invention discloses an IoT routing method based on graph neural networks and reinforcement learning, belonging to the field of IoT communication. Specifically, it involves an intelligent routing method combining graph neural networks, sequence modeling, and deep reinforcement learning to solve the multi-service dynamic routing problem in large-scale IoT networks. Background Technology

[0002] As the scale of the Internet of Things (IoT) rapidly expands from hundreds of thousands of devices to hundreds of millions of nodes, the network architecture exhibits both high complexity and dynamic change. IoT applications have widely covered various scenarios such as smart cities, industrial internet, connected vehicles, smart grids, and agricultural monitoring. Nodes possess characteristics such as low power consumption, heterogeneity, multi-protocol support, and multi-service concurrency, causing the network topology to continuously change as nodes sleep, move, and experience link fluctuations. Simultaneously, data traffic has evolved from periodic sensor data to a mix of services including high-definition video, real-time control, and status monitoring, placing significantly different demands on network performance aspects such as latency, throughput, reliability, and jitter. Within such a vast and complex IoT structure, network instability, large fluctuations in link quality, and diverse service requirements pose enormous challenges to the underlying routing mechanisms.

[0003] Existing IoT routing technologies mainly include traditional protocols such as RPL, AODV, DSR, and OSPF, as well as methods based on statistical models, heuristic algorithms, and some based on simple reinforcement learning. These technologies can achieve basic connectivity in small-scale, low-frequency network environments, but they generally suffer from the following problems: First, traditional protocols rely heavily on fixed rules and periodic broadcasts, making it difficult to adapt to frequently changing link states, especially with a sharp increase in routing overhead at the scale of millions of nodes; Second, existing routing strategies are often based on only a single performance metric (such as hop count, latency, or energy consumption), failing to meet the differentiated needs of multi-service coexistence scenarios; Third, while reinforcement learning-based solutions possess certain learning capabilities, they struggle to capture changes in the overall network topology and lack a deep understanding of node relationships, link correlations, and temporal evolution patterns; Fourth, most existing methods lack the comprehensive modeling capabilities for dynamic link quality, service priorities, and multi-dimensional network resources, failing to achieve true global optimization.

[0004] Based on the aforementioned shortcomings, there is an urgent need for a novel intelligent routing method that combines network topology awareness, dynamic link state modeling, and adaptive scheduling based on different service requirements. This invention proposes an IoT multi-service intelligent routing technology based on graph neural networks and reinforcement learning. This technology integrates node structural features and temporal dynamic features, automatically learns the mapping relationship between path quality and service requirements, and achieves intelligent routing decisions across scenarios, protocols, and scales. This method overcomes the performance limitations of traditional protocols in large-scale, highly dynamic, multi-service IoT networks, providing a novel technical path for improving network throughput, reducing latency, increasing data delivery rate, and enhancing overall system robustness. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing IoT routing methods in large-scale, multi-service, and highly dynamic environments by proposing a multi-service IoT routing method based on graph neural networks and reinforcement learning. This method outputs an IoT network state representation through a graph neural network, comprehensively utilizes the network's static topology information and the dynamic state information of the links, and combines it with deep reinforcement learning (DQN) to design an intelligent routing algorithm applicable to multiple service types, ultimately optimizing the network utility function.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: First, the IoT topology is initialized. Based on the concepts of virtual nodes and logical links, an IoT logical topology is constructed, and a network state model is built according to parameters such as available bandwidth, propagation delay, bit error rate, and link availability. Considering the graph properties of the IoT network, a graph embedding method combining LSTM and Node2vec is adopted to fully utilize the time-series information and structural neighborhood information of nodes, generating accurate node representations that are both dynamic and static. Subsequently, Deep Reinforcement Learning (DQN) is introduced to design a dynamic routing algorithm for multiple service types. With the help of the strong representational capabilities of graph neural networks, the deep reinforcement learning agent can better perceive the dynamic changes of the IoT network. Different reward functions are designed for different service types to achieve multiple optimization objectives such as maximizing throughput, minimizing latency, and maximizing delivery rate, ultimately maximizing the utility function. This is a multi-service routing method for IoT based on graph neural networks and reinforcement learning, referencing... Figure 1 This includes the following steps: Step 1, Construct the IoT Topology The Internet of Things (IoT) architecture includes base stations, routers, edge computing nodes, industrial controllers, sensor nodes, and mobile terminals.

[0007] Step 2: Establish the logical topology of the Internet of Things network Let it be represented as a graph. ,in Represents a set of nodes. Represents a set of links.

[0008] (2a) Construct a static topology with dynamic link states, where the set of link states can be represented as: in, Represents a node and nodes The link between them; (2b) The entire runtime of the Internet of Things network Divided into The duration of each time slice is , Indicates the total time, in addition, ,express Not greater than the minimum time scale for link state stability; (2c) Generate link state representation, link The state is composed of multiple parameters, then Among them, parameters , , These represent the available bandwidth, propagation delay, bit error rate, and available time of the link, respectively. Since each time interval has a short duration, ... , here This indicates that the communication time for each user is greater than the duration of each time slice. Step 3: Define the problem model and establish the objective optimization utility function. link The bandwidth capacity is ,variable Indicates the first The bandwidth resources requested by each user. Before transmitting data, for the first... A user found the source node to the destination node The path. The path is defined as... ;in, It indicates the selected path Does the link have an indicator function? If the link , In the selected path, then ,otherwise .

[0009] (3a) Define the link state representation function: make the number Representing paths The average bandwidth, latency, bit error rate, and availability of users in the data are variables. Indicates the first The path from the source node to the destination node for each user; Among them, function l It is a path The length of the function, and the function It is a link The bandwidth. Similarly, the function , and Especially latency, bit error rate, and link Available time.

[0010] (3b) Generate optimization functions to allow different services to choose different routes, and finally maximize the use of all users' bandwidth, latency and bit error rate.

[0011] Where M is the number of users accessing the IoT network, this is a combinatorial optimization problem. This invention solves it through reinforcement learning, determining the weights of each parameter on the performance of the IoT network based on the service type. Ensure that the bandwidth resources requested by users are less than the total bandwidth resources of each link. This is the indicator link. Indicator function for whether it is in the selected path. This means ensuring that for any intermediate link, the input and output traffic are equal; (3c) Based on the different requirements of services for transmission delay sensitivity, services are divided into three categories: A, B and C. Category A is time-sensitive services, such as voice services, which require the shortest possible transmission delay and can tolerate a small amount of packet loss. Category B is services with relatively high throughput requirements, such as video and image services, which have a certain time tolerance and can choose a path with a slightly longer transmission delay for transmission. Category C is services that transmit with the best effort, such as email services, which have no requirements for transmission delay and can tolerate packet loss.

[0012] Step 4: Generate node representations that integrate temporal and structural information. (4a) Reference Figure 2 Construct a time-series graph. Represent the topological state of the IoT network at consecutive time points as an ordered graph sequence. ,in The size of the time window. The current moment; (4b) Constructing a node temporal neighbor sequence based on a temporal random walk. For the current graph Each node in = 1 to ) within the time window [ t−L+1 , t Within the context of the historical connectivity relationships and link evolution trajectories of the nodes, a length not exceeding [missing information] is generated. k A time-limited random walk sequence, and add all random walk sequences to a set. , used to characterize the temporal neighborhood of a node.

[0013] (4c) Refer to Figure 3 The LSTM autoencoder is used to learn the dynamic representation of nodes. The set... The time-random walk sequence is used as input to train an LSTM autoencoder model, which encodes and reconstructs the sequence; the hidden state vector of the last layer of the encoder is denoted as... , used to represent nodes v The temporal evolution characteristics at the current moment. After training, the weight matrix of the LSTM encoder input layer is used as the initialization parameters for the subsequent Node2vec model.

[0014] (4d) Learn the static structure representation of nodes using Node2vec. On the current time-to-time graph... Perform a biased random walk and use the return parameter. p With exploration parameters q A trade-off between breadth-first search and depth-first search styles is achieved; the obtained node neighbor sequences are input into the Skip-gram model for training, resulting in the static structural embedding vector for each node. In this model, the input layer weights are determined by the steps... The LSTM encoder weights of (4e) are initialized to realize the migration of time information to a static structure representation.

[0015] (4f) Constructing an initial node representation that integrates temporal and structural features. The dynamic and static representations of node v are concatenated to obtain the initial feature vector: ,in This represents a vector concatenation operation. The vector simultaneously describes the node's historical behavior patterns and current topological characteristics.

[0016] (4g) Graph neural network message passing is performed based on the fused representation. = As a feature of the 0th layer node, the graph Input to a multilayer graph neural network; in the first... The layer first aggregates messages from neighboring nodes: in, For the festival The neighborhood group, This is a learnable aggregation function (such as weighted summation, mean, or attention-weighted summation). Then, the node representations are updated using a multilayer perceptron: in () is a non-linear activation function.

[0017] (4h) After multiple layers of message passing, the final representation of the node is obtained, which simultaneously includes the node's historical behavior, topology structure, and neighbor state information. .

[0018] Step 5: Construct the readout layer and generate the state vector. (5a) Refer to Figure 4 The node representations are pooled (sum pooling / mean pooling / attention pooling), and a readout function is constructed using a three-layer fully connected network to obtain the graph-level state representation. ; (5b) For scenarios requiring node-level decision-making, the final node representation is concatenated with the global state as the input to the deep reinforcement learning agent. This state vector comprehensively reflects the local state of the node and the global situation of the network. .

[0019] Step 6: Next-hop selection and utility function optimization based on deep reinforcement learning (6a) State space S It is the set of different state representations learned by the graph neural network. ,in For nodes At any moment The final representation of the graph neural network.

[0020] (6b) The action space is the set of all possible actions that a deep reinforcement learning agent can choose: ,action This indicates that the data packet is forwarded to the number... The nodes. For computational convenience, the actions were one-hot encoded in the simulation; (6c) Reward function: The reward function represents the immediate reward that the DRL agent can obtain from the environment after taking a certain action. Three utility functions are used as reward functions. (6d) Reference Figure 5 A multi-service routing training algorithm based on deep reinforcement learning.

[0021] Step 7, Multi-service routing decision algorithm based on deep reinforcement learning Reference Figure 6 The specific operation is as follows: (7a) Set the routing decision cycle T， Initialize time step ; (7b) If the current node receives a business request, proceed to step (6c); otherwise, proceed to step 7g. (7c) The node checks the routing table. If the route for the requested service exists in the routing table, it jumps to step 7f; otherwise, it calculates the current state. Jump to step (7d); (7d) Utilize = Calculate the next hop with the largest Q value for this service request; (7e) The next hop of this service Write to the node routing table; (7f) Forward the service according to the next hop of the service in the routing table; (7g) = ; (7h) If T If the routing is successful, proceed to step 7b; otherwise, the routing process ends.

[0022] Compared with the prior art, the beneficial effects of the present invention are: (1) The IoT routing method based on graph neural network and reinforcement learning introduces LSTM and Node2vec technology, makes full use of the temporal characteristics of nodes, and integrates the dynamic and static information of IoT nodes to generate a more accurate network state representation. At the same time, graph neural network has excellent generalization ability, which can significantly improve the robustness of the algorithm in large-scale, highly dynamic IoT environment.

[0023] (2) This IoT routing method based on graph neural networks and reinforcement learning leverages the black-box modeling advantage of neural networks to enable deep reinforcement learning agents to more comprehensively perceive the topological changes and dynamic characteristics of IoT links. By designing differentiated reward functions according to different business types, objectives such as maximizing throughput, minimizing latency, and optimizing delivery rate can be achieved respectively, ultimately optimizing the overall utility function. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the LSTM principle in this invention; Figure 3 This is a schematic diagram of the Node2vec principle in this invention; Figure 4 This is a schematic diagram illustrating the principle of generating state representations based on LSTM and Node2vec in this invention; Figure 5 This is a flowchart of the routing training algorithm based on deep reinforcement learning in this invention; Figure 6 This is a flowchart of the IoT multi-service routing algorithm in this invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the scope of protection of the present invention.

[0026] Reference Figure 1 The implementation steps of this invention are as follows: Step 1: Establish the Internet of Things (IoT) network topology (1) Construct the Internet of Things (IoT) topology and abstract the base stations, routers, edge computing nodes, industrial controllers, sensor nodes and mobile terminals in the IoT as nodes in the diagram; Step 2: Establish the logical topology of the IoT network and initialize the IoT network state. (2) Let it be represented as a directed graph. ,in Represents a set of nodes. Represents a set of links.

[0027] (2a) Construct a static topology with dynamic link states, where the set of link states can be represented as: in, Represents a node and nodes The link between them; (2b) The entire runtime of the Internet of Things network Divided into The duration of each time slice is , Indicates the total time, in addition, ,express Not greater than the minimum time scale for link state stability; (2c) Generate link state representation, link The state is composed of multiple parameters, then Among them, parameters , , These represent the available bandwidth, propagation delay, bit error rate, and available time of the link, respectively. Since each time interval has a short duration, ... , here This indicates that the communication time for each user is greater than the duration of each time slice. Step 3: Define the problem model and establish the objective optimization utility function. (3) Generate a routing optimization problem model: link The bandwidth capacity is ,variable Indicates the first The bandwidth resources requested by each user. Before transmitting data, for the first... A user found the source node to the destination node The path. The path is defined as... ;in, It indicates the selected path Does the link have an indicator function? If the link , In the selected path, then ,otherwise .

[0028] (3a) Define the link state representation function: make the number Representing paths The average bandwidth, latency, bit error rate, and availability of users in the data are variables. Indicates the first The path from the source node to the destination node for each user; Among them, function l It is a path The length of the function, and the function It is a link The bandwidth. Similarly, the function , and Especially latency, bit error rate, and link Available time.

[0029] (3b) Generate optimization functions to allow different services to choose different routes, and finally maximize the use of all users' bandwidth, latency and bit error rate.

[0030] Where M is the number of users accessing the IoT network, this is a combinatorial optimization problem. This invention solves it through reinforcement learning, determining the weights of each parameter on the performance of the IoT network based on the service type. Ensure that the bandwidth resources requested by users are less than the total bandwidth resources of each link. This is the indicator link. Indicator function for whether it is in the selected path. This means ensuring that for any intermediate link, the input and output traffic are equal; (3c) Based on the different requirements of services for transmission delay sensitivity, services are divided into three categories: A, B and C. Category A is time-sensitive services, such as voice services, which require the shortest possible transmission delay and can tolerate a small amount of packet loss. Category B is services with relatively high throughput requirements, such as video and image services, which have a certain time tolerance and can choose a path with a slightly longer transmission delay for transmission. Category C is services that transmit with the best effort, such as email services, which have no requirements for transmission delay and can tolerate packet loss.

[0031] Step 4: Generate node representations that integrate temporal and structural information. (4a) Reference Figure 2 Construct a time-series graph. Represent the topological state of the IoT network at consecutive time points as an ordered graph sequence. ,in The size of the time window. The current moment; (4b) Constructing a node temporal neighbor sequence based on a temporal random walk. For the current graph Each node in = 1 to ) within the time window [ t−L+1 , t Within the context of the historical connectivity relationships and link evolution trajectories of the nodes, a length not exceeding [missing information] is generated. k A time-limited random walk sequence, and add all random walk sequences to a set. , used to characterize the temporal neighborhood of a node.

[0032] (4c) Refer to Figure 3 The LSTM autoencoder is used to learn the dynamic representation of nodes. The set... The time-random walk sequence is used as input to train an LSTM autoencoder model, which encodes and reconstructs the sequence; the hidden state vector of the last layer of the encoder is denoted as... , used to represent nodes vThe temporal evolution characteristics at the current moment. After training, the weight matrix of the LSTM encoder input layer is used as the initialization parameters for the subsequent Node2vec model.

[0033] (4d) Learn the static structure representation of nodes using Node2vec. On the current time-to-time graph... Perform a biased random walk and use the return parameter. p With exploration parameters q A trade-off between breadth-first search and depth-first search styles is achieved; the obtained node neighbor sequences are input into the Skip-gram model for training, resulting in the static structural embedding vector for each node. .

[0034] The LSTM encoder weights of (4e) are initialized to realize the migration of time information to a static structure representation.

[0035] (4f) Constructing an initial node representation that integrates temporal and structural features. The dynamic and static representations of node v are concatenated to obtain the initial feature vector: ,in This represents a vector concatenation operation. The vector simultaneously describes the node's historical behavior patterns and current topological characteristics.

[0036] (4g) Graph neural network message passing is performed based on the fused representation. = As a feature of the 0th layer node, the graph Input to a multilayer graph neural network; in the first... The layer first aggregates messages from neighboring nodes: in, For the festival The neighborhood group, This is a learnable aggregation function (such as weighted summation, mean, or attention-weighted summation). Then, the node representations are updated using a multilayer perceptron: in () is a non-linear activation function.

[0037] (4h) After multiple layers of message passing, the final representation of the node is obtained, which simultaneously includes the node's historical behavior, topology structure, and neighbor state information. .

[0038] Step 5: Construct the readout layer and generate the state vector. (5a) Refer to Figure 4The node representations are pooled (sum pooling / mean pooling / attention pooling), and a readout function is constructed using a three-layer fully connected network to obtain the graph-level state representation. ; (5b) For scenarios requiring node-level decision-making, the final node representation is concatenated with the global state as the input to the deep reinforcement learning agent. This state vector comprehensively reflects the local state of the node and the global situation of the network. .

[0039] Step 6: Next-hop selection and utility function optimization based on deep reinforcement learning (6a) State space S It is the set of different state representations learned by the graph neural network. ,in For nodes At any moment The final representation of the graph neural network.

[0040] (6b) The action space is the set of all possible actions that a deep reinforcement learning agent can choose: ,action This indicates that the data packet is forwarded to the number... The nodes. For computational convenience, the actions were one-hot encoded in the simulation; (6c) Reward function: The reward function represents the immediate reward that the DRL agent can obtain from the environment after taking a certain action. Three utility functions are used as reward functions. (6d) Reference Figure 5 Multi-service routing training algorithm based on deep reinforcement learning (6d1) Initialize Q-network parameters: Initialize the online Q-network and target Q network And set its initial parameter to a random value. = ,in, For the weight parameters of the online Q-network, These are the weight parameters for the target Q-network, and both are initialized to be consistent.

[0041] (6d2) Initialize the experience replay pool: Initialize an empty data buffer. D It is used to store the experience sample data generated during the training process.

[0042] (6d3) Set training hyperparameters: Set the maximum number of iterations L This indicates the maximum number of iterations in the entire training process; it sets the time step. TThis indicates the number of time steps to run in each iteration; it also sets the batch size. M This indicates the number of training samples retrieved from the data buffer; it also sets the update interval. C Used to control online Q networks The frequency of network updates; setting a discount factor γ to balance the weights of immediate and future rewards; and a learning rate α to update the Q-network parameters.

[0043] (6d4) Set the number of iterations ; (6d5) If L ; (6d6) Determine the input features of DQN ; (6d7) Set time step ; (6d8) If : (6d9) Based on the current state use The -greedy strategy selects an action. ,in This is the exploration rate, used to balance exploration and utilization.

[0044] (6d10) Perform the selected action. And receive instant rewards based on environmental feedback. And according to step 3), transition to the next state. 。

[0045] (6d11) will use the empirical sample Stored in the experience replay pool D middle.

[0046] (6d12) Randomly selected from the experience replay pool D The batch size for extraction is M The samples form a batch ； (6d13) For each sample Calculate the target value ,in, ; (6d14) For each sample in the batch Update the online function by minimizing the loss function. Network parameters Loss function Mean Squared Error (MSE): in, It is an online Q network for the state and actions Q value, It is the target Q value.

[0047] (6d15) Update parameters using gradient descent: ,in, The learning rate; (6d16) Check if the current iteration number meets the condition. epoch % C == 0, if the condition is met, then the target Q network will be... The parameters are updated to reflect the current online Q network. The parameters, i.e. = If the conditions are not met, the target network parameters remain unchanged. (6d17) After each time step cycle ends, the time step counter is turned on. Increase by 1, that is = ,if T (If the condition is met, proceed to step 5h); otherwise, proceed to step 5r. (6d18) epoch = epoch + 1. If epoch L ， If the algorithm terminates, it will proceed to step 5e; otherwise, it will output the final routing strategy and model parameters. .

[0048] Step 7, Multi-service routing decision algorithm based on deep reinforcement learning Reference Figure 6 The specific operation is as follows: (7a) Set the routing decision cycle T， Initialize time step ; (7b) If the current node receives a business request, proceed to step (6c); otherwise, proceed to step 7g. (7c) The node checks the routing table. If the route for the requested service exists in the routing table, it jumps to step 7f; otherwise, it calculates the current state. Jump to step (7d); (7d) Utilize = Calculate the next hop with the largest Q value for this service request; (7e) The next hop of this service Write to the node routing table; (7f) Forward the service according to the next hop of the service in the routing table; (7g) = ; (7h) If T If the routing is successful, proceed to step 7b; otherwise, the routing process ends.

[0049] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. An IoT routing method based on graph neural networks and reinforcement learning, characterized in that, Includes the following steps: Step 1, Construct the logical topology of the IoT network and initialize the network state model: Build a logical topology diagram based on IoT nodes and links. A dynamic link state set is constructed based on the available bandwidth, propagation delay, bit error rate, and available duration parameters of the link. Step 2, Generate node representations that integrate time and structural information: Construct a time series graph, perform time random walks on nodes to generate walk sequences, use an LSTM autoencoder to extract the temporal evolution features of nodes, use a Node2vec model to extract the static structural features of nodes, and concatenate the two to obtain the initial feature vector of the nodes. Step 3, update node representation based on graph neural network: input the initial feature vector into multilayer graph neural network, update node features by aggregating messages from neighboring nodes, and obtain the final node representation containing historical behavior, topology and neighbor state information; Step 4, Constructing the reinforcement learning state space: The graph node representation is pooled using the readout function to generate a global state vector, and the final node representation is concatenated with the global state vector to form the input state of the deep reinforcement learning agent; Step 5, Multi-service routing decision based on deep reinforcement learning: Set differentiated reward functions for different service types, use deep reinforcement learning agents to calculate the action value function Q based on the current input state, select the next hop node as the routing action, and optimize the routing strategy with the goal of maximizing the network utility function.

2. The IoT routing method based on graph neural networks and reinforcement learning according to claim 1, characterized in that, The node representation for generating fusion time and structure information in step 2 specifically includes: The topological state of the Internet of Things (IoT) network at consecutive moments is represented as an ordered graph sequence, and the time window size L is set. For each node in the current time graph, generate a time random walk sequence along the historical connectivity and link evolution trajectory within the time window; The temporal random walk sequence is input into an LSTM autoencoder for training, and the hidden state vectors at the encoder are extracted as the temporal evolution features of the nodes. ; The weight matrix of the input layer of the trained LSTM encoder is used as the initialization parameter of the Skip-gram model in the Node2vec model; Perform a biased random walk on the current time-mapping graph, input it into the Skip-gram model for training, and obtain the static structure embedding vectors of the nodes. ; Will and The features are concatenated to obtain the initial feature vector. .

3. The IoT routing method based on graph neural networks and reinforcement learning according to claim 1, characterized in that, The specific calculation process for updating the node representation based on the graph neural network in step 3 is as follows: using the initial feature vector... As a feature of the 0th layer node ; In the Layers aggregate nodes through aggregation functions. neighboring nodes The characteristic of is calculated using the following formula: Update node representations using a multilayer perceptron: in, For nodes The neighborhood group, This indicates a splicing operation. It is a non-linear activation function. For the process The final representation of a node after layer message passing.

4. The IoT routing method based on graph neural networks and reinforcement learning according to claim 3, characterized in that: In step 5, differentiated reward functions are set for different business types, specifically as follows: Define link The status parameter is the available bandwidth. Delayed transmission Bit error rate and available time Define the reward function The weighted sum of the above state parameters: in, These are the weighting coefficients for bandwidth, latency, bit error rate, and available time, respectively. ; The weighting coefficients are dynamically adjusted based on the business type. For time-sensitive business category A, set It has the highest weight; For throughput-sensitive business type B, set It has the highest weight; For Category C services that transmit at best, set weight balancing or adjust it according to network congestion.

5. The IoT routing method based on graph neural networks and reinforcement learning according to claim 1, characterized in that: The specific process of making multi-service routing decisions based on deep reinforcement learning in step 5 includes: Construct a DQN network, including an online Q network and a target Q network; Based on the current state ,use Strategy selection action The action This indicates that the data packet will be forwarded to the next-hop neighbor node; the action will be executed. Receive instant rewards And transition to the next state , to use empirical samples Store in the experience replay pool; A batch of samples is randomly drawn from the experience replay pool, and the parameters of the online Q-network are updated by minimizing the mean squared error loss function, which is: in, For the target Q value, This is the discount factor.

6. The IoT routing method based on graph neural networks and reinforcement learning according to claim 1, characterized in that: Step 4, which involves constructing the reinforcement learning state space, specifically includes: The final representation of the nodes output by the graph neural network Pooling operations are performed, and a readout function is constructed through a fully connected network to generate a graph-level state representation. : Node The local final representation Representation of graph-level states By concatenating the data, a state vector is generated that comprehensively reflects both the local state of the nodes and the global state of the network. .

7. The IoT routing method based on graph neural networks and reinforcement learning according to claim 1, characterized in that: When constructing the network state model in step 1, the entire running time will be used. Divided into Each time slice has a duration of [number] time slices. It is not greater than the minimum time scale for link state stability, and satisfies ,in Indicates the duration of user communication.

8. An IoT multi-service routing system based on graph neural networks and reinforcement learning, characterized in that, include: The topology building module is used to construct the logical topology of the Internet of Things network and initialize the network state model containing bandwidth, latency, and bit error rate information. The node representation learning module is used to extract the temporal evolution features of nodes using an LSTM autoencoder and to initialize the Node2vec model with LSTM weights to extract the static structural features of nodes and generate fused node representations. The graph neural network module is used to update node features by aggregating neighbor messages and generate a network state vector containing global information. The routing decision module is used to utilize a deep reinforcement learning model to output the selection strategy for the next-hop routing node based on the current network state vector and the reward function corresponding to the service type.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images