A mobile sensing based intelligent routing method for vehicle-to-rail ad hoc network

Abstract

The application discloses a kind of mobile perception-based car rail ad hoc network intelligent routing method, for the car ground communication scene of car rail self-organizing network, using the way of deep reinforcement learning provides an intelligent routing method, to effectively improve communication efficiency.First, establish the trackside self-organizing network routing model based on clustering.Considering the high-speed mobility of train, LSTM algorithm is added before the optimization problem to predict train trajectory, to improve the accuracy of routing.At the same time, considering the influence of network transmission delay, throughput and energy consumption and other factors, the optimal routing selection process in cluster and between clusters is realized by multi-objective deep reinforcement learning method.Simulation results show that the application can effectively improve the energy efficiency of self-organizing network routing transmission.

Description

Technical Field

[0001] This invention relates to an intelligent routing method for vehicle-track self-organizing networks based on motion sensing, belonging to the field of intelligent control technology for urban rail transit. Background Technology

[0002] With the continuous development of medium and large cities and the rapid growth of urban population, urban rail transit has gradually become an important means to alleviate urban road traffic congestion and the preferred mode of travel for the people. At present, the train control system of urban rail transit uses Communication Based Train Control (CBTC) [1]. As the core component of urban rail transit, the CBTC system can control the speed of trains through technologies such as train positioning, safe route and movement authorization, and achieve the goal of further shortening the train interval and improving the operating efficiency on the basis of ensuring the safe operation of trains [2]. In the existing transportation system, the wireless communication between the train and the ground mainly adopts the wireless local area network (WLAN) technology [3]. However, with the increase in train density and the improvement of public safety requirements, WLAN can no longer meet the requirements of higher communication quality [4]. The urban rail communication system based on self-organizing network has the typical characteristics of large network scale and flexible network deployment, and can exchange control information with the train in a timely manner, which has become the current research hotspot [5].

[0003] Currently, based on different research directions, self-organizing network routing protocols can be divided into three categories: topology-based routing algorithms, geolocation-based routing algorithms, and cluster-based routing algorithms [6]. Cluster routing protocols refer to using the idea of ​​clustering to layer self-organizing networks. Compared with the planar structure, the layered structure is scalable and shows better adaptability in complex traffic scenarios [7]. At the same time, the number of nodes directly participating in network communication in the network will also be reduced, which can effectively solve the shortcomings of planar routing technology in large-scale networks and reduce the additional routing overhead caused by the expansion of network scale [8][9]. For cluster routing algorithms, the literature

[10] proposed a cluster-based adaptive cooperative algorithm (CACA), which considers mobility constraints and QoS requirements. Each vehicle uses a cluster-based QoS algorithm to estimate a reliable low-overhead path. By selecting the cluster head and selecting the MPR, the scalability of OLSR routing is improved, effectively reducing the overhead of topology control packets and end-to-end delay. Reference

[11] proposes a clustered routing (FLBCR) protocol based on fuzzy logic, which uses variable parameters during the network lifetime to determine the probability of selecting a node as a channel and uses the Mamdani model to infer the final probability of selecting a channel. It outperforms the classic LEACH, SEP and DEEC protocols in terms of network stability, lifetime, throughput and average energy.

[0004] With the continuous advancement of routing algorithms, swarm intelligence algorithms have been proposed, including typical examples such as particle swarm optimization, genetic algorithms, and deep reinforcement learning. The particle swarm optimization algorithm proposed in reference

[12] is inspired by birds foraging. Each foraging bird is defined as a particle, and the speed and position information of each particle are continuously updated. Through cooperation and information sharing among particles, the entire flock of birds eventually gathers around the food source. Although this type of algorithm has strong robustness and adaptability, most metaheuristic techniques suffer from premature convergence, getting trapped in local optima, and poor convergence speed. Reference

[13] proposes a routing protocol based on hybrid genetic firefly algorithm (HGFA) to improve the communication speed in VANET, and simulates and compares the performance of transmission time, packet delivery rate, and average throughput. Reference

[14] proposes an intelligent routing scheme based on deep reinforcement learning to support wireless sensor networks of the Internet of Things, so as to significantly reduce latency and extend network lifetime. The algorithm divides the entire network into different unequal clusters according to the data load on the current sensor node, effectively preventing the immature death of the network and effectively improving efficiency in packet transmission, energy efficiency, and communication latency. A survey of clustering and routing algorithms in current clustering routing protocols reveals that broadcast message-based methods incur significant overhead in large-scale networks, such as latency and data throughput. Algorithms based on swarm intelligence can more effectively improve routing efficiency.

[0005] While these excellent research works fully utilize swarm intelligence algorithms to improve routing efficiency, significant latency and energy consumption still occur during the route discovery phase for high-speed urban trains. Inspired by the above analysis, this invention designs an intelligent routing method for vehicle-track self-organizing networks based on motion perception, optimizing routes through a multi-objective deep reinforcement learning method based on train trajectory prediction. First, this invention establishes a scenario model for clustered routing, followed by network transmission, energy consumption, and optimization objectives. Second, it proposes a routing solution based on prediction and deep reinforcement learning, establishing a Long Short-Term Memory (LSTM) network model and inputting it into a multi-objective Markov decision model to achieve performance optimization of route selection. Finally, extensive simulations demonstrate the performance of the proposed solution under different network scenarios. Simulation results show that the model outperforms traditional Internet solutions.

[0006] References:

[0007] [1] Lin Yujun. Urban Rail Transit Signaling [M]. China Railway Publishing House, 2011.

[0008] [2] Wang Xiaoxuan. Reliability Analysis and Optimization of CBTC Wireless Communication System for Urban Rail Transit [D]. Beijing Jiaotong University, 2020.

[0009] [3] L.Zhu, FRYu, B.Ning and T.Tang, "Cross-Layer Handoff Design inMIMO-Enabled WLANs for Communication-Based Train Control (CBTC) Systems," in IEEE Journal on Selected Areas in Communications, vol.30, no.4, pp.719-728, May2012.

[0010] [4] J.Farooq and J.Soler, "Radio Communication for Communications-BasedTrain Control (CBTC): ATutorial and Survey," in IEEE Communications Surveys&Tutorials, vol.19, no.3, pp.1377-1402, thirdquarter 2017.

[0011] [5]M.Amjad, L.Musavian and MHRehmani, "Effective Capacity in WirelessNetworks: A Comprehensive Survey," in IEEE Communications Surveys&Tutorials, vol.21, no.4, pp.3007-3038, Fourthquarter 2019.

[0012] [6] Cao Le. Research on Clustering Routing Protocol for Vehicle-Mounted Ad hoc Networks [D]. Lanzhou Jiaotong University, 2021.

[0013] [7]Alasadi Suad A.andAl Joda Alyaa Abdulhussein and Abdullah EnasFadhil.Mobile adhoc network(MANET) proactive and reactive routing protocols[J].Journal of DiscreteMathematical Sciences and Cryptography,2021,24(7):2017-2025.

[0014] [8] Wu Yongjian, Liu Meng, Luo Hongchi. Research on routing protocols for Ad Hoc networks [J]. China New Communications, 2019, 21(16):5.

[0015] [9]Mahadevaswamy,UB,Jananitha,MP(2020).Congestion Control for ImprovedCluster-Head Gateway Switch Routing Protocol Using Ad-hoc On-demandDistanceVector Algorithm.In:Karrupusamy,P.,Chen,J.,Shi,Y.(eds)SustainableCommunication Networks and Application.ICSCN 2019.Lecture Notes onDataEngineering and Communications Technologies, vol 39. Springer, Cham.

[0016]

[10] N.M.Al-Kharasani,Z.A.Zukarnain,S.K.Subramaniam and Z.M.Hanapi,"AnAdaptive Relay Selection Scheme for Enhancing Network Stability in VANETs,"inIEEEAccess,vol.8,pp.128757-128765,2020.

[0017]

[11] S.Bagga,A.Goyal,G.Gupta and D.K.Sharma,"FLBCR:Fuzzy LogicBasedClustering Routing Protocol,"201910th International Conference onComputing,Communication andNetworking Technologies(ICCCNT),2019,pp.1-7.

[0018]

[12] Xu Bao,Haijian Li,Guoqiang Zhao,Lv Chang,Jun Zhou,Yun Li,Efficient clusteringV2Vrouting based on PSO in VANETs,Measurement,Volume 152,2020,107306,ISSN0263-2241.

[0019]

[13] G.D.Singh,M.Prateek,S.Kumar,M.Verma,D.Singh andH.-N.Lee,"HybridGeneticFirefly Algorithm-Based Routing Protocol for VANETs,"in IEEE Access,vol.10,pp.9142-9151,2022.

[0020]

[14] G.Kaur, P.Chanak and M.Bhattacharya, "Energy-Efficient IntelligentRouting Scheme for IoT-Enabled WSNs," in IEEE Internet of Things Journal, vol.8, no.14, pp.11440-11449, 15July15, 2021. Summary of the Invention

[0021] This invention addresses the shortcomings of existing technologies by proposing a mobile-aware intelligent routing method for trackside ad hoc networks. First, a cluster-based routing model for the trackside ad hoc network is established. Considering the high-speed mobility of trains, an LSTM algorithm is incorporated to predict train trajectories before the optimization problem, improving routing accuracy. Simultaneously, the influence of multiple factors such as network transmission latency, throughput, and energy consumption is comprehensively considered, and optimal intra-cluster and inter-cluster routing selection is achieved through multi-objective deep reinforcement learning. Simulation results demonstrate that this invention can effectively improve the routing transmission energy efficiency of ad hoc networks.

[0022] The technical problem this invention aims to solve is to provide an intelligent routing method using deep reinforcement learning to effectively improve communication efficiency in vehicle-to-ground communication scenarios of vehicle-rail self-organizing networks.

[0023] To address the aforementioned problems, this invention employs the following technical solution: First, the system model of this invention is analyzed and constructed, modeling the optimization problems of latency, throughput, and energy consumption in self-organizing networks as multi-objective optimization problems. Then, for high-speed train operation scenarios, a deep reinforcement learning routing method based on LSTM prediction is proposed, and a multi-objective Markov decision model is established and its performance is theoretically analyzed. Finally, simulation experiments are conducted, and simulation results are presented to evaluate the model performance of this invention. Simulation results show that this invention can effectively improve the efficiency of vehicle-track self-organizing networks.

[0024] A vehicle-track ad hoc network intelligent routing method based on mobility awareness includes the following steps:

[0025] Step 1: Analysis of the self-organizing network scenario and construction of the basic model for rail transit.

[0026] First, a clustered routing scenario model was established based on the operational characteristics of rail transit. Trackside nodes were grouped, and nodes with strong transmission capabilities were elected as cluster heads to communicate directly with the train, replacing ordinary nodes. The process of nodes forwarding data packets to the train was divided into intra-cluster and inter-cluster stages. Then, a self-organizing network transmission model for rail transit was established, and formulas for calculating latency, throughput, and energy consumption were derived based on Shannon's formula and the wireless power consumption formula. Finally, minimizing the latency and energy consumption of the routing process and maximizing the throughput were defined as the collaborative optimization objectives of this invention.

[0027] Step 1.1: Analyze and build the routing model.

[0028] Based on geographical location and the performance of the nodes themselves, the routing nodes next to the track are divided into multiple clusters. Figure 1 The scenario model of this invention includes three participants: cluster head, cluster members, and train. Ignoring the latency and energy consumption of the train sending data packets to the trackside node, the routing process includes two parts: intra-cluster routing and inter-cluster routing.

[0029] Step 1.2: Analyze and construct a clustering model.

[0030] For the routing scenario proposed in step 1.1, clustering models are established from two perspectives: undeployed and deployed.

[0031] Step 1.3: Analyze and construct the transmission model.

[0032] Based on the clustered routing model established in steps 1.1 and 1.2, a network transmission model is constructed according to Shannon's theorem, and end-to-end delay, total routing delay, end-to-end throughput, and network throughput are defined.

[0033] Step 1.4: Analyze and construct a first-order wireless power consumption model.

[0034] Establish an end-to-end energy consumption model for first-order radio and the total energy consumed during the routing process.

[0035] Step 1.5: Analyze and construct optimization objectives.

[0036] Based on the routing scenario, namely the demand of urban rail transit, and the model construction in steps 1.3 and 1.4, this solution establishes three comprehensive optimization objectives: latency, throughput, and energy consumption.

[0037] Step 2: Construct an intelligent routing optimization scheme based on mobility prediction;

[0038] First, the optimization process of routing is discussed based on the scenario model in step 1. Then, an LSTM prediction model and a multi-objective Markov decision model are established. The routing training process is completed using a multi-objective deep reinforcement learning scheme based on motion prediction, and its performance is tested and theoretically analyzed.

[0039] Step 2.1: Discuss routing solutions based on prediction and deep reinforcement learning.

[0040] For the clustering model in step 1, the entire communication process is divided into four modules: environment, LSTM prediction, intra-cluster intelligent routing, and inter-cluster intelligent routing, which work together to complete the routing process.

[0041] Step 2.2: Construct the LSTM prediction model.

[0042] First, construct the LSTM prediction module from step 2.1, establishing an LSTM network model to simulate the train's trajectory. Using the train's historical position and speed as input, and its future position as output, the LSTM network can predict the train's location.

[0043] Step 2.3: Construct a Markov decision model.

[0044] To construct the intelligent routing module in step 2.1, this step combines the routing model and optimization objective from step 1 to establish a multi-objective Markov decision model within and between clusters.

[0045] Step 2.4: Use a deep reinforcement learning model to complete intelligent routing.

[0046] Based on the construction of each module of the routing scheme in steps 2.2 and 2.3, this step establishes a deep reinforcement learning model to realize the LSTM-cooperative multi-objective DDPG routing scheme. Deep reinforcement learning (DRL) is an artificial intelligence algorithm that combines the advantages of deep learning and reinforcement learning. It generates new data and improves its decision-making ability through continuous interaction between the agent and the environment, obtaining the optimal policy through continuous iterative learning. The decision-making algorithm proposed in this step is based on the Deep Deterministic Policy Gradient (DDPG) algorithm, which has the advantages of experience replay and AC network framework, maximizing the cumulative expected reward of the routing process, thereby achieving goal optimization.

[0047] Furthermore, the scenario analysis and basic model construction of the rail transit self-organizing network in step 1 are used to analyze the urban rail self-organizing network architecture, and provide a cluster-based scenario model, transmission model and optimization objective.

[0048] Step 1.1: In the analysis and construction of the routing model, a cluster-based vehicle-rail ad hoc network scenario model is established. The train is a mobile node, traveling along the track on fixed schedules. I wireless ad hoc nodes are deployed alongside the track. Each node in the wireless ad hoc network is uniformly divided into multiple clusters, each cluster consisting of a cluster head node and ordinary member nodes. In the vehicle-rail routing system, each cluster can be represented by the set C = {c1, c2, ..., c...}. k ,...,c K Let} represent the number of clusters, and let K represent the number of clusters. The cluster head node in each cluster is represented by the set U = {u1, u2, ..., u...}. k ,...,u K} represents a set. Ordinary member nodes are represented by the set V = {v1, v2, ..., v...}. i ,...,v I The cluster head is defined as having a transmission capacity much greater than its cluster members, and data is transmitted directly between two cluster heads wirelessly. The intra-cluster network topology is defined as an undirected graph G(V,E), where V is the set of all routing nodes within the cluster, and E represents the set of wireless links between nodes within the cluster. The inter-cluster network topology can be represented by an undirected graph G'(U,E'). U is defined as the set of all cluster heads in the network. E' represents the set of wireless links between cluster heads, and E' represents the set of direct links between cluster heads.

[0049] When a source node intends to send a data packet to a train, it first forwards the information to its cluster head via intra-cluster routing. Then, the cluster head forwards the packet to the destination cluster head closest to the train via inter-cluster routing. Finally, the destination cluster head sends the data to the train. Figure 1 As shown, intra-cluster routing is when a regular routing node s sends a data packet to the cluster head u1 of its cluster, and all nodes within the cluster can participate in this process; inter-cluster routing is when the endpoint cluster head u1 of the above process transmits a data packet to the destination cluster head u3. Only the cluster head node participates in this process u1→u2→u3, which can effectively reduce unnecessary consumption during the transmission of ordinary member nodes.

[0050] Define the data packet transmission path as L = {l 11 ,l 12 ,...,l 21 ,l 22 ,...},{l 11 ,l 12 ,...} represents the transmission path within the cluster, {l 21 ,l 22 ,...} represents the transmission path between clusters, and l represents a single-hop path.

[0051] Furthermore, in step 1.2, the analysis and construction of the clustering model includes two processes: cluster head election and cluster formation. Based on the analysis of train travel characteristics and the layout of the trackside ad hoc network, the following two clustering scenarios will be provided according to the deployment:

[0052] Scenario 1: When no self-organizing network nodes are deployed on the track, a clustering deployment scheme based on transmission power is used. The aim is to save transmission costs in complex network structures by intermittently deploying nodes with high power transmission as cluster heads during the deployment phase of the self-organizing network.

[0053] First, the site is evenly divided into K grids using a grid method, with each grid representing the area of ​​a cluster. High-power transmission nodes are deployed intermittently within each cluster as cluster heads, which can communicate with each other. After selecting cluster heads, low-power nodes are evenly deployed as cluster members within each grid for data transmission within the cluster.

[0054] Scenario 2: For nodes that are already evenly deployed on both sides of the track, a weighted clustering scheme based on K-Means can be used to select the optimal cluster head based on the node's mobility, node difference, location and power consumption, and each node chooses to join the cluster head closest to it.

[0055] The K-Means algorithm initially selects K as the initial number of clusters, and randomly selects K nodes u from the network. k (1≤k≤K) are the cluster heads, where each cluster head belongs to cluster C. k (1≤k≤K), after initially selecting cluster heads, other nodes calculate their distances to each cluster head and assign each point in the self-organizing network to the nearest cluster head within its transmission range. The v of each node is calculated. i The formula for the sum of the squared distances to the cluster head is:

[0056]

[0057] Wherein d(v i ,u k )=||v i -u k || 2 This indicates the distance between the node and the cluster head.

[0058] After assigning nodes to clusters, cluster head weights are calculated using a weighted method. Based on mobility, node difference, location, and power consumption, the node with the smallest weight is selected as the new cluster head. The weighting formula is as follows:

[0059] Weight i =w1D i +w2B i +w3M i +w4Ti

[0060] w1+w2+w3+w4=1

[0061] D i It is the degree difference between the nodes. (The last part, "node v", appears to be a typo and can be left as i The number of neighboring nodes is the number of nodes within its transmission coverage area, denoted as density. Among them, tx range Indicates the transmission distance. h num This represents the number of nodes processed by the cluster head, and the density difference of each node is D. i =|N(i)-h num |

[0062] B i The formula represents the sum of distances between all adjacent nodes:

[0063] M i This represents the mobility of a node, used to calculate the relative speed between the node and the train, and is expressed by the formula M. i =V train ·cosθ i Among them, V train Let θ be the speed of the train. i For node v i Angle with the direction of train travel.

[0064] T i This represents the battery power consumed by the node. The coefficients w1, w2, w3, and w4 are weighting factors for the corresponding system parameters, and their sum is 1. Repeat the process of adding nodes to a cluster and electing a cluster head until all nodes belong to the same cluster, or until the maximum number of iterations is reached, to obtain the optimal clustering result.

[0065] Based on the clustered routing model established in the above steps, step 1.3 further constructs the network model and defines end-to-end delay, total routing delay, end-to-end throughput, and network throughput.

[0066] 1) Delay Model

[0067] The core parameter symbols used in this invention are as follows: Figure 2 As shown. Let B represent the communication bandwidth, and p... up Indicates the transmit power of the terminal on the upload link, g ij Let n represent the channel gain between node i and node j, and n0 be the noise power of the link. According to Shannon's formula, the transmission rate of a single-hop link is... Therefore, node i transmits a size of R to node j. w The latency caused by the data packets is

[0068] The total delay along the transmission path is:

[0069] 2) Throughput Model

[0070] Throughput represents the amount of data successfully transmitted per unit time. End-to-end throughput represents the amount of data successfully transmitted per unit time along a path between the source and destination. The formula is: The maximum throughput of a link is determined by the bottleneck link (minimum throughput) on that path. Network throughput is the average of multiple links and can be used to evaluate the performance of the entire network, expressed as:

[0071] Step 1.4 similarly constructs the network's energy consumption model.

[0072] Referring to the first-order wireless power consumption model, energy consumption during data transmission includes transmitting energy consumption and receiving energy consumption. When data communication occurs between nodes, the energy consumption of a node is related to the distance between the transmitting and receiving ends. The transmission distance between node i and node j is d. ij At that time, transmit b ij The energy consumed by bit data is E ij The calculation is as follows:

[0073]

[0074] Among them, E elc This represents the energy consumed to send or receive 1 bit of data. Channel selection is influenced by the transmission distance. When the transmission interval is less than the threshold d0, a free-space channel model with parameter ε is used. efs If the transmission interval is greater than this value, a multipath attenuation channel model with parameter ε is used. amp .

[0075] Referring to the three cases mentioned in the latency model, the total energy consumed by the data packet on path L can be expressed as:

[0076]

[0077] Based on the transmission model and energy consumption model in steps 1.3 and 1.4, step 1.5 defines that this scheme will optimize multiple objectives during the routing process, including transmission delay, throughput, and energy consumption:

[0078] objective1:min Delay=t L

[0079] objective2: max Throughput = f L

[0080] objective3:min Energy consumption=E L

[0081] Objective 1 is to minimize the total transmission delay of the routing process, using the sum of single-hop delays t. L Objective 2 is to maximize network throughput f. L Objective 3 is to minimize energy consumption, using the sum of single-hop energy consumption E. L express.

[0082] Based on the scenario model established in step 1, step 2 constructs an intelligent routing optimization scheme based on mobility prediction, proposes an intelligent solution, then establishes an LSTM prediction model and a Markov decision model, uses deep reinforcement learning to complete the routing training, and finally tests and analyzes its performance.

[0083] Step 2.1 For the clustering model in Step 1, a routing solution based on prediction and deep reinforcement learning is first discussed.

[0084] The intelligent routing framework proposed in this invention is as follows: Figure 3 As shown, the routing task can be completed collaboratively by four modules: environment, LSTM prediction, intra-cluster intelligent routing, and inter-cluster intelligent routing. The system environment includes high-speed train travel information, trackside ad hoc network clustering, and node-specific routing information. The LSTM prediction module takes train travel data from the environment as its input layer, passes through a hidden layer composed of several LSTM units, and outputs the train's trajectory information. Based on the obtained future train position, the nearest destination node to the receiving train can be calculated.

[0085] The destination node predicted by LSTM is used as the current state input into the Markov decision model. The output of the LSTM prediction is used as the input to the multi-objective DDPG network for the next routing decision. Both intra-cluster and inter-cluster routes take the state of the environment, select the highest-scoring action in the neural network through deep reinforcement learning and obtain a reward, and then iteratively train the Markov decision model to obtain the optimal cooperative routing route.

[0086] Based on the routing scheme proposed in step 2.1, step 2.2 first constructs the LSTM prediction module.

[0087] like Figure 4 The image shows an LSTM unit, consisting of a forget gate, an input gate, and an output gate. The forget gate filters out important information remaining in long-term memory, updating the structural state of long-term memory. The input gate is based on h... t-1 and X tThe amount of new information stored in memory is determined. The output gate is used to compute the state structure of the cell's short-term memory. Due to its special structure, LSTM can conveniently store and access information for long periods of time and model long-term dependencies.

[0088] The goal of an LSTM neural network is to predict the train's position at a specific future time step based on prior information. The future position of the train will be calculated using the following formula:

[0089] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0090] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0091]

[0092]

[0093] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0094] h t =o t *tanh(C t )

[0095] Where x is the input, f is the output of the forget gate, h is the hidden layer state, and i is the portion of the new information retained. The new input brings information, C is the cell state, o is the information output by the output gate, W and b are the weight matrix and bias vector respectively, e represents the scalar product of the two vectors, and σ represents the standard activation function sigmoid function defined in the equation.

[0096] Let the current time be k, then the input data samples from the previous time step are:

[0097] {H(k),ΔP(k),...,H(kN),ΔP(kN)}

[0098] Where H(k) represents the historical speed of the train at time k, and ΔP(k) represents the historical position of the train. Due to the special characteristics of rail transit, the position of the train should be the longitudinal position, and the horizontal position is not considered. The predicted output is the position of the train at the future time M {ΔP(k+M),...,ΔP(k+M)}.

[0099] like Figure 4 As shown in the LSTM prediction model section, the train's running data for the previous N time steps is taken as input, and the predicted running position of the train for the next M time steps is taken as output. Through a multi-layer LSTM network, train trajectory prediction within a long time window can be achieved. Based on the output train position data, the node closest to the train can be determined as the final destination node in the inter-cluster routing step state.

[0100] Based on the routing scheme in step 2.1, step 2.3 further constructs a multi-objective Markov decision model.

[0101] The routing decision process can be viewed as a Markov decision process (MDP), consisting of tuples.<S,A,R,γ> This is represented by the formula. Here, S, A, R, and γ represent the state, action, reward, and discount factor, respectively. In a task, a response action is selected based on the current state. After execution, the routing network provides feedback on the next state and the corresponding reward. This process is repeated until the data packet is successfully transmitted to its destination.

[0102] 1) Cluster Markov decision model;

[0103] Intra-cluster routing is the process by which a source node transmits data packets to the cluster head through intermediate nodes within the same cluster, with the aim of finding the optimal transmission path. The Markov decision model for this process is defined as follows:

[0104] State space S: To represent all possible states in the environment, the state space of a data packet at any given time within the cluster is defined as follows: Where V cur (t) represents the forwarding node where the data packet is located at time t, i.e., the node closest to the sender; V des (t) represents the destination node of the route, i.e., the destination cluster head.

[0105] Action space A: It represents the set of all possible actions. V next (t) represents the next-hop forwarding node at time t.

[0106] Reward R: Packet routing relies on the reward to evaluate its decisions, infer state distributions, and learn about the environment. Based on the three optimization objectives of latency, throughput, and energy consumption, and the established routing model, the reward for the multi-objective Markov decision model is defined as follows:

[0107]

[0108] Where, r dl (t) and r ec (t) represents the time delay reward and energy consumption reward for a single hop, r tp (t) represents the average network throughput of the route. Since the three optimization objectives simultaneously require maximizing and minimizing reward vectors, the reward is designed as a three-dimensional vector: Their corresponding weights are w = [w dl ,w tp ,w ec The total reward is...

[0109] 2) Inter-cluster Markov decision model;

[0110] The inter-cluster routing process is the second stage of path selection. The intra-cluster routing endpoint from the previous stage acts as the initial forwarding node, sending data packets to the destination cluster head (i.e., the route endpoint predicted by LSTM).

[0111] State space S': The current state is still the state of the current node and the destination node. Unlike intra-cluster processes, the inter-cluster network state space is defined as: S' = {[U cur (t),U des (t)]}。where U cur (t) represents the current forwarding node, determined by the destination node of the inter-cluster route; U des (t) represents the target node, which is determined by the LSTM prediction results in step 2.2.

[0112] Action space A': Defines the action space for inter-cluster routing as: A' = {U next (t)}, where U next (t) represents the cluster head node for the next hop forwarding.

[0113] Reward R': Consistent with intra-cluster routing, the comprehensive reward for inter-cluster routing is the product of a three-dimensional reward vector and its weights.

[0114] Based on the optimization process in step 2.1, and combined with the LSTM prediction model and multi-objective Markov decision model constructed in steps 2.2 and 2.3, step 2.4 presents a routing solution based on prediction and deep reinforcement learning, thus completing intelligent route selection.

[0115] First, based on the AC framework and experience replay mechanism of DDPG, the critic network Q(s,a|θ) is initialized. Q ), actor network π(s|θ) π ) and its model parameters θQ and θ π They constructed their target network Q'(s,a|θ) Q' ), π'(s|θ π' ) and its model parameters θ Q' and θ π' Simultaneously, the experience replay pool D needs to be initialized to store the experience data obtained by the agent. Based on the optimization objective and the three-dimensional reward R, the corresponding initial weights w = [w...]. dl ,w tp ,w ec Since deterministic policies output deterministic actions, they lack exploration of the environment. During the training phase, noise needs to be added to the actions output by the Actor network to give the agent some exploratory ability. A random process N is initialized for action exploration.

[0116] Then, the initial observation state s1 is obtained from the environment and input into the LSTM network. The LSTM prediction model from step 2.3 is used to predict the future trajectory of the train, and the node closest to the predicted train location is calculated as the destination node for the route. The LSTM output is used as the input to DDPG to update the state of the environment.

[0117] Randomly select an action a based on the current strategy and exploration noise. t =π(s) t |θ π )+N t Execute action a t The environment will provide an immediate reward R for this action. t And the new status Will The samples are stored in an experience pool O. The neural network is trained using random samples from the experience pool. The formula for the target critic network is as follows:

[0118]

[0119] Where γ represents the discount factor. Then, the network models for the critic and actor are updated by minimizing the critic network error and maximizing the actor network error, with the error formulas shown below:

[0120] L(θ Q )=E[(y i -Q(s i ,a i |θ Q )) 2 ]

[0121] L(θ Q )=E[Q(si ,π(s|θ π )|θ Q )]

[0122] The update formulas for the Actor and critic target networks are as follows:

[0123] θ Q' ←τθ Q +(1-τ)θ Q'

[0124] θ π' ←τθ Q +(1-τ)θ π'

[0125] By iterating through the above process until the reward converges, the optimal routing scheme can be obtained.

[0126] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0127] (1) Based on the shortcomings of existing research, this invention proposes an intelligent routing method for vehicle-track ad hoc networks based on motion prediction. In the scenario of high-speed train operation, the route destination is determined by predicting the future position of the train using LSTM, which effectively reduces the situation where the train has already left the current position after the routing is completed, and reduces the unnecessary overhead caused by route retransmission.

[0128] (2) This invention proposes a routing solution based on prediction-based multi-objective deep reinforcement learning. By combining trajectory prediction with artificial intelligence algorithms, a multi-objective optimization model is theoretically analyzed and established, which effectively optimizes performance such as latency, throughput, and energy consumption while achieving optimal routing.

[0129] (3) The present invention compares the designed scheme with the classic routing scheme, and the simulation results show the superiority of the present invention. Attached Figure Description

[0130] Figure 1 The network model diagram involved in this invention;

[0131] Figure 2 The core parameter symbols used in this invention;

[0132] Figure 3 The LSTM cell diagram in the prediction model used in this invention;

[0133] Figure 4 The framework diagram of the prediction-based multi-objective deep reinforcement learning algorithm proposed in this invention;

[0134] Figure 5Performance evaluation of the present invention: transmission latency of each model under different numbers of nodes;

[0135] Figure 6 Performance evaluation of this invention: throughput of each model under different numbers of nodes;

[0136] Figure 7 The performance evaluation of this invention: energy consumption of each model under different numbers of nodes. Detailed Implementation

[0137] The following section analyzes and compares the performance of the method involved in this invention based on simulation results.

[0138] (1) Simulation settings:

[0139] The simulation environment is Python 3.6. The PC configuration parameters are: Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz 2.11GHz; 16.00GB RAM; and a 64-bit operating system based on the x64 processor, running Windows 11.

[0140] The simulation environment is set as a 400m*40m rectangular area, divided into eight 100m*20m grids, each grid representing a cluster. A high-speed train, 180m long, has several self-organizing network nodes evenly and randomly deployed along the track. Considering the integrated service information and QoS requirements of the Train Autonomous Circumvention System (TACS) based on vehicle-to-vehicle communication, the size of the data packets to be transmitted is set to 1 Mbits. The channel bandwidth of each link is assumed to be 2.5MHz, the transmission power of the transmitting node is 0.5W, the channel gain is -50dB, and the noise power of the receiver is -100dB.

[0141] To demonstrate the superior performance of this invention, the proposed scheme was compared with unpredictable solutions and existing routing models. In the simulation, "PMO-DDPG," "MO-DDPG," and "GPSR" were used to represent these three schemes, respectively. "PMO-DDPG" and "MO-DDPG" are clustering and deep reinforcement learning-based routing solutions, which can select the optimal route by training a neural network. "GPSR" is a classic location-based routing protocol, where routing nodes forward data along straight lines as much as possible based on their geographical location and unified addressing using a greedy algorithm. Compared with GPSR routing, the proposed scheme can effectively reduce the complexity of the routing network, thus achieving more ideal results.

[0142] (2) Performance Analysis:

[0143] Figure 5 The transmission latency of each model varies with the number of nodes. Ignoring the latency consumed by the interaction between onboard and trackside routing, it can be seen that as the number of nodes in the scenario increases, the network topology becomes more complex, and the transmission latency gradually increases. Secondly, the intelligent model proposed in this paper effectively reduces data packet transmission latency compared to the classic GPSR model, and the LSTM-predicted scheme outperforms the unpredicted scheme because LSTM prediction effectively avoids the positional errors caused by high-speed trains.

[0144] Figure 6 The throughput of each model is shown for different numbers of nodes. The results demonstrate that, compared to classic routing schemes, the proposed solution effectively improves the throughput of ad hoc network routing. Regardless of the complexity of the network topology, the network performance of this solution is still superior to the other two schemes. This means that the model is versatile and can be widely applied to complex wireless network environments.

[0145] Figure 7 The energy consumption for transmitting the same-sized data packets to different models is shown. As the number of nodes increases, so does the energy consumption, due to the increased number of transmission hops. Furthermore, as the results show, using a prediction-based multi-objective deep reinforcement learning method can effectively reduce unnecessary energy loss.

Claims

1. A vehicle-track ad hoc network intelligent routing method based on motion sensing, characterized in that, Includes the following steps: Step 1: Analysis of the self-organizing network scenario and construction of the basic model for rail transit; First, a clustered routing scenario model was established based on the operational characteristics of rail transit. The trackside nodes were grouped, and nodes with strong transmission capabilities were elected as cluster heads to communicate directly with the train instead of ordinary nodes. The process of nodes forwarding data packets to the train was divided into two stages: intra-cluster and inter-cluster. Then, a self-organizing network transmission model for rail transit was established, and the calculation formulas for latency, throughput, and energy were derived based on Shannon's formula and the wireless power consumption formula. Finally, minimizing the latency and energy consumption of the routing process and maximizing the throughput were defined as the collaborative optimization objectives. Step 2: Construct an intelligent routing optimization scheme based on mobility prediction; First, the optimization process of routing is discussed based on the scenario model in step 1. Then, an LSTM prediction model and a multi-objective Markov decision model are established. The routing training process is completed using a multi-objective deep reinforcement learning scheme based on mobility prediction, and its performance is tested and theoretically analyzed. The implementation process of step 1 is as follows: Step 1.1: Analyze and build the routing model; Based on geographical location and the performance of the nodes themselves, the routing nodes along the track are divided into multiple clusters; the scenario model includes three participants: cluster head, cluster members, and train; the routing process includes two parts: intra-cluster routing and inter-cluster routing. Step 1.2: Analyze and construct a clustering model; For the routing scenario proposed in step 1.1, clustering models are established from two perspectives: undeployed and deployed. Step 1.3: Analyze and construct the transmission model; Based on the clustered routing model established in steps 1.1 and 1.2, a network transmission model is constructed according to Shannon's theorem, and end-to-end delay, total routing delay, end-to-end throughput, and network throughput are defined. Step 1.4: Analyze and construct a first-order wireless power consumption model; Establish an end-to-end energy consumption model for first-order radio waves, as well as the total energy consumed during the routing process; Step 1.5: Analyze and construct optimization objectives; Based on the routing scenario, namely the demand of urban rail transit, and the model construction in steps 1.3 and 1.4, three comprehensive optimization objectives are established: latency, throughput, and energy consumption. In step 1.2, the analysis and construction of the clustering model includes two processes: cluster head election and cluster formation. Based on the analysis of train travel characteristics and the layout of the trackside ad hoc network, the following two clustering scenarios will be provided according to the deployment: Scenario 1: When no self-organizing network nodes are deployed on the track, a clustering deployment scheme based on transmission power is used. The aim is to save transmission costs in the complex network structure by intermittently deploying nodes with high power transmission as cluster heads during the deployment phase of the self-organizing network. First, the site is evenly divided according to the grid method. Each grid represents the scope of a cluster; high-power transmission nodes are deployed intermittently in each cluster as cluster heads, and cluster heads communicate with each other; after selecting a cluster head, low-power nodes are evenly deployed in each grid as cluster members for data transmission within the cluster; Scenario 2: For nodes that are already evenly deployed on both sides of the track, a weighted clustering scheme based on K-Means is used to select the optimal cluster head based on the node's mobility, node degree difference, location and power consumption. Each node then chooses to join the cluster head closest to it. It is the degree difference between the nodes; the nodes The number of neighboring nodes is the number of nodes within its transmission coverage area, denoted as density. ;in, Indicates transmission distance; This represents the number of nodes processed by the cluster head, and the density difference of each node is... ; This represents the mobility of a node, used to calculate the relative speed between the node and the train, and the formula is: ;in, The train's speed. For nodes Angle with the direction of train travel.

2. The intelligent routing method for vehicle-track ad hoc networks based on mobility sensing according to claim 1, characterized in that, The implementation process of step 2 is as follows: Step 2.1: Discuss routing solutions based on prediction and deep reinforcement learning; For the clustering model in step 1, the entire communication process is divided into four modules: environment, LSTM prediction, intra-cluster intelligent routing, and inter-cluster intelligent routing, which work together to complete the routing process. Step 2.2: Construct the LSTM prediction model; First, construct the LSTM prediction module in step 2.1 and establish an LSTM network model to simulate the train's trajectory. Take the train's historical position and speed as input and the train's position at future time as output. The position of the train can be predicted through the LSTM network. Step 2.3: Construct a Markov decision model; To construct the intelligent routing module in step 2.1, a multi-objective Markov decision model within and between clusters is established, combining the routing model and optimization objective from step 1. Step 2.4: Utilize a deep reinforcement learning model to complete intelligent routing; Based on the construction of each module of the routing scheme in steps 2.2 and 2.3, a deep reinforcement learning model was established to realize the LSTM-cooperative multi-objective DDPG routing scheme; the policy gradient algorithm based on deep determinism maximizes the cumulative expected reward of the routing process, thereby completing the optimization of the objective.

3. The intelligent routing method for vehicle-track ad hoc networks based on mobility sensing according to claim 1, characterized in that, Step 1, "Scenario Analysis and Basic Model Construction of Urban Rail Transit Self-Organizing Network," is used to analyze the architecture of urban rail transit self-organizing networks, and provides a cluster-based scenario model, transmission model, and optimization objectives. Step 1.1: In analyzing and constructing the routing model, a cluster-based vehicle-track ad hoc network scenario model is established; trains are mobile nodes that travel along the track on fixed schedules, and deployment is carried out alongside the track. A wireless ad hoc node; the nodes in the wireless ad hoc network are evenly divided into multiple clusters, each cluster consisting of a cluster head node and ordinary member nodes; in the vehicle track routing system, each cluster uses a set express, The number of clusters is represented by a set, and the cluster head node in each cluster is represented by a set. Representation; ordinary member nodes are represented by sets. This indicates that the cluster head's transmission capacity is much greater than that of its cluster members, and data is directly transmitted wirelessly between two cluster heads; the network topology within the cluster is defined as an undirected graph. It means that, among them The set of all routing nodes within the cluster. This represents the set of wireless links between nodes within a cluster; The inter-cluster network topology consists of an undirected graph. Representation; Definition It is the set of all cluster heads in the network; This represents the set of wireless links between cluster heads. This represents the set of direct links that exist between cluster heads; When a source node intends to send a data packet to a vehicle, it first forwards the information to the cluster head of its own cluster via intra-cluster routing. Then, the cluster head sends the data packet to the destination cluster head closest to the train via inter-cluster routing. Finally, the destination cluster head sends the data to the train.

Images