Adaptive Route Selection in Wireless Mesh Nodes Using Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In wireless ad hoc networks, the task of setting up an efficient route table for path discovery is challenging due to similar reward values leading to inefficient path selection, especially in dynamic environments where energy consumption and topology are dynamic.
Innovation Solution
An adaptive route selection method using Reinforcement Learning (RL) is proposed, where nodes update their route tables based on a reward function that prioritizes shorter routes with stronger signal strength, and applies feedback reward functions to optimize packet routing, incorporating signal strength indicators and packet loss rates to adjust routing behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional routing protocols are used in wireless ad hoc networks, then route discovery can be implemented, but similar reward values lead to inefficient path selection and increased packet loss ratio
Solution Approach 1:
The patent transforms the routing decision from a static equal-cost selection to a dynamic probabilistic selection based on changing network parameters. The reward value is modified to incorporate signal strength (RSSI) and hop count, creating a differentiated selection criterion that changes with network conditions. This resolves the contradiction by making route selection both reliable (preferring better paths) and efficient (avoiding random selection among similar paths).
Solution Approach 2:
The patent implements a feedback mechanism where the routing decision is based on accumulated reward values from previous transmissions. The system learns from past route performance and adjusts future selections accordingly. This feedback loop prevents packet loss by steering traffic away from previously problematic routes while maintaining efficiency through intelligent path selection based on historical performance data.
2Adaptability or versatility
If reactive routing protocols are used, then on-demand route discovery is achieved, but control overhead and energy consumption increase in dynamic topologies
Solution Approach 1:
The patent maintains pre-computed route tables at each node that contain potential routes to various destinations. When a transmission is needed, the node can quickly select from pre-established routes using the reward-based selection mechanism, avoiding the need for complete on-demand route discovery. This preliminary preparation reduces both energy consumption and latency while maintaining adaptability through the learned reward values.
3Loss of information
If proactive routing protocols are used, then global routing information is available, but power consumption and computational complexity increase due to continuous route updates
Solution Approach 1:
Instead of maintaining complete global routing information through continuous updates, the patent uses partial routing information stored in local route tables. Each node maintains routes to known destinations with associated reward values, updating these tables incrementally based on transmission outcomes rather than continuously. This partial information approach reduces power consumption while sufficient for efficient route selection.
4Loss of time
If reinforcement learning is applied for route selection, then route recovery time improves, but computational constraints at network nodes are exceeded
Solution Approach 1:
The patent simplifies the reinforcement learning computation by changing the state representation to rely on pre-computed reward values stored in route tables rather than complex real-time network state analysis. The Q-value updates use simple arithmetic operations based on feedback from transmission outcomes. This parameter simplification enables fast route recovery while respecting the computational constraints of wireless network nodes.
Data Source
AI summary
The proposal concerns a method of adaptive route selection in a node of a wireless mesh communication network. Such method is based on the Reinforcement Learning-based adaptive routing scheme. The proposal includes the idea of estimating initial reward values for the path discovery phase in dependence of the signal strength of the received signal and the number of hops the received route request message (RREQ) or route reply message (RREP) experienced during traversing in the network. Further improvements concern the idea of a feedback reward calculation for the routing absent the path discovery process as well as an improved method of route selection based on calculation of a temperature parameter for a Gibbs-Boltzmann distribution, which allows a dynamic adjustment of the node selection probabilities. Another improvement concerns a part of the reward function calculation, related to an exponential fallback mechanism on packet loss events.