Reinforcement Learning for Relay Positioning in Mobile Networks

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing mobile relay beamforming networks face challenges in determining optimal relay positions due to the impossibility of obtaining future Channel State Information (CSI) in spatiotemporally varying channels.

Innovation Solution

The use of reinforcement learning, specifically through neural networks, to estimate state-action value functions and determine displacement actions for relays, allowing them to move to positions that maximize the cumulative Signal-to-Interference+Noise Ratio (SINR) at the destination.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If predictive fashion is used to estimate future optimal relay positions, then relay positioning performance is improved, but system complexity increases due to requiring full knowledge of CSI statistics

Engineering Contradiction:
Improverelay positioning accuracyVSAvoidsystem complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The relay network performs self-learning through reinforcement learning, where each relay independently learns optimal positioning policies through trial and error without requiring external provision of CSI statistics. The system serves itself by converting the complex task of acquiring CSI statistics into a learning process that automatically adapts to environmental characteristics.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent replaces the traditional mechanical approach of explicitly acquiring and processing CSI statistics with a neural network-based reinforcement learning system. The neural network learns positioning policies directly from environmental interactions, substituting the complex information processing mechanism with a learning-based approach that achieves similar or better performance without requiring explicit CSI statistics.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If full knowledge of CSI statistics is obtained, then optimal relay positions can be determined, but substantial overhead is required in dynamic environments

Engineering Contradiction:
Improvecommunication reliabilityVSAvoidoverhead time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The neural network performs preliminary learning during idle periods or offline phases, accumulating knowledge about the environment and optimal positioning strategies. This preliminary action allows the system to make rapid positioning decisions during actual communication without requiring real-time acquisition of CSI statistics, thus reducing overhead time while maintaining reliability.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The reinforcement learning framework implements continuous feedback loops where the neural network receives feedback from positioning outcomes and channel conditions, continuously refining its policies. This feedback mechanism allows the system to adapt to changing environmental conditions without requiring substantial overhead for explicit statistics collection, as the learning is integrated into the normal operation.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS12301318B2Reinforcement learning for motion policies in mobile relaying networks
Publication Date: 2025.05.13 JERSEY RUTGERS THE STATE UNIV OF N
  • US12301318B2 patent drawing
  • US12301318B2 patent drawing
  • US12301318B2 patent drawing

AI summary

Various embodiments comprise systems, methods, architectures, mechanisms or apparatus for determining a subsequent time slot position for each of a plurality of spatially distributed relays configured for time slot based beamforming supporting a communication channel between a source and a destination.