Wireless Terminal Policy Sharing for Stable Distributed Transmission Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed resource management in wireless networks faces challenges such as stability and convergence issues, especially when packet delivery with low delay is critical, and existing AI-based solutions struggle with large state spaces and inter-agent behavior prediction.
Innovation Solution
A wireless terminal adjusts a local transmission policy using reinforcement learning, exchanging information with neighboring terminals via D2D links to stabilize learning and achieve efficient resource allocation through a common global policy, leveraging neural networks for decentralized decision-making.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If distributed resource management is implemented where each device decides its own transmission parameters based on local observations, then the complexity of centralized control is reduced and scalability is improved, but stability and convergence issues arise leading to poor system performance
Solution Approach 1:
The patent combines centralized policy coordination with distributed execution. Multiple wireless devices share a common global transmission policy that is updated through collaborative learning, merging individual device decisions into a coordinated system-wide policy that ensures stability while maintaining distributed operation
Solution Approach 2:
The patent implements feedback mechanisms where devices exchange information about their local policy adjustments and observations with neighboring devices. This feedback loop enables the global policy to adapt and converge towards optimal performance while maintaining distributed decision-making
2Adaptability or versatility
If distributed training is implemented on each wireless device, then autonomy and scalability are improved, but stability issues arise and lead to unfair medium utilization
Solution Approach 1:
The patent creates equipotentiality by having all devices operate under the same global transmission policy. This ensures fair medium utilization as all devices follow identical policy rules, preventing any single device from gaining unfair advantage while maintaining distributed autonomy
Solution Approach 2:
The patent segments the learning process into local policy adjustment at each device and global policy coordination through information exchange. This segmentation allows devices to maintain autonomy in local decisions while ensuring overall stability through shared global policy updates
3Adaptability or versatility
If reinforcement learning is used for distributed resource management, then adaptability to dynamic environment is improved, but the curse of dimensionality limits application to models with small number of states
Solution Approach 1:
The patent combines reinforcement learning with a shared global policy framework. By merging individual device learning experiences into a common policy, the system can handle large state spaces that would be intractable for single devices, enabling adaptability to complex dynamic environments
4Ease of operation
If devices learn to predict decisions of other agents in distributed RL, then strategic decision-making is improved, but convergence becomes problematic
Solution Approach 1:
The patent introduces a global transmission policy as an intermediary that mediates between individual device decisions. This intermediary coordinates strategic interactions by providing a common framework that all devices follow, enabling strategic decision-making while ensuring convergence through centralized policy updates
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A wireless terminal (120) comprises a communication interface (123) adapted to transmit data to an access point (110) and a processing circuitry (121) adapted to select for a plurality of data transmissions to the access point (110) a respective data transmission configuration based on an global transmission policy and information about a respective first transmission state of the wireless terminal (120). The processing circuitry (121) is further adapted to determine for the respective data transmission a reward for the data transmission configuration based on the data transmission configuration and information about a second transmission state of the wireless terminal (120), and to adjust a local transmission policy based on one or more of the plurality of selected data transmission configurations and the respective rewards. The communication interface (123) is adapted to exchange information about the adjusted local transmission policy with further wireless terminals (130) for adjusting the global transmission policy.