Multi-agent reinforcement learning-based optimal energy sensing threshold control method and device in distributed cognitive radio networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed cognitive radio networks face challenges in accurately detecting vacant spectrum bands while minimizing interference due to dynamic and uncertain wireless environments, leading to inefficiencies in centralized cooperative spectrum sensing methods.
Innovation Solution
A multi-agent reinforcement learning-based approach is employed to determine an optimal energy sensing threshold by constructing a state space, selecting actions through an actor-critic network model, storing experiences in a replay buffer, and updating the network model to maximize detection probability and minimize false alarms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If centralized cooperative spectrum sensing method is used, then detection accuracy can be improved, but operation costs and bottleneck problems increase
Solution Approach 1:
The patent divides the centralized sensing system into multiple distributed secondary terminals, each independently performing spectrum sensing. This segmentation eliminates the central fusion center bottleneck while maintaining collective detection capability through distributed decision-making, thereby reducing operation costs and device complexity.
Solution Approach 2:
Each secondary terminal autonomously determines its own sensing threshold using reinforcement learning without requiring centralized coordination. The terminals self-organize and adapt to the wireless environment independently, eliminating the need for complex centralized control infrastructure and reducing overall system operation costs.
2Reliability
If sensing threshold is adjusted to maximize detection probability, then primary terminal detection improves, but false alarm probability increases
Solution Approach 1:
The patent dynamically adjusts the sensing threshold parameter based on real-time reinforcement learning from environmental feedback. Instead of using fixed thresholds, the system continuously optimizes the threshold value to adapt to changing wireless conditions, achieving a balance between detection probability and false alarm rate through learned parameter optimization.
Solution Approach 2:
The reinforcement learning mechanism incorporates feedback from sensing outcomes (detection results and false alarms) to continuously refine the sensing threshold. This feedback loop allows the system to learn from past performance and adjust thresholds to maximize detection while minimizing false alarms, resolving the trade-off between these two competing objectives.
3Device complexity
If distributed spectrum sensing is used, then operation costs are reduced, but detection accuracy deteriorates due to dynamic wireless environment
Solution Approach 1:
The patent implements dynamic threshold adjustment through reinforcement learning, allowing each distributed terminal to adapt its sensing parameters in real-time to changing wireless conditions. This dynamic adaptation compensates for the lack of centralized coordination, maintaining detection accuracy despite the distributed architecture and environmental uncertainties.
Data Source
AI summary
A multi-agent reinforcement learning-based optimal energy sensing threshold control method in distributed cognitive radio networks includes: (a) constructing a state space for a network environment including a plurality of primary terminals and a plurality of secondary terminals, the state space including each state about whether the primary terminals are occupied; (b) selecting an action in accordance with a policy by applying partial observation of each secondary terminal to a reinforcement learning-based actor-critic network model by means of an agent, and calculating a reward on the basis of a sensing result of the primary terminals on the basis of the selected action in the environment, the action being a sensing threshold; (c) storing the partial observation, the selected action, the reward, and next observation into a replay buffer as experiences; and (d) updating the actor-critic network model on the basis of the experiences stored in the replay buffer.