Dynamic spectrum access method based on policy planning constrain Q study

Abstract

The invention provides a dynamic spectrum access method on the basis that the policy planning restricts Q learning, which comprises the following steps: cognitive users can divide the frequency spectrum state space, and select out the reasonable and legal state space; the state space can be ranked and modularized; each ranked module can finish the Q form initialization operation before finishing the Q learning; each module can individually execute the Q learning algorithm; the algorithm can be selected according to the learning rule and actions; the actions finally adopted by the cognitive users can be obtained by making the strategic decisions by comprehensively considering all the learning modules; whether the selected access frequency spectrum is in conflict with the authorized users is determined; if so, the collision probability is worked out; otherwise, the next step is executed; whether an environmental policy planning knowledge base is changed is determined; if so, the environmental policy planning knowledge base is updated, and the learning Q value is adjusted; the above part steps are repeatedly executed till the learning convergence. The method can improve the whole system performance, and overcome the learning blindness of the intelligent body, enhance the learning efficiency, and speed up the convergence speed.

Description

technical field [0001] The invention relates to the field of cognitive radio, in particular to a dynamic frequency spectrum access method. Background technique [0002] Cognitive radio (CR for short) is a new technology to improve spectrum utilization. It can lend frequency bands temporarily unused by licensed users (LU) in some areas to unlicensed users (also It is called cognitive user (CU for short) to improve spectrum utilization. Cognitive radio is an intelligent wireless communication system that can dynamically adjust its transmission parameters accordingly by learning the radio environment. [0003] In recent years, cognitive radio dynamic spectrum access technology has become a hot issue and has received extensive attention. Researchers have done a lot of research on dynamic spectrum access in terms of spectrum utilization, conflict probability, and spectrum utilization fairness. important for complex systems. Considering the dynamic change and irregularity of b...

AI Technical Summary

Problems solved by technology

However, some problems arise when modeling cognitive wireless networks and their application environments. First, as the number of users (agents) in the network increases, the state space of each user increases exponentially, even for the simplest problems. The learning of the network becomes extremely difficult; secondly, the complexity of the cognitive wireless network itself and the fast-changing characteristics of the environment also make the traditional Q-learning incapable of learning speed; finally, the Q-learning algorithm must learn through repeated experiments. , the efficiency of the algorithm is not high, and blind learning in an unknown environment will take certain risks

Images