Spectrum resource and computing resource joint allocation method based on reinforcement learning

Abstract

The invention discloses a spectrum resource and computing resource joint allocation method based on reinforcement learning, and belongs to the technical field of wireless communication. Wherein reinforcement learning theories are utilized, a URLLC user and an eMBB user in the MEC system are used as agents to learn a distributed autonomous learning spectrum resource and computing resource joint allocation strategy; the purpose of minimizing the total cost of time delay and energy consumption of all URLLC users and eMBB users of the system while ensuring strict time delay constraints of the URLLC users is achieved; meanwhile, a reinforcement learning model for joint allocation of spectrum resources and computing resources of URLLC users and eMBB users is established, an overall optimal solution can be obtained by reasonably designing an action space and a return function, and good training convergence performance is achieved. According to the method, a distributed resource allocation algorithm is designed, each intelligent agent independently maintains one Q value table, action selection is carried out according to own criteria, so that the overall dimension of the Q value table is relatively low, and relatively low algorithm complexity is realized.

Description

technical field [0001] The invention belongs to the field of wireless communication, and relates to joint allocation of spectrum resources and computing resources, and in particular to a method for joint allocation of spectrum resources and computing resources of URLLC users and eMBB users in an MEC network. Background technique [0002] The International Telecommunication Union (ITU) clarified at its 22nd meeting that 5G networks mainly cover three application scenarios: ultra-reliable and low-latency communication (URLLC), enhanced mobile broadband (eMBB) and massive machine communication (mMTC). According to the 5G development strategy, the coexistence of URLLC and eMBB services will be a common 5G scenario. The eMBB service has a large amount of data and a high data rate. The URLLC service has low latency, high reliability, and high priority. URLLC users and eMBB users will inevitably compete for various resources in the system, including spectrum resources and computi...

AI Technical Summary

Problems solved by technology

The centralized resource allocation method assumes that the base station has real-time global channel state information, and the base station controls the allocation of spectrum resources and computing resources for all users. However, the base station needs huge signaling overhead to obtain global channel state information. In the future massive wireless device scenarios Under the circumstances, it is difficult for the base station to have real-time global information

The distributed resource allocation method is mainly completed based on game theory and decomposition technology. The game theory method models users as game players for competitive games until the Nash equilibrium state, but solving the Nash equilibrium state requires a large amount of information exchange between users, and requires It takes a lot of iterations to converge

Decomposition technology can be used to design a heuristic greedy algorithm to optimize resource allocation and unloading decisions in turn. After decomposition, repeated iterations are still required, and the complexity of the algorithm is very high.

Existing related research mainly focuses on optimizing the offloading strategy and spectrum resource allocation, assuming that the computing resources at the MEC server are evenly distributed to each user, ignoring the benefits brought by the joint allocation of spectrum resources and all computing resources

Images