User Equipment Association in 5G Networks via Transferable Meta Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for associating user equipment with base stations in 5G heterogeneous networks face scalability issues due to high complexity and the need for frequent relearning when users join or leave the network, leading to inefficiencies in resource allocation and network performance optimization.
Innovation Solution
A multi-agent reinforcement learning method that uses a meta model for initializing an association strategy, allowing user equipment to project local and global observation vectors into a shared reference space, combine them, and update the association policy using a policy gradient method, enabling adaptive and transferable association policies across network scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a multi-agent reinforcement learning method with Q-learning is used for user association, then the association policy can be learned adaptively, but the system complexity increases and scalability deteriorates when the number of users grows
Solution Approach 1:
The patent segments the user association problem by introducing a coordinating entity that centralizes the reinforcement learning process. Instead of each user equipment independently performing complex Q-learning, the system divides the computational burden by having the coordinating entity manage the neural network and association decisions, while user equipment only needs to transmit observation vectors and receive association directives.
Solution Approach 2:
The patent introduces a coordinating entity as an intermediary between user equipment and the reinforcement learning process. This intermediary receives observation vectors from user equipment, processes them through the neural network to determine optimal associations, and returns association decisions. This mediator architecture simplifies the system by centralizing complex computations and reducing the computational requirements at user equipment.
2Adaptability or versatility
If each user equipment independently learns its association policy using reinforcement learning, then adaptability is improved, but the time required for relearning when users join or leave increases
Solution Approach 1:
The patent implements preliminary action by pre-training a shared neural network model that captures general association policies applicable across different network scenarios. When new users join or existing users leave, the system can quickly adapt by fine-tuning this pre-trained model rather than starting from scratch, significantly reducing relearning time while maintaining adaptability to new conditions.
Solution Approach 2:
The patent merges the association policy learning process by using a single shared neural network model that serves all user equipment. Instead of each user maintaining separate learned policies, the system combines knowledge into a common model that can be efficiently updated and transferred across different network configurations, reducing overall relearning time when network topology changes.
3Measurement precision
If the input variables of the neural network increase with the number of users, then the association accuracy is improved, but the scalability of the system deteriorates
Solution Approach 1:
The patent applies dimensionality change by transforming the high-dimensional input space (which would scale with the number of users) into a lower-dimensional latent representation space through the neural network's hidden layers. The coordinating entity processes observation vectors through this compressed representation, allowing the system to maintain association accuracy while achieving scalability, as the computational complexity no longer directly scales with the number of users.
Data Source
AI summary
A method for associating user equipment with base stations of a cellular network, in particular of a heterogeneous network such as a 5G network, uses a multi-agent reinforcement learning (MARL) algorithm. When a user equipment enters the network, the agent associated with this equipment downloads the instance of an association meta model to a coordinating unit, pre-trained by means of reinforcement learning. The agent subsequently constructs a local observation vector, comprising observables relating to the user equipment, and a global observation vector, including observables relating to the environment of this equipment. The local and global observation vectors are projected into the same reference space, the concatenation of the vectors thus projected encoding the state of the agent. It deduces from the model and from this state, an association decision. The model is updated according to a policy gradient method to consider evolution of the environment of the user.