User Plane Service Function Placement Using Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The placement of service functions in telecommunications networks is complex due to large topologies, numerous constraints, and dynamic changes in user positions and network topology, requiring fast and optimal decision-making to avoid deteriorated user experience and resource overconsumption.
Innovation Solution
A reinforcement learning method is employed to define a state of the system using matrices such as allocation, service function type, key performance indicator, ordering, latency constraint, and co-location goal matrices, with a reward calculator to determine optimal service function placement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional placement algorithms are used to optimize service function placement, then placement optimality is improved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent transforms the placement problem from a complex combinatorial optimization problem into a classification problem by changing the parameter representation. Instead of directly optimizing placement decisions, the system uses machine learning to classify optimal placements based on trained patterns from historical data, significantly reducing computational complexity while maintaining placement quality
Solution Approach 2:
The system performs preliminary training offline using historical placement data to build trained models. During runtime, the pre-trained models quickly classify optimal placements without requiring complex real-time computations, separating the heavy computational work from the time-critical decision-making process
2Manufacturing precision
If traditional placement algorithms are used to optimize service function placement, then placement optimality is improved, but processing speed deteriorates
Solution Approach 1:
The patent changes the computational approach from iterative optimization to direct classification, transforming the problem from finding optimal solutions through complex algorithms to predicting optimal placements using trained models, thereby achieving fast processing speed without sacrificing placement quality
Solution Approach 2:
The system performs computationally intensive work in advance by training models offline using historical data. The trained models are then deployed for rapid real-time inference, ensuring both high placement optimality and fast processing speed during runtime
3Adaptability or versatility
If service functions are dynamically repositioned to adapt to network changes, then adaptability is improved, but computational resource consumption increases
Solution Approach 1:
The system uses reinforcement learning agents that autonomously learn and adapt to network changes without requiring centralized re-optimization. Each agent independently makes placement decisions based on learned policies, reducing overall computational resource consumption while maintaining high adaptability to dynamic network conditions
Solution Approach 2:
The patent transforms the adaptive re-optimization problem into a pattern recognition problem using machine learning. The system learns from historical data what placements work well under different network conditions, enabling fast adaptive decisions without intensive real-time computations
Data Source
AI summary
A method of reinforcement learning is used for placement of a plurality of service functions at nodes of a telecommunications network. The state of the system is defined by an allocation matrix, wherein each first vector of the allocation matrix corresponds to a respective one of the nodes of the telecommunications network, each second vector of the allocation matrix corresponds to a respective one of the plurality of service functions. Moreover, each cell of the allocation matrix contains a value 1 if the one of the plurality of service functions corresponding to the respective second vector is placed on the one of the nodes of the telecommunications network corresponding to the respective first vector, and otherwise contains a value 0.