Reinforcement Learning TCP Optimization in 5G Nodes

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing TCP optimization methods in 5G communications networks are ineffective when conditions change unexpectedly, leading to waste of processing, energy, and time-frequency resources, and result in underperformance due to reliance on static algorithms and configurations that fail to adapt to dynamic network conditions.

Innovation Solution

Implementing a method that uses reinforcement learning to dynamically optimize TCP traffic control in 5G networks, allowing nodes to learn and adapt to changing conditions by initiating a process of optimization based on received indications, which include rules for packet detection and quality of service enforcement, enabling per-user and application-based optimization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If static algorithms and configurations are used for TCP optimization, then device complexity is reduced and ease of operation is improved, but adaptability to changing network conditions deteriorates and productivity decreases

Engineering Contradiction:

Improveadaptability to changing network conditionsVSAvoidcomplexity of optimization process

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements dynamic optimization by transitioning from static TCP algorithms to reinforcement learning-based dynamic adjustment. The system continuously learns and adapts to changing network conditions (traffic patterns, latency, packet loss) by modifying TCP parameters in real-time, making the optimization process responsive to dynamic environmental changes rather than relying on fixed configurations

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The reinforcement learning agent operates autonomously to optimize TCP parameters without requiring manual configuration or intervention. The system self-learns optimal strategies through interaction with the network environment, automatically adjusting congestion control parameters based on observed performance metrics, thereby eliminating the need for complex manual tuning while maintaining high adaptability

Inventive Principle:

Principle #25Self-service

2Productivity

If static optimization methods are employed, then energy consumption is reduced, but productivity and network efficiency deteriorate due to inability to adapt to dynamic conditions

Engineering Contradiction:

Improvenetwork throughput and efficiencyVSAvoidenergy consumption of optimization process

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system dynamically changes TCP parameters (congestion window, slow start threshold, retransmission timing) based on real-time network conditions learned through reinforcement learning. This parameter adaptation allows the system to optimize throughput and efficiency for varying traffic patterns and network states, achieving high productivity while the learning process itself is designed to be computationally efficient

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If per-user and application-based optimization is implemented, then adaptability and network efficiency are improved, but device complexity and difficulty of detecting and measuring increase

Engineering Contradiction:

Improveper-user and application-based optimization capabilityVSAvoidcomplexity of monitoring and control

Core Design Contradiction:

Adaptability or versatilityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent segments the optimization process by implementing separate reinforcement learning agents or specialized handling for different users and applications. Each user or application can receive customized TCP optimization parameters based on its specific requirements and observed behavior, allowing fine-grained adaptability while the modular agent structure manages the complexity of monitoring and control through division of responsibilities

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11785497B2First node, second node, third node, and methods performed thereby, for handling data traffic

Publication Date: 2023.10.10 TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)

AI summary

A method performed by a first node is disclosed herein. The method is for handling data traffic. The first node operates in a communications network. The first node receives an indication from a second node operating in the network. The indication indicates, for a packet forwarding control protocol (PFCP) session, at least one of: i) a first rule for packet detection; and ii) a second rule of enforcement of quality of service. The first node initiates a process of optimization, based on reinforcement learning, of a procedure to control data traffic in the network. The process of optimization is further based on the received indication. A method performed by the second node is also described. The second node receives a first indication from the first node indicating that the first node supports the process of optimization. The second node then sends the indication to the first node.