IoT Bandwidth Allocation Using Transferable Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing bandwidth allocation policies for IoT applications are environment-dependent and struggle to adapt to changes in network conditions due to the need for extensive sampling in reinforcement learning, leading to inconsistent quality of experience (QoE) in IoT applications.
Innovation Solution
A network monitoring system employing transferable reinforcement learning with a flow embedding module and bandwidth allocation module to establish a policy that maximizes QoE by using a multi-perceptron attention mechanism and action shaping techniques, allowing adaptation to various network environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If reinforcement learning is used to establish bandwidth allocation policy, then the policy can be learned without complex induction, but the policy becomes environment-dependent and cannot cope with changes in the network environment due to requiring a lot of samples
Solution Approach 1:
The patent pre-trains the reinforcement learning agent in a simulated network environment before deploying to real networks. This preliminary training allows the agent to learn optimal bandwidth allocation policies in a controlled setting, reducing the need for extensive sampling in production environments and enabling faster adaptation to network changes.
Solution Approach 2:
The patent creates a simulated network environment that copies the essential characteristics of real network conditions. By training the reinforcement learning agent in this simulated environment, the system can transfer learned policies to real networks, avoiding the need to retrain from scratch when network conditions change.
2Ease of manufacture
If heuristics method is used to establish bandwidth allocation policy, then a specific network environment can be assumed and environment-dependent policy can be established, but when the environment is changed, it is necessary to establish the policy by means of re-induction
Solution Approach 1:
The system performs preliminary training in a simulated environment before deployment to real networks. This pre-training phase establishes the initial policy without requiring time-consuming induction processes, and the learned policy can be directly applied to production networks without re-induction when environment changes occur.
Solution Approach 2:
The patent replaces the mechanical induction process with a data-driven reinforcement learning approach. Instead of using complex mathematical induction to derive policies, the system uses sample-based learning from simulated environments, which is computationally more efficient and does not require re-induction when network conditions change.
3Productivity
If the limited bandwidth of the network is shared by a plurality of devices, then the information having an important influence on the QoE is transmitted slowly, but the validity of all information is not guaranteed at all times due to limited resource
Solution Approach 1:
The reinforcement learning agent continuously monitors network conditions and QoE metrics, using this feedback to dynamically adjust bandwidth allocation decisions. The agent learns from the consequences of its actions and adapts its policy to maximize information transmission speed while maintaining QoE reliability under constrained network resources.
Solution Approach 2:
The system dynamically changes bandwidth allocation parameters based on real-time network conditions and QoE requirements. By adjusting allocation ratios and transmission parameters adaptively, the system optimizes the trade-off between transmission speed and information validity for multiple shared devices.
Data Source
AI summary
The present disclosure provides a network monitoring system including a plurality of sensor devices which transmits data packets and a monitoring server which performs the transferable reinforcement learning on the data packets to establish a bandwidth allocation policy in which a quality of experience (QoE) satisfies a set reference QoE to allocate the bandwidth to the plurality of sensor devices.