Action Recommendation Engine for Autonomous Network Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI-based network control systems, particularly those using closed-loop Reinforcement Learning (RL), face challenges such as intensive computations, significant data requirements, and the potential for negative actions due to limitations in reward system design, multiple AI system interactions, and testing limitations, which hinder efficient and safe autonomous network management.
Innovation Solution
The introduction of an Action Recommendation Engine (ARE) that utilizes a Machine Learning (ML) model to classify network states and suggest remediation actions or identify root causes, employing supervised ML or Collaborative Filtering techniques, which require less data and computational intensity compared to traditional RL methods, and operates within a closed-loop feedback system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If Reinforcement Learning (RL) is used for autonomous network control, then the system can learn optimal behaviors through intensive computations and significant data, but the training phase requires excessive computational resources and data before deployment
Solution Approach 1:
The patent pre-trains the RL model offline using historical network data and simulated environments before deployment. This preliminary training phase separates the computationally intensive learning process from the actual network operation, allowing the model to be ready for autonomous control without consuming excessive real-time computational resources during network management.
Solution Approach 2:
The patent uses simulated network environments and digital twins to train the RL model instead of requiring extensive real network data and computations. By creating virtual copies of the network for training purposes, the system can perform intensive computations in a safe, isolated environment without risking actual network operations or requiring proportional real-world resources.
2Reliability
If multiple AI systems are used to check actions against each other, then safety may be improved through majority decision, but the complexity of coordinating multiple systems increases
Solution Approach 1:
The patent introduces a safety layer or intermediary component that acts as a mediator between the RL model and network actions. This safety layer validates proposed actions against predefined safety rules and constraints before execution, providing a simpler alternative to coordinating multiple AI systems while maintaining reliability through a dedicated safety verification mechanism.
3Reliability
If the AI system is tested on a small scale domain, then negative actions may be avoided during testing, but the system may not work properly when deployed to the larger network
Solution Approach 1:
The patent employs dynamic training where the RL model is progressively trained on increasingly complex network scenarios and larger datasets. The training process dynamically adapts from simple to complex situations, allowing the system to learn safe behaviors in controlled environments while gradually developing the capability to handle larger, more complex network deployments through continuous learning and adaptation.
Data Source
AI summary
Systems and methods for recommending actions in a closed-loop system are provided. In one embodiment, an Action Recommendation Engine (ARE) may include a processor and memory configured to store computer programs having instructions that cause the processor to obtain input data pertaining to a state of a network and obtain information regarding one or more historical actions performed on the network. Also, the instructions may cause the processor to utilize a Machine Learning (ML) model for imposing one or more current actions on the network, the one or more current actions selected from the group of procedures consisting of: a) suggesting one or more remediation actions that, when performed, transition the network from a problematic state to a normal state, and b) identifying one or more root causes in response to detecting a transition in the network from a normal state to a problematic state.