LLM Agent Network Troubleshooting via Reinforcement Learning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing LLM-based agents for network troubleshooting do not learn from past experiences and require very capable, large models due to their reliance on zero-shot capabilities.

Innovation Solution

The implementation uses reinforcement learning to train an LLM-based agent to interact with a network, identify the root cause of issues, and solve them, allowing the agent to learn from experiences and improve over time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If zero-shot capabilities are used for LLM-based troubleshooting agents, then the agents can perform tasks without training, but they require very large and capable models with strong reasoning and coding abilities

Engineering Contradiction:
ImproveEase of deploying troubleshooting agentVSAvoidModel size
Core Design Contradiction:
Ease of manufactureVSQuantity of substance

Solution Approach 1:

The patent applies preliminary action by pre-training the LLM-based agent using reinforcement learning before deployment. The agent is trained on a dataset of network troubleshooting scenarios, allowing it to learn effective troubleshooting strategies in advance. This pre-training phase enables the agent to perform complex troubleshooting tasks without requiring an extremely large model architecture, as the capabilities are acquired through learning rather than inherent model size.

Inventive Principle:
Principle #10Preliminary action

2Ease of manufacture

If zero-shot capabilities are used for LLM-based troubleshooting agents, then no training is required, but the agents do not learn from past experiences and have the same likelihood of succeeding on similar questions

Engineering Contradiction:
ImproveEase of deploying troubleshooting agentVSAvoidSuccess rate on troubleshooting tasks
Core Design Contradiction:
Ease of manufactureVSReliability

Solution Approach 1:

The patent implements feedback mechanisms through reinforcement learning where the agent receives rewards or penalties based on the success or failure of its troubleshooting actions. The system evaluates the agent's performance on troubleshooting tasks and uses this feedback to update the agent's policy through reinforcement learning. This closed-loop feedback enables the agent to learn from past experiences and continuously improve its success rate on similar troubleshooting questions over time.

Inventive Principle:
Principle #23Feedback

3Reliability

If reinforcement learning is used to train the LLM-based agent, then the agent learns from past experiences and improves over time, but the training process and evaluation framework are required

Engineering Contradiction:
ImproveSuccess rate on troubleshooting tasksVSAvoidSystem complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent introduces an evaluation framework as an intermediary component that bridges the gap between the LLM-based agent and the reinforcement learning training process. This evaluation framework assesses the agent's troubleshooting performance, generates feedback signals, and enables the reinforcement learning algorithm to update the agent's policy. By separating the evaluation function from the agent itself, the system manages complexity while enabling continuous learning and improvement.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS20250158895A1Teaching LLM-based agents to troubleshoot networks using reinforcement learning
Publication Date: 2025.05.15 CISCO TECHNOLOGY INC
  • US20250158895A1 patent drawing
  • US20250158895A1 patent drawing
  • US20250158895A1 patent drawing

AI summary

In one implementation, a device uses a large language model-based agent to complete a task with respect to a computer network. The device makes an evaluation of an action taken by the large language model-based agent to complete the task. The device selects, based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task. The device updates the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.