LLM AI Agent MCTS Action Search Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI agents lack efficiency in performing tasks as they inefficiently search the entire space of possible actions for every single action, consuming significant power and computational resources.
Innovation Solution
The implementation of a Monte Carlo Tree Search (MCTS) framework within a Large Language Model (LLM) based AI agent, which rapidly explores and exploits the space of possible actions by using a score associated with each action to balance exploration and exploitation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If AI agents search the entire space of possible actions for every single action, then they can find optimal actions, but they consume significant power and computational resources
Solution Approach 1:
The patent segments the action space into multiple discrete actions that can be evaluated independently. Instead of searching the entire space as a monolithic problem, the MCTS framework divides it into individual action nodes that can be scored and prioritized separately, reducing the computational burden of exhaustive search.
Solution Approach 2:
The patent assigns scores to actions based on preliminary evaluation criteria before full execution. The MCTS algorithm performs preliminary exploration and scoring of potential actions, allowing the system to prioritize high-value actions and avoid exhaustive search of low-probability options, thereby reducing computational resource consumption.
2Reliability
If AI agents search the entire space of possible actions, then they can ensure thorough exploration, but they lose time and reduce efficiency
Solution Approach 1:
The patent implements feedback mechanisms where the AI agent receives performance feedback after executing actions. This feedback is used to update the scoring function in MCTS, allowing the agent to learn from past experiences and improve future action selection efficiency without requiring exhaustive search each time.
Solution Approach 2:
The patent dynamically adjusts parameters in the action scoring function based on observed performance and environmental feedback. By changing the weighting parameters and evaluation criteria adaptively, the system can balance exploration and exploitation, reducing the time needed to find effective actions while maintaining task completion accuracy.
3Adaptability or versatility
If AI agents use traditional machine learning systems, then they can perform basic tasks, but they lack efficiency in performing actions on environment
Solution Approach 1:
The patent introduces MCTS as an intermediary layer between the LLM and the environment. This intermediary framework translates the LLM's action suggestions into optimized action sequences by applying scoring and selection mechanisms, thereby improving the efficiency of action execution while maintaining the versatility of the underlying language model.
Data Source
AI summary
Embodiments described herein provide a large language model (LLM) based AI agent that adopts Monte-Carlo Tree Search (MCTS) to execute a task. The LLM is prompted with a task description and it responds with its first attempted list of actions. Based on the success or failure of the first attempt, the LLM is prompted with an updated prompt which includes feedback from the first attempt based on a determined reward. The prompt may include a relative “score” for each action taken at each step. A numeric score may be mapped to a set of pre-defined text labels, such as “high” or “low” value putting the score in a form more suited for an LLM prompt. In this way, the LLM is iteratively given prompts which are updated with the scores from each action taken at each previous iterations so that it traverses different paths on the tree in each iteration.