LLM Multi-Agent Planning for Cost-Efficient ML Task Solving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Large Language Model (LLM) systems are limited in handling complex machine learning tasks due to their reliance on expensive models like GPT-4 and lack of flexibility in exploring out-of-distribution solutions, while AutoML approaches offer limited flexibility and scalability, making them inefficient for nuanced problem-solving.
Innovation Solution
A multi-agent system utilizing a combination of less expensive LLMs like Gemini-Pro and GPT-4, with cascaded invocation based on structured prompts and historical context retrieval, to efficiently tackle complex ML tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If single-agent systems use larger and expensive models such as GPT-4, then task-solving capability is improved, but cost increases significantly
Solution Approach 1:
The system divides the task-solving process into multiple specialized agents (planner agent, writer agent, reviewer agent, etc.), each handling specific subtasks. This segmentation allows using smaller, more cost-effective models for each agent while collectively achieving complex task resolution that previously required single expensive models.
Solution Approach 2:
The planner agent acts as an intermediary that coordinates between different specialized agents (writer, reviewer, executor). This intermediary structure enables cost-effective smaller models to work together systematically, replacing the need for a single expensive model to handle all aspects of task solving.
2Extent of automation
If AutoML-based approaches are used for automating ML tasks, then automation is improved, but flexibility is limited due to predefined constraints
Solution Approach 1:
The system employs dynamic prompting where the planner agent generates adaptive prompts based on task requirements and historical interactions. This dynamic approach allows the system to automatically adapt to different ML tasks and scenarios without being constrained by predefined templates, combining automation with flexibility.
Solution Approach 2:
The system changes the parameters of interaction by using structured prompts that can be dynamically adjusted based on task complexity and agent capabilities. This allows the automated system to flexibly modify its behavior and approach for different ML tasks while maintaining high automation levels.
3Ease of operation
If LLMs are used for complex nuanced tasks, then problem-solving capability is improved, but cost increases due to reliance on expensive models
Solution Approach 1:
Complex nuanced tasks are segmented into multiple subtasks handled by specialized agents (planner, writer, reviewer, executor). Each agent uses smaller, more cost-effective LLMs for its specific function, collectively achieving complex problem-solving without requiring expensive models for every operation.
Solution Approach 2:
The reviewer agent provides feedback on the work produced by the writer agent, and the planner agent adjusts prompts based on interaction outcomes. This feedback mechanism enables cost-effective smaller models to iteratively improve their output quality, achieving nuanced problem-solving capability without relying on expensive models.
Data Source
AI summary
Large Language Models (LLMs) excel in diverse applications including generation of code snippets but often struggle with generating code for complex Machine Learning (ML) tasks. Although existing LLM single agent-based approaches give varying performance depending on the task complexity, they purely rely on larger and expensive models such as Generative Pre-Trained Transformers (e.g., GPT-4). Present disclosure implements a LLM multi-agent-based system which leverages combination of experts using profiling, efficient retrieval of past observations, LLM cascades, and ask-the-expert calls to enable obtain task specification, actions to be performed along with associated justification and reasoning based on which appropriate planner(s) invoke respective agents for execution of a machine learning task.