Research Copilot Prompting for Long-Running Multi-LLM Sessions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) are limited by linear chat sessions and context constraints, which hinder their ability to support complex, branching research activities that require multiple participants, extensive collaboration, and long-term knowledge retention.
Innovation Solution
A science platform integrating a copilot engine that infers user intents and provides prompts to LLMs, allowing for long-running chat sessions and generating responses using multiple LLMs or internal knowledge representations, supporting multistep queries and scientific questions with data integration from various sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLM-backed chat applications support linear chat sessions with fixed number of turns, then the system complexity is low and implementation is simple, but the ability to support complex branching research activities is limited
Solution Approach 1:
The system segments the chat session management into multiple independent components: intent classification module, plan generation module, task decomposition module, and execution module. This allows complex research activities to be handled through coordinated specialized components rather than a monolithic system, resolving the contradiction between versatility and complexity.
Solution Approach 2:
The system introduces an intermediary planning layer between user input and LLM response generation. This planner component translates high-level research intents into structured multi-step plans, enabling complex branching research activities without requiring the LLM itself to handle all complexity directly.
2Duration of action of stationary object
If the LLM context is limited to the number of tokens it can support, then memory usage is controlled, but long-term knowledge retention and long-running sessions are hindered
Solution Approach 1:
The system extracts and separates long-term context storage from the LLM's immediate context window. Session history and knowledge are stored externally in databases, with only relevant portions retrieved and injected into the LLM context when needed. This allows indefinite session duration without being constrained by token limits.
Solution Approach 2:
The system performs preliminary retrieval of relevant context and knowledge from external storage before presenting it to the LLM. This pre-fetching and filtering of information ensures the LLM receives optimized context within token limits while maintaining access to extensive long-term knowledge through external repositories.
3Productivity
If the LLM generates outputs based only on the prompt provided, then the response generation is straightforward, but the ability to provide detailed plans and visualizations for multistep queries is limited
Solution Approach 1:
The system performs preliminary plan generation and task decomposition before LLM response generation. The planner component breaks down multistep queries into structured action sequences with intermediate goals, allowing the LLM to focus on generating specific content for each step rather than figuring out the entire workflow from scratch.
Solution Approach 2:
The system introduces intermediary components including a plan generator, task decomposer, and visualization engine that bridge between user queries and LLM outputs. These intermediaries transform simple prompts into detailed structured plans with multiple steps, enhancing research productivity without overloading the LLM with complex processing requirements.
Data Source
AI summary
The present disclosure relates to methods and systems for using large language models to support research activities. The methods and systems include a copilot engine that creates input prompts to provide to the large language model to use in generating responses to input messages. The copilot engine infers an intent of the input messages and sends the intent with the input message in the input prompt to the large language model. The large language model generates different types of responses for different intents.