Copilot Engine for Multi-User Chat Sessions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large language model (LLM) backed chat applications are limited by linear, sequential chat sessions with fixed turn limits, restricted context due to token limitations, and inability to support multi-user, multi-step activities effectively.
Innovation Solution
The implementation of a science platform that utilizes a copilot engine to facilitate multi-user and multi-step activities by sharing a single chat session with multiple LLMs, allowing for dynamic updates, long-running conversations, and maintenance of context across multiple threads and side conversations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLM-backed chat applications use linear sequential chat sessions with fixed turn limits, then the system structure remains simple and manageable, but the ability to support multi-user multi-step activities and maintain context is limited
Solution Approach 1:
The chat session is segmented into multiple independent threads, each representing a separate conversation stream. This allows the system to handle multi-user multi-step activities by distributing context across multiple threads while maintaining the simplicity of individual linear chat sessions within each thread.
Solution Approach 2:
The system transitions from a single-dimensional linear chat session to a multi-dimensional structure by introducing threads as an additional dimension. This enables parallel conversation streams to coexist, supporting complex multi-user activities without increasing the complexity of individual chat sessions.
2Duration of action of moving object
If LLM context is limited to token capacity, then memory usage remains controlled, but long-running conversations and maintenance of context across multiple threads become difficult
Solution Approach 1:
The system extracts and stores chat session metadata and context information in external storage structures beyond the LLM's token-based memory. This allows long-running conversations to maintain context across sessions by retrieving stored information rather than relying solely on in-memory token context.
Solution Approach 2:
The system performs preliminary actions by pre-processing and storing chat session context, metadata, and conversation history in structured formats before they are needed. This enables efficient retrieval and maintenance of context in long-running conversations without overwhelming the LLM's token capacity during active sessions.
3Productivity
If chat sessions are limited to a fixed number of turns, then system resources remain manageable, but collaborative activities requiring extended interactions are restricted
Solution Approach 1:
The system enables continuous chat sessions that can run indefinitely by implementing automatic context management, thread persistence, and state preservation mechanisms. This allows collaborative activities to continue without artificial turn limits, maintaining productivity while managing resources through efficient context handling.
Data Source
AI summary
The present disclosure relates to methods and systems for sharing with a plurality of users a chat session that uses large language models to provide responses for input messages received for the chat session. The methods and systems provide access to the chat session to the users and update the chat session in response to any changes made to the chat session by any of the users. The methods and systems allow the users to resume the chat session at a future time using the chat session history.