LLM Interaction Management via Policy Analysis and Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for managing interactions with large language models (LLMs) are inadequate as they lack regulation of interactions, comprehensive storage of interaction histories, and provision of additional information alongside responses.
Innovation Solution
A method and system that involve receiving prompts from user devices and policy data from organization devices, analyzing prompts based on policy data, generating output data using a large language model, storing prompt, output, and identifier data, and transmitting output data back to user devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If comprehensive policy analysis and data storage are implemented, then interaction management quality improves, but system complexity increases
Solution Approach 1:
The system is divided into distinct functional modules: policy data receiving unit, prompt data receiving unit, analysis unit, generation unit, storage unit, and transmission unit. Each module handles a specific aspect of the interaction management process, making the complex system more manageable and maintainable while ensuring comprehensive policy analysis and data storage.
Solution Approach 2:
The communication device acts as an intermediary between user devices, organization devices, and the LLM. It receives prompts from user devices, obtains policy data from organization devices, and transmits generated outputs back to user devices, thereby coordinating the complex interactions without requiring direct connections between all components.
2Loss of information
If detailed interaction histories are stored, then data completeness improves, but storage requirements increase
Solution Approach 1:
The system performs preliminary analysis of prompt data against policy data before generating outputs. This ensures that only policy-compliant interactions are processed and stored, reducing the need to store and later filter large volumes of non-compliant data, thereby optimizing storage requirements while maintaining data completeness.
Solution Approach 2:
The system stores both prompt data and corresponding output data along with identifiers in the storage device, creating a feedback loop where past interactions can be reviewed and analyzed. This comprehensive storage enables the system to learn from previous interactions and improve future responses without requiring excessive storage capacity.
Data Source
AI summary
The present disclosure provides a method of managing an interaction with a large language model (LLM). Further, the method may include receiving a prompt data from a user device associated with an organization. Further, the method may include receiving a policy data from an organization device associated with the organization. Further, the policy data may be associated with the organization. Further, the method may include analyzing the prompt data based on the policy data. Further, the method may include generating an output data based on the analyzing. Further, the generating may be based on the LLM. Further, the method may include storing each of the prompt data, the output data, and an identifier associated with one or more of the user device, the organization device, and the organization. Further, the method may include transmitting the output data to the user device.