Stateful Text Generation Using Knowledge Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) lack memory and visibility into knowledge relationships, leading to overlapping text generations across different sessions, making it difficult to produce unique and new results.
Innovation Solution
A method to generate unique text across sessions by extracting relationship information from LLMs, storing it implicitly, and using a knowledge graph to guide text generation, ensuring distinct outputs through path traversal in the graph.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLMs generate text without memory of previous sessions, then text generation is simple and stateless, but the same text may be generated across different sessions reducing uniqueness
Solution Approach 1:
The system performs preliminary actions by extracting keywords and relationships from the LLM's context window before text generation. These extracted elements are stored in a knowledge graph that persists across sessions, allowing the system to recall and utilize previous interactions without requiring the LLM itself to maintain memory.
Solution Approach 2:
A knowledge graph serves as an intermediary between the stateless LLM and the need for session memory. The graph stores extracted keywords and their relationships, acting as a mediator that bridges the gap between the LLM's lack of memory and the requirement for unique, context-aware text generation across multiple sessions.
2Loss of information
If LLMs process entire context windows, then complete context is available, but computational complexity and processing time increase
Solution Approach 1:
The system extracts only the most relevant elements (keywords and their relationships) from the LLM's context window rather than processing the entire context. This extraction approach captures essential information while significantly reducing the complexity of subsequent processing operations.
Solution Approach 2:
The context window is segmented into discrete keywords and relationships, which are then organized in a knowledge graph structure. This segmentation transforms the monolithic context processing task into manageable discrete units that can be efficiently stored, retrieved, and processed across sessions.
3Reliability
If knowledge relationships are explicitly stored, then text uniqueness is improved, but storage requirements and system complexity increase
Solution Approach 1:
Instead of storing complete text generations or entire context windows, the system creates simplified copies in the form of keywords and relationship structures. These compact representations capture the essential knowledge relationships needed for text uniqueness while requiring minimal storage space and system complexity.
Data Source
AI summary
Systems and methods for stateful text generation using large language models are described. For example, a method may include inputting a first prompt to a large language model to cause the large language model to output a list of keywords based on a context window; inputting a second prompt to the large language model to cause the large language model to output an adjacency matrix for keywords in the list of keywords that indicates which of the keywords in the list of keywords are related in the context window; and determining a graph including nodes corresponding to respective keywords in the list of keywords and edges corresponding to relationships between keywords indicated by the adjacency matrix.