User-Specific Knowledge Graphs for Confidential LLM Prompting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing generative AI systems struggle with understanding buried concepts within data, leading to inaccurate and unreliable responses, especially in enterprise environments where user-specific information is dynamic and confidential, necessitating frequent and costly retraining.
Innovation Solution
Generating user-specific knowledge graphs (KGs) from enterprise data and using them as prompts for large language models (LLMs) to enhance understanding of relationships between concepts and entities, thereby improving accuracy and reliability without the need for frequent retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If generative AI systems are trained on enterprise data to improve understanding of user-specific information, then response accuracy improves, but training costs increase and data confidentiality risks arise
Solution Approach 1:
The system pre-generates knowledge graphs from enterprise data before they are needed for inference. These KGs capture user-specific information, relationships, and concepts in advance, so when a query arrives, the system can directly utilize the pre-processed knowledge without expensive retraining. This preliminary structuring of data resolves the contradiction by preparing the knowledge base ahead of time.
Solution Approach 2:
The patent introduces knowledge graphs as an intermediary between the raw enterprise data and the generative AI model. Instead of training the model directly on raw data (which is costly and risks confidentiality), the KG serves as a structured intermediary that preserves essential information while enabling efficient querying. The KG acts as a bridge that allows accurate responses without direct model retraining on sensitive data.
2Reliability
If generative AI systems are retrained frequently on dynamic enterprise data, then response reliability improves, but computational costs and time increase
Solution Approach 1:
The system performs preliminary generation of knowledge graphs from dynamic enterprise data, updating the KG structure as new information arrives. This allows the system to adapt to changing data without time-consuming retraining cycles. The pre-structured KG can be incrementally updated and immediately utilized, resolving the time-cost contradiction.
Solution Approach 2:
Instead of retraining the expensive generative AI model on updated enterprise data, the system creates updated knowledge graphs that copy and structure the new information. These KG copies can be generated quickly and fed to the model as context, achieving adaptation without the time penalty of full retraining cycles.
3Reliability
If enterprise data is used directly for training, then model understanding of user-specific concepts improves, but data confidentiality is compromised
Solution Approach 1:
The knowledge graph serves as a confidential intermediary layer between the sensitive enterprise data and the generative AI model. The KG structures and anonymizes user-specific information, relationships, and concepts in a controlled manner that preserves understanding capability while preventing direct exposure of confidential data to the model during training or inference.
Solution Approach 2:
The system creates structured copies of enterprise data in the form of knowledge graphs that capture essential relationships and concepts without containing the raw confidential information. These KG copies enable the model to learn from the data patterns while the original sensitive data remains protected and isolated from direct model access.
Data Source
AI summary
A user-specific knowledge-based graph (KG) is generated from enterprise data corresponding to a user. The user-specific KG is stored in a data store. A query processing system receives a user query that is to be used in prompting a large language model (LLM) in an LLM service. The query processing system identifies portions of the KGs in the data store that relate to the query. A prompt generator generates a prompt to the LLM service using the identified portions of the KG and the query. A response processor receives a response from the LLM service and generates a response to the query based on the response received from the LLM service.