Semantic CDN Caching for RAG LLM Access-Control Compliance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
RAG-based Large Language Models (LLMs) face challenges in ensuring access permission constraints are observed in a CDN-based front door application, as different users may have varying access levels to external documents, and dynamic changes in these permissions can violate DAC policies.
Innovation Solution
Implement a CDN that caches LLM responses with document citations and user tags, verifying access permissions before providing cached responses and purging cached data based on DAC updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a CDN caches LLM responses to improve response speed and reduce server load, then productivity and speed are improved, but access permission constraints may be violated when different users request the same cached content
Solution Approach 1:
The patent segments the caching system by creating user-specific cache namespaces where each user's cached responses are separated from others. This is achieved by associating cached content with user identifiers and access permissions, ensuring that users can only access cached content they have permission to view. The segmentation resolves the contradiction by maintaining high productivity through caching while ensuring reliability through user-isolated access control.
Solution Approach 2:
The patent introduces an intermediary permission verification layer between the CDN cache and users. This intermediary checks user access permissions against the cached content's permission requirements before serving responses. The intermediary resolves the contradiction by enabling fast cached responses while maintaining access permission compliance through automated verification.
2Productivity
If cached content is shared across all users to maximize CDN efficiency, then productivity is improved, but access permission constraints are violated
Solution Approach 1:
The patent applies local quality by making cache accessibility user-specific rather than universally shared. Each cached content item is tagged with permission metadata indicating which users can access it. The CDN efficiently serves cached content to authorized users while automatically blocking unauthorized access. This resolves the contradiction by maintaining CDN efficiency through widespread caching while preventing permission violations through localized access control.
Solution Approach 2:
The patent introduces dynamic permission checking that adapts to each user's access level. The system dynamically determines whether to serve cached content based on the requesting user's permissions rather than using a static shared-cache approach. This dynamic behavior resolves the contradiction by maximizing CDN efficiency for authorized users while automatically preventing permission violations for unauthorized users.
3Reliability
If the system performs semantic search and permission verification for each request to ensure accuracy, then reliability is improved, but response time increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and caching the semantic search results and permission verification outcomes. When content is cached, the system pre-associates it with relevant user groups and permission requirements. Subsequent requests benefit from this preliminary work, avoiding redundant semantic search and permission verification. This resolves the contradiction by maintaining response accuracy through pre-validated cached content while reducing response time by eliminating repetitive processing.
Solution Approach 2:
The patent uses copying by creating cached copies of LLM responses that include embedded permission metadata and semantic search context. These copies are stored in the CDN with all necessary verification information pre-included. When serving users, the system copies the appropriate cached response to the user without requiring重新-execution of semantic search or permission checks. This resolves the contradiction by ensuring response accuracy through pre-validated copies while minimizing response time through efficient copying operations.
Data Source
AI summary
A Content Distribution Network (CDN) may implemented as a front door to a RAG-based LLM for the purpose of semantically caching LLM responses to natural language prompts. More specifically, the CDN may also cache document citation(s) and/or user tag(s) along with the LLM response for purposes of ensuring that access permission constraints of the RAG are observed when providing cached LLM response as direct responses to semantically similar natural language prompts. Additionally, the CDN may be configured to modify and/or purge cached data from the CDN's cached memory database based on instructions received from a data access control (DAC) entity of the organization or enterprise client. This may ensure that the CDN observes any changes to the access permission constraints that might be made by the DAC entity.