Question Embedding Routing for Secure CMS-LLM Query Handling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies struggle to generate accurate and relevant responses from generative AI entities due to the influence of diverse training data, leading to multiple valid but differing answers, and often require user interaction to refine prompts, while also facing challenges in handling proprietary information within secure environments.

Innovation Solution

The use of example question embeddings to classify user queries and select appropriate prompt generation techniques, combined with local processing to avoid data leakage and proprietary information exposure, reduces memory and processing demands, and enhances response relevance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If broad training data is used for generative AI, then the model can generate diverse responses, but the responses may include biased or proprietary information leading to relevance issues

Engineering Contradiction:

Improveresponse diversityVSAvoidresponse relevance

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent segments the training data into different types (publicly available documents vs. proprietary confidential information) and processes them through different pathways. The system separates the handling of general knowledge from organization-specific information, allowing each segment to be used appropriately for different aspects of query response generation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by treating different portions of the training corpus with different levels of access and processing. Confidential proprietary information is handled differently from public documents, with specific controls on which segments can be accessed and used for generating responses based on the query and user permissions.

Inventive Principle:

Principle #3Local quality

2Adaptability or versatility

If comprehensive training data is used, then the AI can answer more questions, but memory and processing demands increase

Engineering Contradiction:

Improvequestion coverageVSAvoiddata volume

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent extracts only the necessary portions of training data that are relevant to each specific query rather than processing the entire corpus. The system identifies and retrieves specific documents or passages that contain information needed to answer the user's question, reducing the amount of data that needs to be loaded into memory and processed.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent performs preliminary indexing and organization of training data before query processing. Documents are pre-processed, tagged, and structured in ways that enable efficient retrieval during query execution, reducing the computational burden during actual inference while maintaining comprehensive coverage.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If proprietary information is included in training data, then the AI can provide more useful organization-specific answers, but data privacy risks increase

Engineering Contradiction:

Improveanswer usefulnessVSAvoidprivacy risk

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent introduces an intermediary layer between the proprietary information and the generative AI model. This intermediary system manages access to confidential data, controls what information is exposed to the model, and ensures that proprietary information is used only under appropriate security conditions and user permissions.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates a secure, controlled environment for processing proprietary information. Confidential data is handled in isolated processing contexts with strict access controls, ensuring that proprietary information remains protected while still being available for generating useful responses when appropriate.

Inventive Principle:

Principle #39Inert atmosphere (Inert environment)

Data Source

PatentUS12614080B2Using sample question embeddings to choose between an LLM interfacing model and a non-LLM interfacing model

Publication Date: 2026.04.28 BOX INC

AI summary

Methods, systems, and computer program products for managing interactions between a content management system (CMS) and a large language model (LLM) system. The semantics of user questions can be considered before prompting an LLM, or alternatively, before querying datasets that are local to the CMS. Given a user question to be answered, the embedding of the user question can be matched against preconfigured sample question embeddings to determine a best match. A prompt corresponding to the determined best match is then configured based on identification of the class or classes that correspond to the matched question. Prompts for provision to LLMs can be synthesized based on a particular user's identity and/or based on the particular user's historical collaboration activities over objects of the CMS. The LLM can be hosted by a third-party provider. Alternatively all or portions of a large language model system can be hosted within the CMS.