Split LLM Layers for Confidential Domain-Specific Inference

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current approaches for specialized large language models (LLMs) require uploading confidential information to third-party servers, violating legal or contractual obligations and risking exposure of sensitive data.

Innovation Solution

The LLM is split such that layers trained with confidential information are executed on-premises, ensuring confidentiality by keeping sensitive data within the local network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If confidential information is uploaded to third-party servers for specialized LLM training, then the model gains domain-specific knowledge, but data confidentiality and security are compromised

Engineering Contradiction:

Improvedomain-specific knowledgeVSAvoiddata confidentiality risk

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The LLM is divided into two distinct parts: a base model that remains on third-party servers and specialized model layers that are trained locally with confidential information. This segmentation allows the system to gain domain-specific knowledge through local training while preventing exposure of confidential data to external servers.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The specialized model layers are extracted and trained separately from the base model. By taking out the training process for domain-specific knowledge and performing it locally rather than uploading data externally, the system achieves adaptability while preserving data confidentiality.

Inventive Principle:

Principle #2Taking out (Extraction)

2Ease of operation

If all LLM layers are hosted externally for easy access, then operational simplicity is improved, but control over confidential information is lost

Engineering Contradiction:

Improveremote access convenienceVSAvoidconfidentiality assurance

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The model is segmented into base layers (hosted externally for ease of access) and specialized layers (hosted locally for confidentiality control). This allows the system to maintain operational simplicity for general operations while ensuring reliability and control for confidential information processing.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different parts of the model are placed in different locations with different quality characteristics: the base model remains externally accessible for convenience, while the specialized layers are locally hosted to provide enhanced confidentiality assurance where needed.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20260065014A1Confidentiality-preserving splitting of a large language model

Publication Date: 2026.03.05 CISCO TECHNOLOGY INC

AI summary

In one implementation, a device in a local network receives, via a user interface, a prompt for input to a large language model that is external to the local network. The device sends the prompt to the large language model, wherein the large language model sends an intermediate embedding as a response to the prompt for input to one or more model layers split from the large language model that is hosted in the local network. The device receives an answer to the prompt from the one or more model layers hosted in the local network. The device provides the answer to the user interface for presentation to a user.