Selective FHE and PHE Encryption for Distributed 1-Bit LLMs

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large Language Models (LLMs) face significant challenges due to high memory and processing requirements, which limit their accessibility and pose security risks for sensitive data, necessitating efficient and secure deployment methods.

Innovation Solution

A 1-bit distributed LLM architecture using Partially Homomorphic Encryption (PHE) and Fully Homomorphic Encryption (FHE) is employed, where operations are distributed between client devices and servers, with PHE used for addition operations and FHE for more complex computations, ensuring data confidentiality and reducing computational overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If Fully Homomorphic Encryption (FHE) is used for all operations, then data security is maximized, but computational complexity and resource consumption increase significantly

Engineering Contradiction:

Improvedata securityVSAvoidcomputational complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the encryption approach by operation type: FHE is applied only to matrix multiplication operations that require high security, while PHE is used for addition operations. This selective segmentation reduces overall computational complexity while maintaining security where most needed.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different encryption schemes are applied to different parts of the computation based on their specific security and performance requirements. The system uses FHE locally for matrix multiplication and PHE locally for addition operations, optimizing each section according to its characteristics.

Inventive Principle:

Principle #3Local quality

2Productivity

If Partially Homomorphic Encryption (PHE) is used for addition operations, then computational efficiency is improved, but security coverage is limited compared to FHE

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidsecurity coverage

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent segments the operation types and applies different encryption schemes accordingly: PHE for addition operations where efficiency is critical, and FHE for matrix multiplication where security is paramount. This operational segmentation allows each scheme to operate in its optimal domain.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system applies PHE locally to addition operations and FHE locally to matrix multiplication operations, optimizing computational efficiency where addition is used and security where matrix multiplication occurs.

Inventive Principle:

Principle #3Local quality

3Quantity of substance

If 1-bit quantization is applied to reduce memory footprint, then memory requirements are significantly reduced, but computational precision is limited

Engineering Contradiction:

Improvememory footprintVSAvoidcomputational precision

Core Design Contradiction:

Quantity of substanceVSMeasurement precision

Solution Approach 1:

The patent changes the precision parameter to 1-bit quantization for model weights and activations to dramatically reduce memory footprint. This parameter change enables deployment on devices with limited resources while the homomorphic encryption operations maintain functional accuracy through mathematical properties.

Inventive Principle:

Principle #35Parameter changes

4Adaptability or versatility

If LLMs are deployed on devices with limited resources, then accessibility is improved, but computational capability is reduced

Engineering Contradiction:

ImproveaccessibilityVSAvoidcomputational capability

Core Design Contradiction:

Adaptability or versatilityVSPower

Solution Approach 1:

The patent segments the computational workload by applying 1-bit quantization to model components and using homomorphic encryption to enable computation on encrypted data. This segmentation allows the system to run on devices with limited resources while maintaining the ability to perform complex LLM operations securely.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

By changing the precision parameter to 1-bit and using homomorphic encryption, the system enables LLM deployment on resource-constrained devices. The parameter changes allow the model to operate with reduced memory footprint while the encryption enables maintained computational capability through homomorphic operations.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20250317424A1Fully homomorphic encryption (FHE) and partially homomorphic encryption (PHE) in distributed 1-bit large language model (LLM) architecture

Publication Date: 2025.10.09 SIT AUTONOMOUS AG

AI summary

A system determines whether a first operation performed by an MLM is compatible with one of a first encryption scheme and a second encryption scheme, wherein the MLM is distributed over at least one client device and at least one server. In response to determining that the first operation is compatible with the first encryption scheme, the system: encrypts data associated with the first operation using the first encryption scheme; and transmits the data encrypted by the first encryption scheme to the at least one server configured to apply the first operation. In response to determining that the first operation is incompatible with the first encryption scheme, the system: encrypts the data associated with the first operation using the second encryption scheme; and transmits the data encrypted by the second encryption scheme to the at least one server configured to apply the first operation.