FHE Neural Layer Parallelization for Sign and Normalization Bottlenecks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing homomorphic encryption (HE) schemes face challenges in managing computational bottlenecks, particularly due to the resource-intensive sign function, which limits the efficiency of deep learning models, especially in applications like computer vision and natural language processing.

Innovation Solution

Rearrange the layers in deep learning models to include a sign function layer adjacent to a normalization layer, allowing for parallel computation and reducing the computational burden of the sign function by pre-computing or ignoring certain components, thereby converting non-polynomial functions into polynomial form for execution under homomorphic encryption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If standard deep learning layer arrangements are used with activation functions and normalization, then model accuracy is maintained, but computational bottlenecks increase due to sequential processing of sign functions

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidlatency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments the deep learning model into distinct computational layers, specifically separating the sign function layer from the normalization layer. This segmentation allows independent optimization of each layer's computation, enabling parallel processing of previously sequential operations and reducing overall computational bottlenecks.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transforms the computational structure by rearranging layers in the depth dimension of the neural network architecture. By placing the sign function layer adjacent to the normalization layer rather than having them separated by multiple layers, the patent enables parallel computation across what was previously a sequential depth dimension, effectively adding a parallel processing dimension to the model execution.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If sign function computations are performed sequentially in deep learning models, then computational accuracy is maintained, but resource consumption increases

Engineering Contradiction:

Improvecomputational throughputVSAvoidresource consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent merges the sign function computation with adjacent layer computations by positioning them next to each other in the network architecture. This merging enables shared computational resources and parallel execution of previously separate operations, reducing overall resource consumption while maintaining computational accuracy through coordinated processing.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent performs preliminary arrangement of layers during model training or deployment preparation, positioning the sign function layer adjacent to the normalization layer before actual inference occurs. This preliminary structural configuration enables subsequent parallel computations during inference, reducing real-time resource consumption without affecting model accuracy.

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If non-polynomial functions are used in deep learning models, then model expressiveness is improved, but compatibility with homomorphic encryption decreases

Engineering Contradiction:

Improvemodel expressivenessVSAvoidHE compatibility

Core Design Contradiction:

Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

The patent changes the parameter representation of non-polynomial activation functions by expressing them in terms of sign functions, which have simpler mathematical properties. This parameter transformation maintains the expressive power of non-polynomial functions while making them compatible with homomorphic encryption operations, as sign functions can be efficiently computed in encrypted domains.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20260005830A1Parallelizing Computations of Neural Activations and Layer Normalizations in FHE Environments of Deep Learning Models

Publication Date: 2026.01.01 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Parallelizing functions in deep learning models within homomorphic encryption environments is provided. The method comprises arranging layers in a deep learning model architecture. The layers comprise a first layer computed using a sign function and a second layer having components that can be pre-computed or ignored once computing the sign function on the second layer, wherein the first layer and second layer are adjacent within the deep learning model architecture. The deep learning model architecture is trained with a number of hyper-parameters, and the trained deep learning model architecture is run under homomorphic encryption.