Optical Hardware Accelerator for Neural Network Inference

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Artificial neural networks (ANNs) require significant computing power and memory bandwidth due to large number of multiplications, posing challenges for mobile and power-constrained devices.

Innovation Solution

An optical hardware accelerator (OHA) with an optical computing engine (OCE) that uses quantized phase shift values to perform ANN operations, reducing power consumption and memory bandwidth by converting ANN weights into phase shift values and applying them using optical units for faster computation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If conventional GEMM operations are used for convolutional layers, then computation can be performed using standard hardware, but power consumption and computing resource usage increase significantly

Engineering Contradiction:

Improvecompatibility with standard hardwareVSAvoidpower consumption

Core Design Contradiction:

Ease of manufactureVSUse of energy by moving object

Solution Approach 1:

The patent replaces conventional electronic computing operations with optical computing operations. Specifically, it uses optical modulators to perform multiplication operations and optical detectors to perform accumulation operations, substituting electronic mechanical systems with optical systems that consume less power for the same computational tasks

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the fundamental operating parameter from electronic signals to optical signals. By using light intensity and phase modulation instead of electronic voltage/current operations, the system achieves lower power consumption while maintaining computational functionality for neural network inference

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If conventional GEMM operations are used for convolutional layers, then standard processors can be utilized, but the number of multiplications required consumes significant computing power

Engineering Contradiction:

Improveflexibility of implementationVSAvoidcomputing power efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent substitutes optical computing operations for electronic multiplication operations. Optical modulators modulate light signals according to input weights and data, performing multiplication inherently through the modulation process, thereby eliminating the need for separate electronic multiplication circuits and reducing overall computing power requirements

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If ANNs are implemented on mobile devices, then classification and recognition problems can be solved, but storage and power constraints are violated

Engineering Contradiction:

Improvecapability to solve classification and recognition problemsVSAvoidpower consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by stationary object

Solution Approach 1:

The patent implements optical computing operations within mobile devices to replace power-consuming electronic operations. By using optical modulators and detectors integrated into the mobile device architecture, the system maintains the capability to perform classification and recognition while significantly reducing power consumption suitable for mobile battery operations

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the operational mode from electronic signal processing to optical signal processing within the mobile device. This parameter change enables the device to perform ANN inference tasks with lower power consumption, making it feasible to run such computationally intensive tasks on power-constrained mobile platforms

Inventive Principle:

Principle #35Parameter changes

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The OHA significantly accelerates ANN computations, reduces power consumption, and decreases memory bandwidth requirements compared to conventional hardware accelerators, making it suitable for mobile and power-constrained devices.

Implementation Method 1

each MZI configured to apply a phase shift equal to the quantized phase shift value θi of a corresponding ANN weight to an optical signal

Methodology Applied

Scientific EffectPhase shift: Phase Modulation

Implementation Method 2

each single-node phase shifter configured to apply a phase shift equal to the phase shift value φi of a corresponding ANN weight to the optical signal

Methodology Applied

Scientific EffectPhase shift: Phase Modulation

Data Source

PatentUS11526743B2Artificial neural network optical hardware accelerator

Publication Date: 2022.12.13 ARM LTD

AI summary

The present disclosure advantageously provides an Optical Hardware Accelerator (OHA) for an Artificial Neural Network (ANN) that includes a communication bus interface, a memory, a controller, and an optical computing engine (OCE). The OCE is configured to execute an ANN model with ANN weights. Each ANN weight includes a quantized phase shift value θi and a phase shift value ϕi. The OCE includes a digital-to-optical (D/O) converter configured to generate input optical signals based on the input data, an optical neural network (ONN) configured to generate output optical signals based on the input optical signals, and an optical-to-digital (O/D) converter configured to generate the output data based on the output optical signals. The ONN includes a plurality of optical units (OUs), and each OU includes an optical multiply and accumulate (OMAC) module.