Lensless Optical Convolution for Low-Power Deep Learning Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing optical processing methods for deep learning are limited by high power consumption, cost, and lack of non-linear capabilities, with conventional deep-diffraction neural networks being constrained by light attenuation and requiring powerful lasers.
Innovation Solution
A system employing inexpensive micro-displays, an active pixel sensor, and a computer to perform lensless incoherent convolutions at the speed of light, using a hybrid electro-optical device with a backlit micro-display, transparent display, and active pixel sensor to capture spatial convolutions for deep learning inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If conventional optical processing methods are used for deep learning, then processing capability is achieved, but power consumption is high and cost is high
Solution Approach 1:
The patent replaces conventional electronic computing systems with an optical processing system that uses light to perform convolution operations. The system employs spatial light modulators to encode neural network layers as optical elements, enabling parallel optical computation that consumes significantly less power than traditional electronic GPUs or FPGAs while maintaining deep learning processing capability.
Solution Approach 2:
The patent creates a universal optical processing platform that can implement various deep learning models and convolutional neural network architectures using the same core optical components. The system can be reconfigured to perform different computational tasks by changing the spatial light modulator patterns, providing multi-functionality without requiring separate hardware for each application.
2Adaptability or versatility
If deep-diffraction neural networks are used, then neural network implementation is achieved, but light attenuation limits the number of layers
Solution Approach 1:
The patent extracts the convolution operation from the full optical path and implements it through spatial light modulators that can be independently controlled. By separating the convolution function from the diffraction-based neural network layers, the system avoids the light attenuation problems of fully optical approaches while maintaining the ability to implement deep neural networks through sequential optical convolution operations.
3Power
If full-optics approaches are used, then neural network processing is achieved, but light attenuation effects limit performance
Solution Approach 1:
The patent introduces spatial light modulators as intermediary elements that perform convolution operations without requiring light to traverse long optical paths. These modulators act as computational intermediaries that transform input patterns into convolved outputs directly, eliminating the light attenuation problems associated with traditional full-optics approaches while maintaining parallel processing capability.
4Productivity
If optical processing with optical fibers is used, then high throughput rates are achieved, but powerful lasers are required and power consumption is high
Solution Approach 1:
The patent replaces expensive, high-power laser systems with inexpensive ambient light sources and spatial light modulators. The system uses readily available light (such as room light or LED illumination) combined with programmable spatial light modulators to achieve the same high throughput convolution operations, dramatically reducing power consumption and system cost while maintaining computational performance.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables low-power, low-cost deep learning inference with non-linear capabilities, achieving efficient image capture and processing, and reducing Poisson noise through parallel optical convolutions.
Implementation Method 1
provide for display of patterned light corresponding to the feature maps
Implementation Method 2
perform lensless incoherent convolutions at the speed of light
Implementation Method 3
capture spatial convolutions at a corresponding imaging plane
Data Source
AI summary
A method, apparatus, and system for deep learning inference are provided including a system that employs inexpensive micro-displays, an active pixel sensor, and a computer to perform lensless incoherent convolutions at the speed of light. An apparatus is provided including processing circuitry and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processing circuitry, cause the apparatus to at least: receive feature maps of an image; receive one or more convolutional kernel; provide for display of patterned light corresponding to the feature maps; apply the one or more convolutional kernel; capture spatial convolutions at a corresponding imaging plane; and provide the spatial convolutions as training data for deep learning.