Optical Neural Network Backpropagation Using Saturable Absorbers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing optical neural networks (ONNs) face challenges in implementing backpropagation optically, as the backpropagating signal must be modulated by the derivatives of the activation function, requiring digital assistance and complex network architectures, limiting their ability to operate independently of digital computers.
Innovation Solution
An optical neural network design utilizing saturable optical absorption or gain materials that allow optical signals to propagate nonlinearly in the forward direction and linearly in the backward direction, enabling optical implementation of backpropagation without digital assistance by using saturable absorbers or gain materials with distinct threshold powers for forward and backward signal transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital assistance and complex network architectures are used to implement backpropagation in ONNs, then the ability to modulate the backpropagating signal by activation function derivatives is improved, but the autonomy from digital computers and system complexity deteriorates
Solution Approach 1:
The patent extracts the backpropagation functionality from digital computers and implements it entirely in the optical domain. The optical transmission element is designed to automatically perform the mathematical operation of multiplying the backpropagating signal by the activation function derivative through its physical transmission characteristics, eliminating the need for digital assistance in this critical function.
Solution Approach 2:
The optical transmission element serves multiple functions simultaneously: it acts as both the activation function application mechanism during forward propagation and the derivative multiplication mechanism during backpropagation. This multi-functionality reduces the need for separate components and digital processing stages.
2Ease of manufacture
If digital computers are used to assist backpropagation in ONNs, then the implementation of gradient calculation is improved, but the independence from digital systems deteriorates
Solution Approach 1:
The optical transmission element is designed to self-perform the gradient calculation function during backpropagation. By configuring the element's transmission characteristics to match the activation function derivative, the system automatically executes the mathematical operation without external digital computation, achieving self-service in the gradient calculation process.
3Adaptability or versatility
If optical transmission elements with nonlinear response are used for forward propagation, then the activation function application is improved, but the linearity required for accurate backpropagation deteriorates
Solution Approach 1:
The patent applies dynamics by making the optical transmission element's response characteristic depend on the direction of signal propagation. The element exhibits nonlinear behavior when activated by forward-propagating signals (applying the activation function) but maintains linear behavior when traversed by backward-propagating signals (enabling accurate gradient computation). This dynamic adaptability to propagation direction resolves the contradiction between nonlinear activation and linear backpropagation requirements.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for the optical implementation of backpropagation within ONNs, enabling them to train independently of digital computers and improving their efficiency and autonomy in machine learning tasks, such as image recognition, by using saturable absorbers or gain materials to apply nonlinear and linear responses respectively.
Implementation Method 1
The optical transmission element comprises a saturable optical absorption material or a saturable optical gain material, having a saturation threshold-power; wherein the optical neural network is arranged such that optical signals propagating in a forward direction have a power above the saturation threshold-power, and transmission of the optical signal through the optical transmission element in a forward direction is nonlinear
Implementation Method 2
wherein the optical neural network is further arranged such that optical signals propagating in a backward direction have a power below a second threshold-power, lower than the saturation threshold-power, and transmission of the optical signal in a backward direction through the optical transmission element is approximately linear
Data Source
AI summary
An optical neural network having at least one layer including: an optical transmission element arranged such that the signal of each node passes through the optical transmission element in both forward and backpropagation; wherein the optical transmission element comprises a saturable optical absorption material or a saturable optical gain material, having a saturation threshold-power; wherein optical signals propagating in a forward direction have a power below the saturation threshold-power at least some of the time, such that transmission of the optical signal through the optical transmission element in a forward direction is nonlinear; and wherein optical signals propagating in a backward direction have a power below a second threshold-power, lower than the saturation threshold-power, and transmission of the optical signal in a backward direction through the optical transmission element is approximately linear.