Optical Neural Network Elements for ReLU and Linear Computation

The patent replaces electronic computing operations with optical computing operations. Specifically, it uses optical modulators to perform linear operations (matrix multiplication) directly in the optical domain, eliminating the need to convert optical signals to electronic signals for processing. This substitution of electronic systems with optical systems removes the electronic bottleneck and enables the system to operate at the full bandwidth of optical signals, achieving speeds limited only by the optical carrier frequency rather than electronic processing speeds.

The patent changes the operational parameters of the optical system to achieve optimal nonlinear operations. It uses optical modulators that can be programmed to implement ReLU (rectified linear unit) activation functions, which are known to be optimal for neural network performance. By adjusting the transfer function of the optical modulator from the default detector response (L2 norm) to a ReLU-like response, the system achieves both ease of implementation through optical means and high computational accuracy through the superior properties of ReLU neurons.

The patent makes the optical modulator perform multiple functions: it serves as both the nonlinear activation function implementor and the linear operation performer (matrix multiplication). By programming the transfer function of the optical modulator, the same device can implement different nonlinearities (ReLU, sigmoid, etc.) and different linear transformations by changing the programming of the modulator. This multi-functionality reduces the overall system complexity compared to having separate dedicated components for each operation, as the optical modulator replaces both the electronic nonlinear processor and the electronic linear processor.