Systems and methods for analog computing using linear photonic processors - Patents.com
Photonic processors overcome electrical limitations by using optical signals to perform matrix multiplications efficiently, achieving faster and more power-efficient computations for machine learning and neural networks.
Patent Information
- Application Number
- JP2022506291
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-11-22
- Filing Date
- 2020-07-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-07-28
AI Technical Summary
Traditional computing processors face speed and efficiency limitations due to electrical characteristics such as impedance, leading to significant time losses and heat dissipation issues, especially in processes requiring billions of operations.
The use of optical signals in photonic processors, which travel at the speed of light and do not dissipate power, allowing for superior speed and efficiency performance by implementing matrix-vector and matrix-matrix multiplications through a configurable array of active optical components.
Photonic processors achieve highly parallelized linear transformations like matrix multiplication in a few hundred picoseconds, significantly outperforming traditional electronic processors, and are suitable for machine learning algorithms and neural networks.
Smart Images

Figure 0007579841000066 
Figure 0007579841000067 
Figure 0007579841000068
Abstract
Description
[Background technology]
[0001] Traditional computing uses processors that contain circuits of millions of transistors to implement logic gates on bits of information represented by electrical signals. Traditional central processing units (CPUs) have architectures designed for general-purpose computing, but are not optimized for specific types of algorithms. Therefore, specialized processors have been developed with architectures that are better suited for specific algorithms. Graphical processing units (GPUs), for example, have highly parallel architectures that make them more efficient than CPUs for running image processing, graphical manipulation, and other parallelizable applications, such as for neural networks and deep learning. Summary of the Invention [Means for solving the problem]
[0002] Some embodiments are directed to an apparatus for implementing a signed numeric value, the apparatus comprising: a photodetector including a first terminal and a second terminal; a first switch coupling the first terminal of the photodetector to either a node or a reference voltage; a second switch coupling the second terminal of the photodetector to either the node or a voltage rail; and control circuitry configured to generate, at least in part, a positive signed numeric output by setting the first switch to couple the first terminal to the reference voltage and the second switch to couple the second terminal to the node, and to generate, at least in part, a negative signed numeric output by setting the first switch to couple the first terminal to the node and setting the second switch to couple the second terminal to the voltage rail.
[0003] Some embodiments include an optical processing system including: a first plurality of optical modulators, each configured to receive an input optical signal, modulate the input optical signal, and output a first optical signal representing an element of a vector; a second plurality of optical modulators, each configured to receive the first optical signal, modulate the first optical signal, and output a second optical signal representing a portion of a matrix-vector multiplication between the vector and a matrix; and a plurality of optical detectors, each configured to optically couple to a plurality of optical modulators of the second plurality of optical modulators and convert the second optical signal into an electrical signal representing the portion of the matrix-vector multiplication, each optical detector of the plurality of optical detectors including a first terminal and a second terminal; the first switch coupling a terminal to either an output node or a reference voltage; a second switch coupling a second terminal of the first photodetector to either the output node or a voltage rail; and control circuitry configured to generate, at least in part, a positive signed numerical output by setting the first switch to couple the first terminal of the first photodetector to the reference voltage and to configure the second switch to couple the second terminal of the first photodetector to the output node, and to generate, at least in part, a negative signed numerical output by setting the first switch of the first photodetector to couple the first terminal to the output node and to configure the second switch of the first photodetector to couple the second terminal to the voltage rail.
[0004] Some embodiments provide a method for implementing a signed numerical value output by a photodetector of an optical processor, comprising: converting an output optical signal into a first electrical signal using a photodetector including a first terminal and a second terminal; the output optical signal being output by a portion of the optical processor, the method comprising: determining, using at least one conventional processor coupled to the optical processor, whether the first electrical signal represents a positive signed numerical value or a negative signed numerical value; and, in response to determining whether the first electrical signal represents a positive signed numerical value or a negative signed numerical value, adjusting, using control circuitry of the optical processor, a setting of a first switch coupled to the first terminal and a setting of a second switch coupled to the second terminal; the control circuitry is configured to generate, at least in part, a positive signed numerical output by setting a first switch to couple the first terminal to a reference voltage and setting a second switch to couple the second terminal to a node, and to generate, at least in part, a negative signed numerical output by setting the first switch to couple the first terminal to the node and setting the second switch to couple the second terminal to a voltage rail; and outputting a first electrical signal from the photodetector such that the first electrical signal passes through either the first switch or the second switch based on a determination of whether the first electrical signal represents a positive signed numerical value or a negative signed numerical value.
[0005] Some embodiments are directed to an optical processor for implementing a matrix-vector multiplication operation comprising: a first plurality of optical modulators, each configured to receive an input optical signal, modulate the input optical signal, and output a first optical signal representing an element of a vector; a second plurality of optical modulators optically coupled to one of the first plurality of optical modulators and each configured to receive the first optical signal, modulate the first optical signal, and output a second optical signal representing a portion of a matrix-vector multiplication between the vector and a matrix; a plurality of optical detectors coupled to a plurality of the optical modulators of the second plurality of optical modulators and each configured to convert the second optical signal to an electrical signal representing a portion of the matrix-vector multiplication; and a plurality of switches configured to implement values of zero in the matrix-vector multiplication operation by preventing transmission of an optical or electrical signal if values of the vector and / or matrix include zero, a switch of the plurality of switches coupled to an output of each of the first plurality of optical modulators or each of the plurality of optical detectors.
[0006] Some embodiments are directed to a method of performing a matrix-vector row multiplication operation using an optical processor, the method including: modulating, using a first optical modulator, an input optical signal to optically represent an element of the vector in a first optical signal; modulating, using a second optical modulator, the first optical signal to optically represent an augend in a second plurality of optical signals, the augends which when summed represent a product between the elements of the vector and a row of the matrix; converting, using an optical detector, the second plurality of optical signals into a plurality of augend electrical signals; and causing a switch coupled to an output of the first optical modulator to prevent transmission of the first optical signal to the second optical modulator if the value of an element of the vector is equal to zero and / or causing one or more switches coupled to the output of the optical detector to prevent transmission of the augend electrical signal if the value of one or more elements of the row of the matrix is equal to zero.
[0007] Some embodiments are directed to at least one non-transitory computer readable medium including instructions that, when executed by at least one optical processor, cause the optical processor to perform a method of modulating, using a first optical modulator, an input optical signal to optically represent an element of a vector in a first optical signal; modulating, using a second optical modulator, the first optical signal to optically represent an augend in a second plurality of optical signals, the augends which, when summed, represent a product between the vector and a row of a matrix; converting, using an optical detector, the second plurality of optical signals into a plurality of augend electrical signals; and causing a switch coupled to an output of the first optical modulator to prevent transmission of the first optical signal to the second optical modulator if a value of an element of the vector is equal to zero and / or causing one or more switches coupled to an output of the optical detector to prevent transmission of the augend electrical signal if a value of one or more elements of a row of the matrix is equal to zero.
[0008] Some embodiments are directed to a method of performing a matrix-matrix operation using an optical processor, the method including the steps of programming a first matrix into a first optical sub-processor; programming a second matrix into a second optical sub-processor, the second optical sub-processor including an input coupled to an output of the first optical sub-processor; inputting a plurality of one-hot vectors into the first optical sub-processor as optical signals; and outputting an output vector from the second optical sub-processor representing a portion of the multiplication of the first matrix and the second matrix.
[0009] Some embodiments are directed to an optical processor configured to perform matrix-matrix operations, comprising: a first optical sub-processor configured to optically perform matrix-vector multiplication of a one-hot vector and a first matrix to obtain a first vector; and a second optical sub-processor configured to receive an output signal from the first optical sub-processor and to optically perform matrix-vector multiplication of the first vector and a second matrix.
[0010] Some embodiments include an optical processor configured to perform a matrix-matrix operation, the first optical sub-processor including a first plurality of optical modulators each configured to receive an input optical signal, modulate the input optical signal, and output a first optical signal representing an element of a one-hot vector; a second plurality of optical modulators optically coupled to one of the first plurality of optical modulators and each configured to receive the first optical signal, modulate the first optical signal, and output a second optical signal representing a portion of a matrix-vector multiplication between the one-hot vector and a first matrix; a first plurality of optical detectors coupled to one of the second plurality of optical modulators and configured to convert the second optical signal into an electrical signal representing the portion of the matrix-vector multiplication; and a first plurality of electrical summers coupled to outputs of two or more optical detectors of the first plurality of optical detectors and each configured to output an electrical signal representing an element of a vector resulting from a summation of the portions of the matrix-vector multiplication. and a second optical sub-processor including a third plurality of optical modulators each receiving an output electrical signal from an electrical summing unit of a plurality of electrical summing units of the processor, the third plurality of optical modulators each configured to receive an input optical signal, modulate the input optical signal in accordance with the received output electrical signal, and output a third optical signal representing an element of a vector; a fourth plurality of optical modulators optically coupled to one of the optical modulators of the third plurality of optical modulators and each configured to receive the third optical signal representing the element of the vector, modulate the third optical signal, and output a fourth optical signal representing a portion of a matrix-matrix multiplication between a first matrix and a second matrix; a second plurality of optical detectors coupled to one of the optical modulators of the third plurality of optical modulators and each configured to convert the third optical signal into an electrical signal representing the portion of the matrix-matrix multiplication; and a second plurality of electrical summing units coupled to outputs of two or more optical detectors of the second plurality of optical detectors and each configured to output an electrical signal representing an element of a matrix resulting from a summation of the portion of the matrix-matrix multiplication.
[0011] Various aspects and embodiments are described with reference to the following drawings. It should be understood that the drawings are not necessarily drawn to scale. In the drawings, identical or nearly identical components illustrated in various drawings are represented by similar numerals, respectively. For ease of understanding, not every component may be labeled in every drawing. [Brief description of the drawings]
[0012] [Figure 1] 1 is a schematic diagram illustrating an example of a photonic processing system in accordance with some embodiments of the techniques described herein. [Diagram 2] FIG. 1 is a schematic diagram illustrating a linear photonic processor in accordance with some embodiments of the techniques described herein. [Figure 3A] FIG. 1 is a block diagram illustrating an example of a photonic architecture for implementing matrix-vector operations in accordance with some embodiments of the techniques described herein. [Figure 3B] FIG. 1 is a block diagram illustrating an example of a photonic architecture layout for minimizing electrical path lengths, in accordance with some embodiments of the techniques described herein. [Figure 4A-4C] 1 is an illustrative example of an intensity modulator configured to optically represent a zero, in accordance with some embodiments of the techniques described herein. [Diagram 5] FIG. 1 is a block diagram illustrating an example of a photonic architecture including wavelength division multiplexing (WDM) in accordance with some embodiments of the techniques described herein. [Figure 6A] FIG. 2 is a schematic diagram of a circuit for implementing signed values in accordance with some embodiments of the techniques described herein. [Figure 6B] FIG. 2 is a schematic diagram of a Boolean circuit for implementing signed values in accordance with some embodiments of the techniques described herein. [Figure 7]1 is a schematic diagram of an optical circuit for distributing code bits in accordance with some embodiments of the techniques described herein. [Figure 8] 1 is a flowchart illustrating a process for implementing signed values in accordance with some embodiments of the techniques described herein. [Figure 9A] FIG. 1 is a schematic diagram illustrating photonic processors arranged in sub-matrices in accordance with some embodiments of the techniques described herein. [Figure 9B] FIG. 1 is a schematic diagram illustrating photonic processors arranged in sub-matrices in accordance with some embodiments of the techniques described herein. [Figure 9C] FIG. 1 is a schematic diagram illustrating photonic processors arranged in sub-matrices in accordance with some embodiments of the techniques described herein. [Figure 9D] FIG. 1 is a schematic diagram illustrating photonic processors arranged in sub-matrices in accordance with some embodiments of the techniques described herein. [Figure 10A] FIG. 1 is a schematic diagram illustrating photonic processors arranged in sub-matrices whose outputs are summed locally, in accordance with some embodiments of the techniques described herein. [Figure 10B] FIG. 1 is a schematic diagram illustrating photonic processors arranged in sub-matrices whose outputs are summed locally, in accordance with some embodiments of the techniques described herein. [Figure 11] FIG. 1 is a schematic diagram illustrating a photonic processor configured to use input light having multiple wavelengths in accordance with some embodiments of the techniques described herein. [Figure 12A] FIG. 1 is a schematic diagram of an optical combiner configured for use with multiple wavelengths of light in accordance with some embodiments of the techniques described herein. [Figure 12B] 12B is an illustrative plot of the free-spectral-range (FSR) of the combiner of FIG. 12A in accordance with some embodiments of the techniques described herein. [Figure 12C]12B is an illustrative plot of transmission as a function of wavelength for several of the combiner of FIG. 12A in accordance with certain embodiments of the techniques described herein. [Figure 13A] FIG. 1 is a block diagram illustrating a photonic processor configured to implement sparse vectors and / or matrices in accordance with some embodiments of the techniques described herein. [Figure 13B] FIG. 1 is a block diagram illustrating a photonic processor configured to implement sparse vectors and / or matrices in accordance with some embodiments of the techniques described herein. [Figure 13C] FIG. 1 is a block diagram illustrating a photonic processor configured to implement sparse vectors and / or matrices in accordance with some embodiments of the techniques described herein. [Figure 14] 1 is a flowchart illustrating a process for performing a matrix-vector operation involving sparse vectors and / or matrices in accordance with some embodiments of the techniques described herein. [Figure 15A] 1 is a schematic diagram illustrating a method of signal readout of a photonic processor according to some embodiments of the technology described herein. [Figure 15B] 1 is a schematic diagram illustrating a method of signal readout of a photonic processor according to some embodiments of the technology described herein. [Figure 15C] 1 is a schematic diagram illustrating a method of signal readout of a photonic processor according to some embodiments of the technology described herein. [Figure 15D] 1 is a schematic diagram illustrating a method of signal readout of a photonic processor according to some embodiments of the technology described herein. [Figure 16A] FIG. 1 is a schematic diagram illustrating the architecture of a photonic processor having clustered readout circuits in accordance with some embodiments of the techniques described herein. [Figure 16B] FIG. 1 is a schematic diagram illustrating the architecture of a photonic processor having clustered readout circuits in accordance with some embodiments of the techniques described herein. [Figure 16C] FIG. 1 is a schematic diagram illustrating the architecture of a photonic processor having clustered readout circuits in accordance with some embodiments of the techniques described herein. [Figure 16D] FIG. 1 is a schematic diagram illustrating the architecture of a photonic processor having clustered readout circuits in accordance with some embodiments of the techniques described herein. [Figure 17] FIG. 1 is a block diagram of a photonic processor configured to implement matrix-matrix operations in accordance with some embodiments of the techniques described herein. [Figure 18] 1 is a flowchart illustrating a process for performing a matrix-matrix operation using a photonic processor in accordance with some embodiments of the techniques described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Processors based on electrical circuits face limitations in speed and efficiency due to electrical properties such as impedance. For example, conductive traces with non-zero impedance are used to connect multiple processor cores to memory and / or to connect a single processor core. Large impedance values limit the maximum speed at which data can be transferred through the traces with negligible bit error rates. For processes involving billions of operations, these delays can result in significant time loss. In addition to the poor speed efficiency of electrical circuits, heat generated by energy dissipation caused by the impedance of the circuits also poses a barrier in developing electrical processors.
[0014] The inventors recognize and understand that the use of optical signals instead of electrical signals or in combination with electrical signals overcomes many of the problems with electrical computing discussed above. Optical signals travel in a medium in which light travels at the speed of light. Thus, the latency of photonic signals is much less limiting than electrical propagation delays. In addition, power is not dissipated by increasing the distance that optical signals travel, allowing for new topologies and processor layouts that were not possible using electrical signals. Thus, optical-based processors, such as photonic-based processors, may have speed and efficiency performance superior to conventional electrical processors.
[0015] The inventors recognize and understand that optically-based processors, such as photonic-based processors, may be well suited for certain types of algorithms. For example, many machine learning algorithms, such as support vector machines, artificial neural networks, and probabilistic graphological model learning, rely heavily on linear transformations on multidimensional arrays / tensors. The simplest example is multiplying a vector by a matrix, which takes approximately O(N 2 ), where N is the number of dimensions of a square matrix being multiplied by a vector of the same dimension. The inventors recognize and appreciate that photonic-based processors can perform linear transformations such as matrix multiplications in a highly parallelized manner by propagating a particular set of input optical signals through a configurable array of active optical components. Using such an implementation, a matrix-vector multiplication of dimension N=512 can be completed in hundreds of picoseconds, as opposed to tens to hundreds of nanoseconds using conventional electronic circuit-based processing.
[0016] General matrix-matrix (GEMM) operations are widespread in software algorithms, including graphical processing, artificial intelligence, neural networks, and deep learning software. In modern computers, GEMM computations are typically performed using transistor-based systems, such as GPU systems or systolic array systems. GEMM computations can also be performed in the photonic domain by mixing an array of input optical signals representing the elements of an input vector with a mesh array of interferometers that represent the elements of the input matrix.
[0017] Matrix-vector multiplication using photonic arrays can be very power efficient compared to electronic ones because optical signals can propagate in semiconductor substrates with minimal loss. The inventors recognize and understand that there are many challenges associated with the use of such photonic arrays. Although interference is described mathematically by a unitary matrix, the scheme can be generalized to any linear transformation by deliberately adding losses to a mesh array of interferometers. However, photonic arrays often use virtually loss-free interferometers that do not exhibit phase-dependent losses. This characteristic can limit the modulation schemes that are acceptable within the system. In particular, it precludes the use of high bandwidth (e.g., up to 100 GHz) splicing-based modulation schemes. In addition, the number of optics that each optical path "sees" scales with the dimensionality of the matrix. This imparts a larger amount of loss to each optical signal the larger the matrix. The non-zero insertion loss of modulators in the real world sets a practical limit to the size of matrices that can be represented in such photonic processing systems. Moreover, programming the matrix into an array of interferometers can be a challenging task, since the elements of the matrix must be algorithmically converted into phase settings for each interferometer.
[0018] Therefore, the inventors have developed a novel photonic processing architecture for performing matrix-vector multiplication, the core operation of GEMM operations, that avoids or mitigates the above-mentioned problems by modulating the intensity of an array of optical signals to encode elements of an input vector, attenuating or amplifying the array of optical signals to perform multiplication between elements of the input matrix and elements of the input vector, detecting the optical signals with an array of output detectors, and summing the resulting photodetector currents to generate a final output result representing the matrix-vector product.
[0019] The inventors recognize and understand that such photonic processing architectures can utilize incoherent light (e.g., light in which electromagnetic waves do not maintain a fixed and predictable phase relationship with one another over time) for computation. An advantage of using an array of incoherent light sources is that no phase correction schemes (e.g., to correct thermal drift and / or device manufacturing imperfections) are required. In addition, the inventors recognize and understand that in such photonic processing architectures, the matrix elements can be directly encoded in the attenuator. Finally, the inventors recognize that the light path in such photonic processing architectures passes through two modulators (e.g., a vector modulator and a matrix modulator) regardless of the size of the matrix being encoded. Thus, the photonic processing architectures described herein enable combined amplitude and phase modulation modulation schemes as well as dynamic loss, loss scaling that does not scale with matrix size, and more direct encoding schemes, while retaining the power efficiency advantages of photonic-based GEMM processors.
[0020] In the following, various concepts related to techniques for performing GEMM operations using linear photonic processors and embodiments of those techniques are described in more detail. It should be understood that the various aspects described herein can be implemented in any of many ways. Examples of specific implementations are described herein for illustrative purposes only. In addition, the various aspects described in the following embodiments can be used alone or in any combination, and are not limited to the combinations expressly described herein.
[0021] 1 is a schematic diagram of a photonic processing system implementing photonic processing techniques according to some embodiments of the techniques described herein. Photonic processing system 100 includes a controller 102, a light source 108, and a photonic processor 110. Photonic processing system 100 receives input vectors and / or matrices represented by a set of input bit strings as input from an external processor (e.g., a CPU) and generates output vectors and / or matrices represented by a set of output bit strings. For example, if the input vector is an M-dimensional vector, the input vector may be represented by M separate bit strings, each bit string representing a respective component of the vector. Alternatively or additionally, for example, if the input matrix is an N×N type matrix, the input matrix may be represented by M separate bit strings, each bit string representing a respective component of the vector. 2 The input bit string may be represented by separate bit strings, each representing a respective element of the input matrix. The input bit string may be received as an electrical or optical signal from an external processor, and the output bit string may be transmitted as an electrical or optical signal to an external processor.
[0022] In some embodiments, the controller 102 does not necessarily output an output bit string after every iteration of the process. Instead, the controller 102 may use one or more output bit strings to determine a new input bit stream to feed through the components of the photonic processing system 100. In some embodiments, the output bit string itself may be used as an input bit string for a subsequent iteration of the process implemented by the photonic processing system 100. In other embodiments, multiple output bit streams are combined in various ways to determine a subsequent input bit string. For example, one or more output bit strings may be added together as part of the determination of a subsequent input bit string.
[0023] In some embodiments, the controller 102 includes a processor 104 and a memory 106 for controlling the light source 108 and / or the photonic processor 110. The memory 106 can be used to store input and output bit strings and / or results from the photonic processor 110. The memory 106 can also store executable instructions that, when executed by the processor 104, control the light source 108 and / or control components (e.g., encoders, phase shifters, and / or detectors) of the photonic processor 110. For example, the memory 106 can store executable instructions that cause the processor 104 to determine new input values to send to the photonic processor 110 based on the number of computational iterations that have occurred. In this way, the output matrix sent by the photonic processing system 100 to an external processor can be the result of multiple accumulated multiplication operations rather than just one multiplication operation. In another embodiment, the results of the computations by the photonic processing system 100 can be digitally computed by the processor 104 before being stored in the memory 106. Operations on bit strings can be linear, but also non-linear or more generally Turing complete.
[0024] The optical source 108 may be configured to provide N optical signals to the photonic processor 110 according to some embodiments of the present technology. The optical source 108 may include, for example, one or more coherent and / or incoherent optical sources configured to generate the N optical signals. The optical source 108 may, in some embodiments, include a laser configured to emit light at a wavelength λ0. The wavelength of the emitted light may be in the visible, infrared (including near infrared, mid infrared, and far infrared) or ultraviolet portion of the electromagnetic spectrum. In some embodiments, λ0 may be in the O-band, C-band, or L-band. In some embodiments, the optical source 108 may emit light at different wavelengths λ1, λ2, ..., λ n The optical fiber may include a plurality of lasers configured to emit light at
[0025] Each output of the light source 108 may be one-to-one coupled to a single input of the photonic processor 110 in accordance with some embodiments of the techniques described herein. In some embodiments, the light source 108 may be provided on the same substrate (e.g., the same chip) as the photonic processor 110. In such embodiments, the optical signal may be transmitted from the light source 108 to the photonic processor 110 in a waveguide (e.g., a silicon photonic waveguide) provided on the same substrate. In other embodiments, the light source 108 may be provided on a separate substrate from the photonic processor 110. In such embodiments, the optical signal may be transmitted from the light source 108 to the photonic processor 110 through one or more optical fibers.
[0026] The photonic processor 110 may perform matrix-vector, matrix-matrix, and / or tensor-tensor multiplication operations according to some embodiments of the techniques described herein. In some embodiments, the photonic processor 110 includes two parts: a modulator (see, e.g., amplitude modulators 204 and 208 in FIG. 2) configured to encode the elements of an input vector, matrix, and / or tensor of the amplitude and / or intensity of the optical signal from the light source 108, and an optical detector (see, e.g., detector 210 in FIG. 2) configured to detect and convert the optical signal into an electrical signal proportional to the product of the encoded elements. The photonic processor 110 outputs these electrical signals to the controller 102 and / or an external processor for further processing.
[0027] In some embodiments, one or more of the input matrices or tensors may be too large to be encoded in the photonic processor using a single optical path. In such a situation, a portion of the large matrix may be encoded in the photonic processor and a multiplication process may be performed on only the portion of the large matrix and / or the large matrices. The result of this first operation may be stored in memory 106. A second portion of the large matrix may then be encoded in the photonic processor and a second multiplication process may be performed. This "tiling" of the large matrix may continue until the multiplication process has been performed on all portions of the large matrix. The results of the multiple multiplication processes may then be stored in memory 106 and combined to form the final result of the tensor multiplication operation.
[0028] In some embodiments, the photonic processor 110 can convert the N separate optical pulses into electrical signals. In some embodiments, the intensity and / or phase of each of the optical pulses can be measured by a photodetector within the photonic processor 110, as described in more detail in connection with at least Figures 2 and 3. The electrical signals representing those measurements can then be electrically summed and / or output to the controller 102 for use in further calculations and / or display.
[0029] 2 is a schematic diagram illustrating an example of a linear photonic processor 200 in accordance with some embodiments of the techniques described herein. The linear photonic processor 200 may be implemented as the light source 108 and photonic processor 110 of the photonic processing system 100 as described in relation to FIG.
[0030] In some embodiments, the linear photonic processor 200 may be configured to perform a matrix-vector multiplication operation.
[0031]
number
[0032]
number
[0033]
number
[0034] In some embodiments, the first amplitude modulator 204 can be configured to encode elements of an input vector based on a respective input bit string (e.g., from a controller as in FIG. 1) into the amplitude of the optical signal received from the light source 202. Modulation mechanisms include, for example, electromechanical modulation, plasma dispersion modulation, electro-optical modulation (χ (2) ,χ (3) ,χ (4) ...), thermo-optical modulation and / or piezoelectric electro-optical modulation. j Let x be the intensity of the received optical signal that is the input of the j-th first amplitude modulator. The first amplitude modulator j modulates the optical intensity, and each first amplitude modulator j modulates the optical intensity to obtain an intensity x j I j to output a first optical signal having j Encode the value of .
[0035] In some embodiments, the first amplitude modulator 204 may be a variable attenuator controlled by a DAC (not depicted) or any other suitable amplitude modulator, which may be further controlled by a controller (e.g., controller 102 in FIG. 1). Several amplitude modulators are known for telecommunications applications and may be used in some embodiments. In some embodiments, a variable beam splitter may be used as the first amplitude modulator 204, where only one output of the variable beam splitter is kept and the other output is discarded or ignored. Other examples of amplitude modulators that may be used in some embodiments include traveling wave modulators, cavity-based modulators, Franz-Keldysh modulators, Plasmon-based modulators, 2D material-based modulators, and nano-opto-electromechanical switches (NOEMS).
[0036] In some embodiments, the first optical signal from the first amplitude modulator 204 may be split log2(P) times and transmitted to the P second amplitude modulators 208 using the photonic waveguide 206. The photonic waveguide 206 may include, for example, a silicon photonic waveguide or any other suitable dielectric photonic waveguide material. The intensity of the first optical signal after splitting (e.g., when received by each second amplitude modulator 208) may be expressed as x j I j / P.
[0037] In some embodiments, the second amplitude modulators 208 can each be configured to encode one value of one element of the matrix w. The second amplitude modulators 208 can be the same type of modulator as the first amplitude modulator 204, or can alternatively be a different type of modulator than the first amplitude modulator 204. There can be a total of P×Q second amplitude modulators to represent the entire matrix w. j I j The received first optical signal having an intensity w / P is modulated by the second amplitude modulator 208 in the p-th row of the matrix w to obtain an intensity w pj xj I j A second optical signal having a power of 1 / P can be generated, the power of which is determined by the matrix element w pj and vector element x j Represents multiplication with .
[0038] In some embodiments, the output second optical signal output by each second amplitude modulator 208 may be sent to and detected using a photodetector 210. The photodetector 210 may be, for example, a photodetector configured to generate a photocurrent proportional to the intensity of light incident on the detector. In particular, the photocurrent generated by the photodetector 210 located at row p and column j may be
[0039]
number
[0040] In some embodiments, the photocurrents generated from the photodetectors 210 in each row are combined to produce a summed output
[0041]
number
[0042]
number
[0043]
number
[0044] It can be appreciated that in some embodiments, the number of photodetectors 210 can be reduced by routing the optical signals output by the second amplitude modulator 208 to the same detector. This change will still produce a suitable electrical signal output, since the matrix-vector multiplication information is encoded into the light intensity that is directly proportional to the photocurrent generated by the photodetector. To prevent undesired interference between signals routed to the same detector, a sufficiently incoherent light source can be used, or the photodetectors 210 can be time multiplexed such that the first output signal arrives first, the second output signal arrives after the first output signal, the third output signal arrives after the second output signal, and so on. The detector readout circuitry can use an electrical storage circuit to store the accumulated charge in such an embodiment.
[0045] In prior art photonic processing architectures, the inputs to the modulators that encode the matrix w must be computed using a computationally expensive decomposition procedure. In the described linear photonic processor 200, the inputs to each matrix modulator are the elements of the matrix w itself, and no decomposition is required prior to performing the optical computations.
[0046] In addition, as a direct consequence of the "Euclidean space" representation of the matrix w, rather than the "phase-space" representation used by prior art photonic processing architectures, the linear photonic processing architectures described above have N 2 lightpaths, each associated with a single element of the w matrix. This result has two important implications for the performance and scaling of this architecture.
[0047] First, each optical path contains two modulators regardless of the size of the matrix represented. In contrast, in some photonic processing architectures, an optical path contains O(N) modulators. Real-world modulators suffer from non-zero insertion loss, which limits the size of the matrix that can be represented for a given allowable optical loss. The linear photonic processor 200 described above does not suffer from this limitation on matrix size.
[0048] Second, in other photonic processing architectures, each matrix element is generally associated with many optical paths through the optical array. The number of paths through any given "phase-space" modulator affects many elements of the represented matrix. Moreover, the number of elements affected by a particular modulator is not constant, resulting in non-uniform error sensitivity. For example, a modulator near the center of a modulator array may be much more sensitive than one at an edge or corner. This is because the central modulator is on a propagation path with a larger number of input optical signals than a modulator at an edge or corner. In contrast, errors in each "Euclidean space" matrix modulator in the linear photonic processor 200 described above only affect that particular value of the matrix element.
[0049] It may further be appreciated that in some embodiments, the linear photonic processor 200 described above may reduce the number of photonic crossings (e.g., of the waveguides 206) within the photonic processing architecture. Due to the data flow nature of matrix-vector multiplication (e.g., a single vector element may be broadcast to multiple rows, and signals from different columns of each row may be bundled together to generate a single output vector element), there is inherently crossing data paths that may exist in the photonic domain or the electrical domain. For example, in the case of a photonic processor, the partial products M ij x j To find x j This involves broadcasting to multiple rows i. At this point, no crossover is necessary. However, the summation of different columns within a single row, i.e., Σ j M ij x j To achieve this, photonic or electrical crossovers are required.
[0050] The crossing between two photonic waveguides can be physically realized by using a multi-mode interferometer (MMI) crossing in the same semiconductor layer or by using another layer of the semiconductor substrate. In contrast to an electrical crossing, a photonic crossing is desirable because it can reduce the overall capacitance of the circuit, which can adversely affect the bandwidth of the system. However, MMI-based crossings can induce significant crosstalk and loss to the optical signals.
[0051] Value x j The properties of the binary tree used to broadcast x are in fact well suited to reducing the number of crossings required. For example, jConsider the case where N is split (multiplication between an NxN matrix and a vector of size N). If the splitting is performed with a single 1-to-N splitter, then on the order of N / 2 photonic crossovers may be required. On the other hand, if the splitting is performed with binary 1-to-2 splitters spaced in a tree, then on the order of N photonic crossovers may be required to clear the broadcasting waveguide.
[0052] In some embodiments, if the number of photonic waveguide crossings results in high crosstalk and loss, it may be desirable to design the crossing in the electrical domain at the expense of extra capacitance. A crossing in the electrical domain consists of routing two signals in two different metal layers of the same semiconductor substrate. The electrical crossing can be placed at the output of the photodetector.
[0053] 3A is a block diagram of an illustrative example of a linear photonic processor 300a for implementing matrix-vector operations in accordance with some embodiments of the techniques described herein. The linear photonic processor 300a is similar to the linear photonic processor 200 described in connection with FIG. 2 and performs the same matrix-vector multiplication operations.
[0054]
number
[0055]
number
[0056]
number
[0057] In some embodiments, the linear photonic processor 300a may include a single light source 302 configured to output an optical signal. The light source 302 may be a coherent light source (e.g., a laser) or an incoherent light source (e.g., a thermal light, a superluminescent diode, an LED, etc.). In some embodiments, the optical signal output by the light source 302 may pass through a beam splitter 304 configured to split the optical signal into Q optical signals and send each optical signal to a first amplitude modulator 306.
[0058] As in the case of the linear photonic processor 200 of FIG. 2, the first amplitude modulator 306 can be configured to encode the elements of an input vector into the amplitude of the optical signal received from the light source 302. The first amplitude modulator 306 can be a variable attenuator controlled by a DAC (not depicted) or any other suitable amplitude modulator, which can be further controlled by a controller (e.g., the controller 102 of FIG. 1). Several amplitude modulators are known for telecommunications applications and can be used in some embodiments. In some embodiments, a variable beam splitter can be used as the first amplitude modulator 204, where only one output of the variable beam splitter is kept and the other output is discarded or ignored. Other examples of amplitude modulators that can be used in some embodiments include traveling wave modulators, cavity-based modulators, Franz-Keldysh modulators, Plasmon-based modulators, 2D material-based modulators, and nano-opto-electromechanical switches (NOEMS).
[0059] The first amplitude modulator 306 may then output a first optical signal that represents an element of the input vector in terms of the amplitude (e.g., intensity) of the first optical signal. The first optical signal may be sent through the additional beam splitter 304 such that the first optical splitter is split P times. The split first optical signal may then be sent to P second amplitude modulators 308.
[0060] In some embodiments, the second amplitude modulators 308 may be configured to each encode one value of one element of the matrix w. The second amplitude modulators 308 may be the same type of modulator as the first amplitude modulator 306, or may alternatively be a different type of modulator than the first amplitude modulator 306. The received first optical signal is modulated by the second amplitude modulator 208 to encode the matrix elements w pj and vector element x j A second optical signal may be generated having an intensity that represents the multiplication by the amplitude of the output second optical signal.
[0061] In some embodiments, the output second optical signal output by each second amplitude modulator 308 may be sent to and detected using a photodetector 310. The photodetector 310 may be, for example, a photodetector configured to generate a photocurrent proportional to the intensity of light incident on the detector.
[0062] In some embodiments, the outputs of some of the photodetectors 310 can be sent to an electrical summing circuit 312. As shown in the example of FIG. 3A, a row of a matrix (e.g., element w 11 , w 12 and w 13 The outputs of the photodetectors 310, coupled to the second amplitude modulator 308 configured to represent x, y, y, y, and z, are sent to the same electrical summing circuit 312. In such an embodiment, the electrical summing circuit 312 may include circuitry configured to sum the photocurrents received from the photodetectors 310 such that the output electrical signal from the electrical summing circuit 312 represents the product of the input vector and a row of a matrix (e.g., x, y, for the first row of matrix w). 11 +x2w 12 +x3w 13 is expressed electrically).
[0063] In some embodiments, the electrical summing circuit 312 may include a voltage summation circuit. For example, the electrical summing circuit 312 may include a resistor network and an amplifier circuit. In some embodiments, the electrical signals output by the photodetectors 310 may be summed by simply tying the output nodes together. Performing this electrical summation before reading out the output results has two advantages. First, such pre-summing reduces the amount of mixed signal readout circuitry used in the photonic processing architecture, such that only P readout circuitry components are used in the architecture (rather than a total of P×Q circuitry as in the example linear photonic processor 200 of FIG. 2). Second, pre-summing increases the size of the photocurrents that are available for detection by the readout circuitry. Before the optical signals reach each second amplitude modulator 308, the optical signals are split in P ways. Combining the photocurrents generated from the Q different photodetectors can offset the loss in intensity due to splitting (for Q≧P).
[0064] 3B is a block diagram illustrating an example of a linear photonic processor 300b having a layout configured to minimize the length of electrical paths, according to some embodiments of the techniques described herein. Because electrical wires experience parasitic resistance, inductance, and capacitance, it may be useful to design the layout of such wires to minimize these parasitic effects (e.g., by minimizing electrical trace lengths). The electrical signals output by the photodetector 310 are currents, and the speed at which these currents can switch may depend on the electrical parasitics mentioned above. To enable high speed operation, these electrical parasitics must be minimized.
[0065] The linear photonic processor 300b has the same components as the linear photonic processor 300a of the example of Figure 3A, except that the beam splitter 304 is not shown for clarity. However, in the example of Figure 3B, the photodetectors 310 are arranged in a ring configuration around the electrical summing circuit 312. This ring configuration minimizes the length of the electrical traces between the photodetectors 310 and the electrical summing circuit 312, which minimizes electrical parasitics such as resistance, inductance and capacitance.
[0066] I. Nonlinear Relationship Between Codes and Signals An analog computer typically takes a string of bits as input, converts it into some physical process (e.g., electrical process, photonic process) on which it can perform a calculation, and then outputs a string of bits based on one or more measurements of these physical processes.
[0067] In some embodiments, the relationship between the output signal and the output bit string, as well as the relationship between the input bit string and the modulated signal, need not be linear. In fact, a non-linear signal-to-code relationship may be advantageous for certain algorithms. For example, in some deep learning algorithms for image classification, distinguishing between multiple small values may be more valuable, while distinguishing between multiple large values may be less valuable or unnecessary. Thus, such algorithms can be implemented with input DACs and output ADCs that use a large percentage of the codebook to code values near zero and a small percentage of the codebook to code large values near the maximum input / output. The effective dynamic range of the input and output can be expanded by the non-linear mapping.
[0068] II. Error Correction Analog computers may experience errors during calculations. Analog computers using electronics may experience Johnson-Nyquist noise and electrical shot noise, which are essentially noise that can cause errors during the readout process. When operating close to the noise floor of the output readout circuitry, electrical noise may have a low probability of causing a bit flip in the least significant bit (LSB) of the output. The probability of an LSB flip increases with the bandwidth of the readout circuitry. These LSB flip errors are synonymous with gate errors in digital computing circuits (e.g., multiply-accumulate units), but only affect the LSB. Thus, when LSB flip errors do occur, they are only a few percent off the correct output. For example, for an N-bit output, the bit flip error in an LSB is approximately 1 / 2 N This corresponds to a relative error of 100×. This is in contrast to digital computing circuits where a gate error, although unlikely, can cause an error in the most significant bit (MSB) because digital circuits treat all bits equally. In analog processing systems such as the linear photonic processors of FIGS. 2, 3A, and 3B, the LSB has a signal power close to the noise power, while the more significant bits have a signal power exponentially stronger than the noise power. As a result, the more significant bits may be exponentially less likely to tolerate a bit-flip error compared to the LSB.
[0069] One way to mitigate bit-flip errors is to perform error correction during the calculation. The simplest error correction algorithm possible is to increase redundancy by performing the same calculation multiple times (e.g., at least three times) and perform a majority vote to determine the correct result with a higher probability. Due to the nature of errors in analog computers that affect the LSBs, it is not necessary to vote on the entire output bit string. Rather, voting only needs to be done on the LSB or at least a few of the LSBs.
[0070] When performing algorithms that are robust to small errors, such as artificial neural networks or ordinary differential equation solvers, the resilience of the higher bits to errors provides an advantage to analog processors. The inventors have recognized that it is possible to obtain more resilient algorithms using high-speed analog processors, but at the expense of a higher probability of bit-flip errors in the LSB. In some deep learning algorithms (e.g., for image classification), small errors simply reduce the confidence of the prediction, but do not necessarily reduce the prediction accuracy.
[0071] The use of intensity-based optics, such as the linear photonic processors of Figures 2, 3A and 3B, has many advantages, including significantly reduced sensitivity to temperature fluctuations and manufacturing defects. Magnetic field-based photonic systems often require significant stabilization and trimming to function reliably. The proposed architecture above is largely insensitive to such phase errors and fluctuations. Furthermore, the intensity-based architecture is suitable for intensity modulation schemes coupled with phase modulation, since it does not use phase information in its calculations.
[0072] III. Partial products and their sums In some embodiments, the matrix element w ij and vector element x j is expressed using a fixed-point representation. In this representation,
[0073]
number
[0074]
number
[0075]
number
[0076] Building an ADC with high bit precision at a bandwidth corresponding to the speed at which an input vector in the form of an optical signal is sent through a photonic processing system can be a daunting task. Thus, the bit precision of an ADC is typically limited to a factor of 100,000,000 matrix elements w, if perfectly accurate computations are desired. ij and vector element x j In some cases, it may be necessary to limit the bit precision with which w is represented. Therefore, we have developed a method to obtain the output vector with full precision, which can be arbitrarily high, by calculating partial products and sums as described below. For ease of understanding, let w ij or x j Assume that the number of bits required to represent either m1 or m2 is the same, say m1 = m2 = m.
[0077] First, the matrix element w ij and x j The bit-string representation of can be divided into d partitions, each containing k=m / d bits. As a result, the matrix element w ij , w ij =w ij [0] 2 k(d-1) +w ij [1] 2 k(d-2) +···+w ij [d-1] 2 0 It can be written as follows: ij [a] , w ijis the k-bit value of the most significant a-th k-bit string of ij , w ij =w ij [0] w ij [1] ...w ij [d-1] Similarly, x j x j =x j [0] 2 k(d-1) +x j [1] 2 k(d-2) +···+x j [d-1] 2 0 It can be written as follows: j is the bit string in terms of x j =x j [0] x j [1] x j [d-1] x j [d-1] Multiplication
[0078]
number
[0079]
number
[0080] The linear photonic processor then computes the matrices w ij [a] and the input vector x j [b] The matrix vector multiplication produces the following intermediate results:
[0081]
number
[0082]
number
[0083] The above method allows the user to obtain perfectly accurate calculations by manipulating partial products and their sums, even when the available ADC cannot instantly capture with perfect accuracy.
[0084] IV. Generalization to GEMM using tensors via serialization The linear photonic processors described herein (e.g., in relation to Figures 2, 3A and 3B) can be extended in some embodiments from matrix-vector multiplication operations to matrix-matrix multiplication. Given a matrix A of type IxJ and a matrix B of type JxK, the linear photonic processor can be configured to generate a matrix C=AB of type IxK. Matrix A can be encoded in a second amplitude modulator (e.g., second amplitude modulator 208 or 308) in the photonic processor, and column b of matrix B can be encoded in a second amplitude modulator (e.g., second amplitude modulator 208 or 308) to generate a matrix C=AB of type IxK. k can be encoded into a first amplitude modulator (e.g., the first amplitude modulator 204 or 306). The result of each matrix-vector multiplication is represented as column c k b kMatrix C can be constructed column-by-column by performing multiplications on different columns of and storing the results in a memory (e.g., in controller 102). Similarly, matrix B can be encoded into a second amplitude modulator in the photonic processor, where row a of matrix A can be i can be encoded into the first amplitude modulator. The result of the matrix-vector multiplication is then row c of the matrix C. i and the output matrix C can be constructed row by row.
[0085] In some embodiments, matrices may be too large to be encoded in the photonic processor. In this case, a matrix-matrix multiplication may be performed between a portion of the first large input matrix and a portion of the second large input matrix. The result of this multiplication is stored in memory. Subsequently, a second portion of the first large input matrix may be encoded in the photonic processor and a second matrix-matrix multiplication may be performed with the portion of the second large input matrix. This "tiling" of the large matrices may continue until the multiplication process has been performed on all portions of the two large matrices. The results of the multiplication process may then be combined to produce a final result of the multiplication of the two large matrices. This idea of serialization may be applied to tensor-tensor multiplication by processing a slice of a tensor at a time, storing the results in memory, and then later combining the results to form an output tensor.
[0086] V. Applications to neural networks and deep learning The linear photonic processor described herein (e.g., in connection with Figures 2, 3A and 3B) has broad applicability because it can accelerate various GEMM operations and make them more power efficient. Currently, GEMM is used in linear algebra calculations such as performing eigenvalue decomposition, singular value decomposition or matrix inversion. One important application of GEMM is artificial neural networks.
[0087] Deep artificial neural networks, at their most basic level, involve multiple layers, up to hundreds, of tensor-tensor multiplications, each with a linear transformation followed by a nonlinear activation function. Consider a neural network with dimensionality D. O(N D ) elements and D For a weight tensor with N elements, the amount of computation required to perform a tensor-tensor multiplication is O(N 2D-1 ), while the computational complexity required to execute the activation function is O(N D ) Deep neural network computations are therefore typically dominated by tensor-tensor multiplications, and therefore the photonic processor proposed herein can be used to speed up artificial neural network computations.
[0088] Training an artificial neural network typically involves running the backpropagation algorithm. A single-layer deep artificial neural network with a weight matrix w and a bias vector
[0089]
number
[0090]
number
[0091]
number
[0092] Applying the chain rule of calculus, we find the weight matrix associated with this single input vector x, defined here for ease of expression as
[0093]
number
[0094]
number
[0095]
number
[0096]
number
[0097]
number
[0098]
number
[0099]
number
[0100]
number
[0101]
number
[0102]
number
[0103]
number
[0104]
number
[0105]
number
[0106]
number
[0107] VI. Bandwidth vs. Signal Tradeoff The inventors have demonstrated that the linear photonic processor architecture described herein provides a method for, in the current domain, solving problems related to partial product-to-partial product (e.g.,
[0108]
number
[0109] VII. Rescaling Analog computing systems for matrix processing have a finite dynamic range limited by physical noise limitations (e.g., shot noise, thermal noise, etc.) or architectural limitations. In dissipation-based (e.g., light or power dissipation) computing schemes, this dissipation fundamentally impacts the link budget, SNR, and processor accuracy. To get around these limitations and increase the amount of signal transmitted through the system, we realize that it is possible to rescale the rows of a matrix to minimize dissipation while still performing a computation that is directly proportional to the desired computation. A matrix m can be rescaled row by row. In the following, a matrix m is multiplied by a vector x. Each row of m is scaled by a different scale factor α i These scale factors can be, for example, either continuous variables ranging from 0 to infinity or powers of 2. In some embodiments where the rescaling is performed using a digital computer, the scaling and rescaling operations can be performed using either floating point or fixed point numbers. It is also possible for the matrix m to be rescaled on a tile-by-tile or matrix-by-matrix basis. To obtain tile-by-tile scaling, one simply sets different row scale factors to the same value, and to obtain matrix-by-matrix scaling, different tile scale factors can be set to the same value.
[0110]
number
[0111] VIII. Real-valued matrix computation using a positive-only processor. The inventors recognize that analog processors can often only encode positive-valued matrices and tensors. For example, when using an incoherent light source, the photonic processor may only modulate the intensity of the optical signal, and not the phase. Physically, the intensity is a non-negative number. The inventors recognize that in most applications, the photonic processor will perform multiplication between matrices and vectors that may contain positive and / or negative valued elements. Therefore, the inventors have developed a method for performing matrix-vector multiplication operations between real-valued matrices and real-valued vectors using only amplitude modulation by offsetting and / or rescaling the number line.
[0112] Let M be the original real-valued matrix, and let its elements be M ij ∈R, the original real-valued vector is x, and its elements are x j ∈R. Each entry of the original matrix is a constant value c M Offset by M ij '=M ij +c M A new matrix M' can be created such that the constant value c M is the absolute value of the most negative entry of the matrix, e.g.
[0113]
number
[0114] Output Vector
[0115]
number
[0116]
number
[0117]
number
[0118]
number
[0119]
number
[0120]
number
[0121]
number
[0122]
number
[0123] If the first amplitude modulator is capable of encoding negative numbers (but the second amplitude modulator is not), then the second term
[0124]
number
[0125]
number
[0126] The inventors have found that this method does not require encoding negative numbers in the photonic processor, and that it is possible to generate a new matrix M whose elements are outside the encoding range of the photonic processor. ij ' or a new vector x j It is further understood that it is possible to generate a matrix '. Without loss of generality, the input matrix and vector can be normalized such that each entry is within the coding range. For example, the original matrix elements and the original vector elements have values between -1 and 1, say M ij ∈[-1,1] and x j ∈[-1,1]. A constant offset c M and c x Adding , the modified matrix element values are in the new range M ij '∈[-1+c M ,1+c M ] ≥ 0, and the modified vector element values are in the new range x j '∈[-1+c x ,1+c x ] ≧0. If the photonic processor modulator can only encode values between 0 and 1, then in some embodiments the scale factor α M and α x In such an embodiment, α M and α x is the range of possible values of the photonic processor, α M =1 / (1+c M ) and α x =1 / (1+c x ), and α M M ij '∈[(-1+c M ) / (1+c M ),1] and α x x j'∈[(-1+c x ) / (1+c x ),1]. In other words, as mentioned above, y j Instead of calculating
[0127]
number
[0128] The offset and scaling method described above eliminates the need to code negative numbers in the first and second amplitude modulators of the processor. This method also provides the additional benefit of increasing the accumulation of signal at the output. Because the coded matrix and coded vector have non-negative elements, the summed current flow at the output has the same sense, leading to an increase in the amount of charge that is accumulated, which is sampled by the output sampling circuit. This results in a larger signal at the output that encodes the output vector y. The inventors have recognized that a larger signal output is important for processors that typically operate close to the electronic noise floor, which is composed of Johnson-Nyquist noise, electronic shot noise, and photonic shot noise.
[0129] VIII. Loss-based modulation and optical representation of zero values 4A-4C show illustrative examples of amplitude modulators that may be used in some embodiments as the first and / or second amplitude modulators, as described in connection with the examples of FIGS. 2, 3A, and 3B, according to some embodiments of the techniques described herein. FIG. 4A shows a modulator 400a that uses imperfect amplitude or phase modulators 404 and 408 to achieve a "good" zero value in an interferometer configuration. Beam splitters 402 and 410 split and recombine, respectively, the optical signal passing through the modulator 400a. To increase the extinction ratio, a phase shifter 406 can be added to bias the interferometer, and the modulator 408 is used for loss matching. In some embodiments, the modulator 400a may be used in a push-pull mode. FIG. 4B shows a modulator 400b that uses a perfect extinction electroabsorption modulator 412 in some embodiments to increase the extinction ratio of the modulator. 4C illustrates a modulator 400c that uses a ring resonator 414 or a cavity tuned to an off-resonance state. In some embodiments, the ring resonator 414 is a 1 / 2" optical modulator. b It is possible to adjust the frequency to a non-resonant state so that a portion of the frequency passes through at resonance, where b is the desired calculation accuracy.
[0130] Since the goal of such an amplitude modulator is only to modulate the light intensity, almost any intensity modulation method (including, for example, combined phase modulation) can be used for amplitude modulation. To perform more accurately, the extinction ratio of the amplitude modulator should be as large as possible. In some embodiments, it may be desirable to chain modulators (e.g., place modulators in series) to increase the extinction ratio or phase shift. However, as explained below, this accuracy can come at the expense of an effective insertion loss in the resulting output.
[0131] The same matrix-vector multiplication architecture will be applied if one switches any of the intensity modulations from loss-based modulation to gain-based modulation as shown in Figures 4A-4C. This suggests that the same architecture will work for integrated optical circuit platforms that include semiconductor optical amplifiers (e.g., Indium Phosphide (InP) or other III-V semiconductor platforms). In some embodiments, it may be advantageous to select a combination of loss-based and gain-based modulation. Loss-based modulation can increase power efficiency, and gain-based modulation can be used to counter losses in the circuit.
[0132] When using only loss-based modulation schemes, the entries of the input matrix and the entries of the input vector can only have their light intensity reduced. Mathematically, this means that the entries with values less than 1, i.e. x j ≦1 and w pj ≦1. To achieve this, the input matrices and vectors are normalized.
[0133]
number
[0134]
number
[0135]
number
[0136]
number
[0137]
number
[0138]
number
[0139]
number
[0140] Since the signal is encoded in light intensity, the matrix-vector multiplication described above will only apply to matrices and vectors of non-negative values. The algorithm can be augmented by using four multiplications to compute the matrix-vector multiplication between any real-valued matrix and vector of real values. To do this, the input matrix can be split into positive and negative components. For example, w=w + -w - where w + (w - ) corresponds to a matrix that contains only positive (negative) components. Similarly, we can split an input vector into positive and negative components. For example,
[0141]
number
[0142]
number
[0143]
number
[0144]
number
[0145]
number
[0146] Amplitude modulators generally cannot (1) extinguish light perfectly, and (2) pass light perfectly. However, these two properties are important parameters of an amplitude modulator. The former property is related to the extinction ratio, and the latter property is related to the insertion loss of the modulator. At first glance, the techniques described herein require that the amplitude modulators encoding w and x are able to extinguish light perfectly (e.g., modulators with an arbitrarily high extinction ratio). However, to subtract this "common-mode offset" resulting from the imperfect extinction in the w and x modulators, four terms are added:
[0147]
number
[0148] The DMM method described above allows general matrix multiplication using amplitude modulators that have poor extinction ratios and cannot encode values near zero. The jth amplitude modulator, “AM”, has an intensity modulation range x j ∈[x j min >0,x j max<1]. The extinction ratio of this modulator has a finite value x j max / x j min The electronic circuitry that drives the modulator includes a digital-to-analog converter that scales the range x j min ~x j max Similarly, the amplitude modulator "M" in the pth column and jth row is w pj ∈[w pj min >0,w pj max The extinction ratio of this modulator is w pj max / w pj min The circuitry that drives this modulator also has a range of w pj min ~w pj max Discretize with .
[0149] New modulation range x' j ≡x j -x j min is x' for the first amplitude modulator. j ∈[0,x j max -x j min ] range. Also, a new modulation range w' pj ≡w pj -w pj min w' for the second amplitude modulator pj ∈[0,w pj max -w pj min ]. Then, the matrix-vector product can be calculated as follows:
[0150]
number
[0151]
number
[0152]
number
[0153]
number
[0154] IX. wavelength division multiplexing 5 is a block diagram illustrating an example of a linear photonic processor 500 including wavelength division multiplexing (WDM) according to some embodiments of the technology described herein. The linear photonic processor 500 is similar to the linear photonic processors of the examples of FIGS. 2, 3A, and 3B, but is augmented with WDM circuits 506 and 512. Alternatively, in some embodiments, these WDM circuits 506 and 512 can be polarization demultiplexing circuits. The linear photonic processor 500 does not involve interference between any optical paths, facilitating assembly from entirely broadband photonic elements and relaxing the constraints on the wavelength region used for WDM.
[0155] The example linear photonic processor 500 of FIG. 5 is a linear photonic processor that accepts three 3-element vectors xir , x ig and x ib The linear photonic processor 500 is configured to multiplex a 3×3 matrix w having a wavelength of 1000 MHz. The linear photonic processor 500 can include multiple light sources 502 a, 502 b, and 502 c configured to generate optical signals having different wavelengths. For example, the light sources 502 a, 502 b, and 502 c can be configured to generate optical signals that are red, green, and blue, respectively. Each element of the vector x ir , x ig and x ib can therefore be encoded by the first amplitude modulator 504 into the amplitude of optical signals having different wavelengths.
[0156] After passing through the first WDM circuit 506, the first optical signals may be split by a beam splitter 508 and sent to a second amplitude modulator 510 configured to encode the matrix-vector element products into output second optical signals. These output second optical signals may be received by a second WDM 512 and then detected by a photodetector 514 and converted to electrical signals. The electrical signals may be summed and / or read out as previously described herein.
[0157] X. Implementing Signed Values In some embodiments, negative matrix and / or vector values can be realized using circuitry. For example, FIG. 6A is a schematic diagram of a circuit 600a for implementing negative and positive values according to some embodiments of the techniques described herein. The photodetector 606 can have a first terminal and a second terminal (e.g., a cathode and an anode in an embodiment in which the photodetector 606 includes a photodiode as depicted in FIG. 6A). The first terminal can be coupled to either a rail voltage 602 or a summing node through a switch 604. The second terminal can be coupled to either a reference voltage 608 (e.g., ground) or a summing node through a switch 605. The switches 604 and 605 can be, for example, transistor switches. By connecting either the first terminal or the second terminal to the summing node using the switches 604 and 605, the direction of the current output from the photodetector 606 can be changed such that a signage (e.g., positive or negative) is implemented. The switch states of the transistor switches may be controlled by additional control circuitry in the photonic processor and / or additional control circuitry in a controller coupled to the photonic processor (not depicted).
[0158] In addition, as shown in FIG 6B, it is possible to implement signed matrix and / or vector values using an XOR operation to pre-calculate the sign of the calculation and then set the sign of the output electrical signal of the photodetector 606. In some embodiments, the circuit 600b can include an XOR operation 610. The orientation of the cathode (e.g., the orientation of the photodetector 606) is determined by the input vector element x j Sign and matrix element w ij and performing an XOR operation on these signals. The output cathode direction bit can be set whether the current coming from the photodetector is positive or negative (e.g., the output cathode direction bit can trigger a change in the configuration or setting of switches 604 and 605 of circuit 600a).
[0159] The inventors have further developed a method for distributing these "code bits" throughout the photonic processor: the sign bit of each vector element can be transported close to each first amplitude modulator configured to encode the vector element value, where after performing an XOR operation between this vector sign bit and the matrix sign bit, the sign bit signal can be used to control the detector current flow.
[0160] Alternatively or additionally, it may be possible to use electrical communication to distribute the code bits. This communication can be performed using standard digital design techniques to minimize skew and jitter. For example, a single bit can be distributed to many locations simultaneously using a tree or mesh topology. The time delay of this tree can exceed the vector rate of the processor, at which point multiple codes will be sent to the modulator simultaneously.
[0161] The inventors further recognize that the code bits may be photonically distributed. For example, using additional waveguides, it may be possible to encode and transmit the code bits. However, the code bits may also be transmitted using the optical signal of an unused degree of freedom. For example, the optical signal may be modulated with a different polarization or wavelength to encode and distribute the code bits. The code bits may also be encoded in a logical selection of polarization or wavelength. For example, the photonic processor may use light of wavelength λ1 when the vector sign bit is positive, and the photonic processor may use light of wavelength λ2 when the vector sign bit is negative. Photonic code bit distribution provides a number of advantages. First, the sign bit signal and the matrix-vector product signal may propagate at the same propagation speed. Second, the sign bit signal and the matrix-vector product signal propagate in the same optical waveguide, thus precluding the use of additional waveguides in the system.
[0162] FIG. 7 illustrates a polarization-based scheme of code bit distribution in a photonic processor according to some embodiments of the techniques described herein. A first beam splitter 702 can split an input optical signal, and the vector element values can be encoded into the amplitude of the light at a first amplitude modulator 704. The code bits can be encoded into the polarization of the optical signal using a polarization rotator 706 and a code modulator 708. The two optical signals can then be recombined using a polarizing beam splitter 710. It can be seen that a polarizing beam splitter can be used to (1) split two differently polarized input lights into two output paths according to their polarization, and (2) combine the light from the two input paths, each with its own polarization, into a single output light of two different polarizations.
[0163] The recombined optical signal may be split and sent to a second amplitude modulator 712 configured to output a second optical signal representing the vector element-matrix element product. The second optical signal may pass through an additional polarizing beam splitter 716 that allows for separation of the sign 716 and the value 718.
[0164] 8 is a flow chart illustrating a process 800 for implementing signed values according to some embodiments of the techniques described herein. Process 800 may be performed by any suitable computing device. For example, in some embodiments, process 800 may be performed by a controller (e.g., controller 102 as described in connection with FIG. 1) coupled to a photonic processor. In some embodiments, process 800 may be performed by one or more processors located remotely from the photonic processor (e.g., as part of a cloud computing system).
[0165] Process 800 begins at operation 802, where a photodetector can convert a received optical signal into a first electrical signal, which is output by a portion of a photonic processor. The photodetector can include a first terminal and a second terminal. For example, the photodetector can include a photodiode, and the first terminal and the second terminal can be an anode and a cathode, respectively. In some embodiments, the first electrical signal can be a photocurrent.
[0166] In some embodiments, after operation 802, process 800 may proceed to operation 804, where at least one conventional processor coupled to the optical processor may determine whether the first electrical signal represents a positive signed numerical value or a negative signed numerical value. The at least one conventional processor may determine that a sign of the first electrical signal is based at least in part on the signs of the input vector elements and the signs of the input matrix elements. For example, the at least one conventional processor may determine the sign of the first electrical signal using an XOR operation, as described in connection with FIG. 6B.
[0167] After operation 802, process 800 may proceed to operation 806, where control circuitry of the photonic processor may adjust settings of a first switch coupled to a first terminal of the photodetector and a second switch coupled to a second terminal of the photodetector in response to determining whether the first electrical signal represents a positive signed numerical value or a negative signed numerical value. For example, in some embodiments, the first switch and / or the second switch may include transistor switches, and adjusting settings of the first switch and / or the second switch may include applying or removing a gate voltage to enable the first switch and / or the second switch.
[0168] Operation 806 may further proceed to two sub-operations 806a and 806b. In sub-operation 806a, the control circuitry may generate a positive signed numeric output, at least in part, by setting a first switch to couple the first terminal to a reference voltage and setting a second switch to couple the second terminal to a node (e.g., an output node). Setting the first and second switches in this manner may cause the photodetector to output a positive current. In sub-operation 806b, the control circuitry may generate a negative signed numeric output, at least in part, by setting the first switch to couple the first terminal to the node and setting the second switch to couple the second terminal to a voltage rail. Setting the first and second switches in this manner may cause the photodetector to output a negative current.
[0169] After operation 806, process 800 may proceed to operation 808, where the photodetector may output the first electrical signal such that the first electrical signal passes through either the first switch or the second switch. The path of the first electrical signal is based on a determination of whether the previous first electrical signal represents a positive signed numerical value or a negative signed numerical value.
[0170] XI. Submatrix Processing Architecture Matrix-matrix, matrix-vector and tensor-tensor multiplication operations are recursive in nature. 11 ,M 12 ],[M 21 ,M 22 Consider multiplication between a vector x = [x1,x2] and a vector x = [x1,x2]. The output y1 = M 11 x1+M 12 x2 and y2=M 21 x1+M 22 Multiplication by x2 is done by the submatrix M 11 ,M 12 ,M 21 ,M 22 This logic can be considered as a multiplication between the partial vectors x1 and x2. 11,M 12 ,M 21 ,M 22 and x1, x2 can be recursed until they are scalar elements. N ×2 N A matrix of type and size 2 N where N is an integer. Given a matrix of any size, zeros can be added as necessary to meet these size requirements. However, it should be understood that the matrix need not be split into two rows and two columns. The matrix could be split into R rows and C columns, resulting in different scaling.
[0171] Matrix processors can also be configured in this recursive manner using sub-matrix banks. Figures 9A-9D are schematic diagrams illustrating photonic processors 900 arranged into sub-matrix processors, in accordance with some embodiments of the techniques described herein. Each sub-matrix processor 902 is associated with a symbol M (ij) , and the vector 904 is denoted by x (j) 9A-9D, i,j∈{1,2,3,4}, it can be understood that in some embodiments, the sub-matrices and vectors can be of other sizes. The sub-vectors are split and sent to the appropriate sub-matrix processors. A matrix-vector multiplication is performed in each sub-matrix processor between the sub-matrix and the sub-vector at an amplitude modulator 906. The matrix-vector multiplication is then optically sent to a photodetector 908. The photodetector 908 is configured to convert the received optical signal to an electrical signal. The appropriate sub-vector electrical signals can then be summed together between the different sub-matrix processors. In FIGS. 10A-10B, another embodiment of a photonic processor 1000 according to some embodiments is shown. The sub-vector outputs are converted to bit strings locally using an ADC 1010, and the final outputs are summed using digital circuitry (not shown).
[0172] XII. Photonic processing using multiple wavelengths of light Thus far, we have described an architecture for a linear photonic processor that uses electronic circuitry to perform summation. FIG. 11 shows a schematic diagram illustrating an alternative embodiment of a linear photonic processor 1100 configured to perform summation in the optical domain and to use input light having multiple wavelengths, in accordance with some embodiments of the techniques described herein. In the example of FIG. 11, each first amplitude modulator 1102 receives light of different wavelengths, λ1, λ2, . . . , λ1, λ2, . . . Q 11. The vector modulation and matrix modulation methods in the linear photonic processor 1100 are similar to those performed by the linear photonic processors of FIGS. 2, 3A, and 3B. However, in the linear photonic processor 1100, the optical signal is not detected immediately after being modulated by the second amplitude modulator 1104. Rather, the optical signal can be sent to an optical combiner circuit 1106 before being detected by the photodetector 1108. Thus, there are only P photodetectors 1108 in the embodiment of FIG. 11, compared to a total of P×Q photodetectors in the linear photonic processors of FIGS. 2, 3A, and 3B described above.
[0173] FIG. 12A is a schematic diagram of an optical combiner 1106 configured for use with multiple wavelengths of light, according to some embodiments of the techniques described herein. The optical combiner 1106 can include an add-drop ring filter 1210. An optical signal that is resonant with the ring and received from input 1 (e.g., from the second amplitude modulator 1104) can be taken out to the output bus waveguide, and an optical signal that is not resonant with the ring and received from input 2 (e.g., from the bus) passes through the ring to the output bus waveguide. Thus, the ring is tuned to resonate with the wavelength of light arriving at the corresponding second amplitude modulator 1104, but not with any other wavelengths used by the processor. Thus, the ring filter, at least in some embodiments, can have a large enough free spectral range (FSR) to support at least Q total resonances within a single FSR.
[0174] 12B is an illustrative plot of the FSR of the combiner of FIG. 12A in accordance with some embodiments of the techniques described herein. The ring filter can be designed to have a large free spectral range (FSR) but a high enough quality factor so that many of the resonant peaks of the input from the second amplitude modulator are captured.
[0175] 12C is an illustrative plot of transmission as a function of wavelength for several of the combiners of FIG. 12A, according to some embodiments of the techniques described herein. Each combiner can be detuned from the other so that only one wavelength of light is taken from input 1 (e.g., from the second amplitude modulator 1104) to the output. It is also possible to use a y-combiner or other non-resonant multi-wavelength combiner such as a directional coupler to combine output light of different wavelengths at the expense of some optical loss.
[0176] XIV. Sparse matrix and / or vector implementation Sparse matrices (e.g., matrices with only a few non-zero elements) are common in many computing fields. In some embodiments, as shown in the examples of Figures 13A-13C, sparse entries can be implemented in a matrix w or vector x by using electrical or optical switches arranged in the processor architecture. The linear photonic processors 1300a, 1300b, and 1300c are similar to the linear photonic processor 300a as described in connection with Figure 3 and include a light source 1302, a first amplitude modulator 1306, a second amplitude modulator 1308, a photodetector 1310, and an electrical summing circuit 1312. However, the linear photonic processors 1300a, 1300b, and 1300c include additional electrical or optical switches to implement the sparse matrix or vector entries.
[0177] As in the linear photonic processor 1300a of FIG. 13A, the sparse entries of the matrix w can be implemented by placing an electrical switch 1311 after the optical detector 1310. Alternatively or additionally, as shown in the linear photonic processor 1300b of FIG. 13B, an electrical switch 1313 can be placed after the electrical summing circuit 1312 to implement sparse rows in the matrix w. Alternatively or additionally, as shown in the linear photonic processor 1300c of FIG. 13C, an optical switch 1314 can be placed after the first amplitude modulator 1306 to implement sparse entries in the vector x. It can be understood that any combination of switches 1311, 1313 and 1314 can be used in the linear photonic processor to implement sparse matrix and / or vector entries. It is possible to save energy for the sparse matrix entries by making sure that the zero value corresponds to the default state of the second amplitude modulator 1308. In this way, no energy is wasted on applying zeros.
[0178] Alternatively or additionally, the use of precomputation may make it possible to implement sparse or identity submatrices. Analog processors usually benefit from dense (non-sparse) operations that are very computationally intensive. Given a matrix M and a vector x. If the rows of matrix M are sparse, it may be beneficial to perform the operations individually on a digital computing system. For example, if an entire row i of M contains zero entries, the computer can simply iterate over its vector entries x without performing any computations. i For the other extreme, if the entire matrix is an identity matrix, the computer may simply return the vector x. Both examples require no computations, just a simple mapping between inputs and outputs, which can be done efficiently using digital circuitry. A compiler system external to the photonic processor may be implemented to determine whether to compute a particular multiplication between a row of matrix M and vector x using digital circuitry, as in the previous case, or using an analog processor.
[0179] 14 is a flow chart illustrating a process for performing a matrix-vector operation involving sparse vectors and / or matrices according to some embodiments of the techniques described herein. Process 1400 may be performed in part by any suitable computing device in combination with a photonic processor. For example, in some embodiments, process 1400 may be performed by a controller coupled to a photonic processor (e.g., controller 102 as described in connection with FIG. 1 coupled to a photonic processor as described in connection with FIGS. 13A-13C).
[0180] In some embodiments, process 1400 begins with operation 1402, where an input optical signal may be modulated using a first optical modulator. The input optical signal may be provided, for example, by an optical source (e.g., light source 1302 of FIGS. 13A-13C). The input optical signal may be modulated by the first optical modulator to optically represent an element of a vector in a first optical signal output by the first optical modulator. For example, the first optical modulator may be configured to modulate the amplitude (e.g., intensity) of the input optical signal to encode the value of the element of the vector into the amplitude of the light.
[0181] Process 1400 may proceed to operation 1404, where a first optical signal may be modulated by a second optical modulator. The first optical signal may be split (e.g., using a beam splitter) and sent to a number of second optical modulators. The second optical modulators may be programmed with elements of a row of a matrix. The second optical modulators may generate a second optical signal that optically represents the value of an augend (e.g., a product of an element of a vector and an element of a row of a matrix) by modulating the amplitude of the first optical signal. The augends, when summed together, may represent a product between an element of a vector and a row of a matrix.
[0182] After operation 1404, process 1400 may proceed to operation 1406, where the second optical signal may be converted by a photodetector into a plurality of electrical signals of summands. In some embodiments, each optical signal may be received by a photodetector configured to convert the amplitude of the received light into an electrical signal. For example, the photodetector may include a photodetector configured to output a photocurrent having a magnitude proportional to the intensity of the light received by the photodetector. In some embodiments, the multiple second optical signals may be received by a single photodetector (e.g., at the same time or at different times, e.g., by time division multiplexing).
[0183] Next, operation 1400 may proceed to operation 1408, where a switch coupled to the output of the first optical modulator and / or a switch coupled to the output of the optical detector may prevent the transmission of one or more signals. For example, in some embodiments, an optical switch may be coupled to the output of the first optical modulator to prevent the transmission of the first optical signal to the second optical modulator if the value of an element of the vector is equal to zero. The optical switch may, for example, be opened to prevent the transmission of the first optical signal. In this way, a true zero value may be propagated through the photonic processor. Alternatively or in addition, one or more switches coupled to the output of the optical detector may prevent the transmission of the summand electrical signal if the value of one or more elements of a row of the matrix is equal to zero. For example, an electrical switch coupled to the output of the optical detector may be opened if the value of the corresponding element of the row of the matrix is equal to zero. In some embodiments, an additional switch coupled to the output of the multiple optical detectors (e.g., coupled to the output of the electrical summing circuit 1312) may be configured to prevent the transmission of the summed electrical signal if the value of an entire row of the matrix is equal to zero.
[0184] XIV. Signal readout method For large matrices, the above method of electrically bundling the detector outputs to add together the currents of the rows of the matrix can result in significant parasitic capacitances. These parasitic capacitances can load the receiver circuitry and make high speed operation difficult. To avoid this, the inventors have developed several signal readout methods according to several embodiments described herein, as shown in Figures 15A-15D.
[0185] FIG. 15A shows an illustrative readout system 1500a based on optical-electronic-optical conversion, according to some embodiments. In this illustrative readout system 1500a, the matrix-vector element multiplication is performed optically by modulating an optical signal using a first amplitude modulator 1502 and a second amplitude modulator 1504. The optical signal from the second amplitude modulator 1504 can be converted to an electrical signal by a photodetector 1506 and then summed in the current domain. The summed electrical signal can be converted to an amplified voltage using a transimpedance amplifier (TIA) 1508 and then drive an optical modulator 1510. The signal from the optical modulator 1510 can be sent to a photodetector 1512 and converted to an electrical signal. These electrical signals can then be converted to a voltage using a TIA 1514 and then read out as a digital string using an analog-to-digital converter (ADC) 1516. Such an embodiment may enable long-distance routing of subgroup signals (e.g., by routing the signals using photonic waveguides between the optical modulators 1510 and the optical detectors 1512 rather than routing the signals in the electrical domain) without introducing large parasitic capacitances.
[0186] 15B shows an illustrative readout system 1500b based on subgroup current amplification according to some embodiments described herein, which can be performed, for example, using a current mirror circuit 1518. The amplified current can then be converted to a voltage using a TIA 1514 and read out as a digital string using an ADC 1516.
[0187] 15C shows an illustrative readout system 1500c based on intra-subgroup summation in the current domain and subgroup summation in the voltage domain according to some embodiments described herein. The intra-subgroup output of the photodetector 1506 can be performed in the current domain. The intra-subgroup output can be converted to a voltage using a TIA 1508. Larger subgroup summation can then be performed, for example, using an operational amplifier coupled as input to multiple resistors (e.g., R1, R2). The summed voltage signal can be read out as a digital string using an ADC 1522.
[0188] 15D shows an illustrative readout system 1500d based on digital summation according to some embodiments described herein. The subgroup outputs of the photodetectors 1506 can be performed in the current domain and then converted to a digital voltage signal using a TIA and ADC 1524. Digital summation can then be performed by using a digital summer 1526.
[0189] The layout of a photonic processor on a semiconductor substrate can have a large effect on the processor's performance (e.g., speed and / or power). One common method to reduce the capacitance of the readout circuitry (e.g., including the TIA and ADC) is to group the photodetectors and readout circuitry close to each other to reduce the length of the electrical connections. The electrical connections (e.g., via wires) can introduce additional capacitance to the output readout circuitry and limit the gain or bandwidth of the readout circuitry.
[0190] On the other hand, photonic connections (e.g., via photonic waveguides) do not add capacitance to the system. Therefore, another way to lay out a photonic processor on a semiconductor substrate, where devices are typically laid out in a two-dimensional plane, is to have four separate blocks that are mirror images of each other along the x- and y-axes, as shown in layout 1600 in Figures 16A-16D. Optical modulators 1602 and 1604 can be connected to a photodetector 1608 through a waveguide 1606. The mirrored blocks allow the photodetector 1608 to be clustered close to each other, reducing the amount of additional capacitance that would be added to the system if the connections were made electrically.
[0191] If the system can be laid out in a three-dimensional block (on an advanced future semiconductor substrate), it may be advantageous to divide the system into eight separate blocks, each mirrored along the x-, y-, and z-axes (not shown). The output photodetectors can be clustered adjacent to one another. In general, if the system can be laid out in an N-dimensional block, it may be advantageous to divide the system into eight separate blocks, each mirrored along one axis of the N possible dimensions, such that the output photodetectors are clustered adjacent to one another. N It is advantageous to have separate blocks.
[0192] XV. Matrix-matrix operations FIG. 17 is a block diagram of a photonic processor 1700 configured to implement a matrix-matrix operation according to some embodiments of the techniques described herein. The photonic processor 1700 includes two matrix-vector sub-processors. These matrix-vector sub-processors have a similar architecture to the linear photonic processor 300 as described in connection with FIG. 3. Both matrix sub-processors include a light source 1702, a beam splitter 1704, and a first amplitude modulator 1706. The first optical signal output by the first amplitude modulator 1706 can be split by an additional beam splitter 1704 and sent to a second amplitude modulator 1708. The optical signal output by the second amplitude modulator 1708 can be detected by a photodetector 1710 and converted to an electrical signal. The electrical signal output by the photodetector 1710 can be summed by an electrical summing circuit 1717. However, the photonic processor 1700 uses the electrical output of the first sub-processor as an input to the second sub-processor. In some embodiments, the electrical output of the first sub-processor may be amplified by an amplifier 1714 before being sent to the second matrix sub-processor.
[0193] Consider a multiplication between two matrices A and B where the output matrix C=AB. Matrix B can be programmed into the second amplitude modulator 1708 of the first matrix-vector processor, and matrix A can be programmed into the second amplitude modulator 1708 of the second matrix-vector processor. To read out the resulting matrix C, a one-hot vector (e.g., a vector with one entry whose value is one and all other entries whose values are zero) can be programmed into the first amplitude modulator 1706 of the first sub-matrix vector such that only one modulator is on at any single time.
[0194] In some embodiments, one-hot vectors propagate through the photonic processor 1700. As the one-hot vectors propagate through the second amplitude modulator 1708 of the first matrix sub-processor, they carry information representing the columns of matrix B. The columns of matrix B can be sent to and programmed into the first amplitude modulator 1706 of the second matrix processor. The optical signal from the light source 1702 can then send the columns of matrix B to the second amplitude modulator 1708 of the second matrix sub-processor, which is programmed with the elements of matrix A. The output vectors correspond to the columns of the final matrix C. More specifically, the user can receive the vector e i Sending e1 = (a vector with all zeros except for the i-th element, which is 1) will result in the output being the i-th column of C. For example, sending e1 = (1 0 0 0 ...) will return the first column of C. In this way, the multiplication between the columns of matrix B and the elements of matrix A can be performed and stored digitally (e.g., by an external memory). The entire matrix-matrix multiplication operation can be performed by propagating different one-hot vectors through the photonic processor 1700.
[0195] 18 is a flow chart illustrating a process of performing a matrix-matrix operation using a photonic processor according to some embodiments of the techniques described herein. Process 1800 may be performed in part by any suitable computing device in combination with a photonic processor. For example, in some embodiments, process 1800 may be performed by a controller coupled to a photonic processor (e.g., controller 102 as described in connection with FIG. 1 coupled to a photonic processor as described in connection with FIG. 17).
[0196] In some embodiments, process 1800 may begin with operation 1802, where a first matrix may be programmed into a first optical sub-processor. For example, the first matrix may be programmed into a second amplitude modulator (e.g., second amplitude modulator 1708) of the first optical sub-processor. The first matrix may be programmed into the second amplitude modulator, for example, based on a bit string received from an external controller. Individual elements of the first matrix may be programmed into individual amplitude modulators of the second amplitude modulator, respectively. For example, a first matrix element with a value of one may be programmed into a first one of the second amplitude modulators, which may cause the first one of the second amplitude modulators to pass the intensity of a received optical signal through the amplitude modulator without modification. A second matrix element with a value of zero may be programmed into a second one of the second amplitude modulators, which may cause the second one of the amplitude modulators to extinguish the intensity of the received optical signal and output an optical signal having an amplitude at or near zero.
[0197] In some embodiments, process 1800 may next proceed to operation 1804, where the second matrix may be programmed into the second optical sub-processor. The second matrix may be programmed into a second amplitude modulator (e.g., second amplitude modulator 1708) of the second optical sub-processor. The second matrix may be programmed into the second amplitude modulator, for example, based on a bit string received from an external controller. As with the first matrix, individual elements of the second matrix may be programmed into individual amplitude modulators of the second amplitude modulator of the second optical sub-processor. In some embodiments, the second optical sub-processor may include an input coupled to an output of the first optical sub-processor.
[0198] After operation 1804, process 1800 may proceed to operation 1806, where a plurality of one-hot vectors are input to a first optical sub-processor. For example, the plurality of one-hot vectors may be programmed into a first amplitude modulator 1706 of the first optical sub-processor. By propagating an optical signal (e.g., propagating the one-hot vector) from the optical source through the first amplitude modulator to a second amplitude modulator programmed with the first matrix, the first optical sub-processor may propagate a first set of matrix elements (e.g., rows of the matrix, columns of the matrix) to the second optical sub-processor. For example, the optical signal output from the first optical sub-processor may be used to program a first set of matrix elements of the first matrix into a first amplitude modulator of the second optical sub-processor. By propagating another optical signal (e.g., originating from an optical source) through the first amplitude modulator and the second amplitude modulator of the second optical sub-processor, a multiplication may be performed between the first set of matrix elements of the first matrix and the elements of the second matrix.
[0199] In some embodiments, the second optical sub-processor may then output an output vector representing a portion of the multiplication of the first matrix with the second matrix, at operation 1808. For example, the second optical sub-processor may output an electrical signal representing the product of the summed elements of the first matrix with the elements of the second matrix (e.g., from the electrical summation circuit 1712).
[0200] Having thus described several aspects of at least one embodiment of the present technology, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art.
[0201] The above embodiments of the technology described herein may be implemented in any of many ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on any suitable processor or collection of processors, whether located on a single computer or distributed among multiple computers. Such processors may be implemented as integrated circuits, with one or more processors in integrated circuit components, including commercially available integrated circuit components known in the art under names such as CPU chips, GPU chips, microprocessors, microcontrollers, or coprocessors. Alternatively, the processors may be implemented in custom circuitry, such as ASICs, or semi-custom circuitry resulting from the construction of programmable logic devices. As yet a further alternative, the processor may be part of a larger circuit or semiconductor device, whether commercially available, semi-custom, or custom. As a specific example, some commercially available microprocessors have multiple cores such that one or a subset of those cores may constitute a processor. However, the processor may be implemented using circuitry in any suitable format.
[0202] Additionally, the various methods or processes outlined herein may be coded as software executable on one or more processors running any one of a variety of operating systems or platforms. Such software may be written using any of a number of suitable programming languages and / or programming tools, including scripting languages and / or tools. In some cases, such software may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine. Additionally or alternatively, such software may be interpreted.
[0203] The techniques disclosed herein may be embodied as a non-transitory computer readable medium (or multiple computer readable media) (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memories, circuitry in a field programmable gate array or other semiconductor device, or other non-transitory tangible computer storage media) encoded with one or more programs that, when executed on one or more processors, perform methods implementing the various embodiments of the present disclosure described above. The computer readable medium or multiple computer readable media may be portable such that the program or programs stored thereon are loaded onto one or more different computers or other processors to implement the various aspects of the present disclosure described above.
[0204] A computing device may additionally have one or more components and peripherals, including input devices and output devices. These devices may be used, among other things, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visually presenting output and a speaker or other sound generating device for audibly presenting output. Examples of input devices that may be used for a user interface include keyboards and pointing devices such as mice, touchpads, and digitizing tablets. As another example, a computing device may receive input information through voice recognition or in other audible forms. As another example, a computing device may receive input from a camera, lidar, or other device that generates image data.
[0205] An embodiment of a computing device may also include a photonic processor as described herein. The processor of the computing device may communicate with the photonic processor via one or more interfaces. The communicated information may include settings of a detector of the photonic processor and / or measurements from a detector of the photonic processor.
[0206] The terms "program" or "software" are used herein to refer to any type of computer code or set of computer-executable instructions that can be used to program one or more processors to implement various aspects of the present disclosure, such as those described above. Furthermore, in accordance with one aspect of the present embodiments, it should be understood that one or more computer programs that, when executed, perform the methods of the present disclosure need not reside on a single computer or processor, but may be distributed in a modular manner among several different computers or processors to implement various aspects of the present disclosure.
[0207] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or distributed as desired in various embodiments.
[0208] Additionally, the data structures may be stored on the computer-readable medium in any suitable form. For ease of illustration, the data structures may be depicted as having fields that are related through their locations within the data structure. Similarly, such relationships may be achieved by allocating storage for the fields to locations within the computer-readable medium that convey the relationships between the fields. However, relationships between information in fields of the data structures may be established using any suitable mechanism, such as using pointers, tags, or other mechanisms that establish relationships between data elements.
[0209] Various aspects of the technology described herein may be used alone, in combination, or in a variety of configurations not specifically described in the above embodiments, and therefore are not limited in their application to the details and arrangements of components set forth in the above description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.
[0210] The techniques described herein may also be embodied as methods provided herein, examples of which may include those associated with Figures 8, 15, and 19. The acts performed as part of the methods may be ordered in any suitable manner. As such, embodiments may be constructed in which acts are performed in an order different from that shown, which may include performing some acts simultaneously even if shown as sequential acts in the illustrative embodiments.
[0211] Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "including," or "having," "containing," "involving," and variations thereof, as used herein is meant to encompass not only the items listed thereafter and equivalents thereof, but also additional items. The technical ideas that can be understood from the above-described embodiment will be described below as supplementary notes. [Appendix 1] 1. An apparatus for implementing signed numeric values, comprising: a photodetector including a first terminal and a second terminal; a first switch coupling the first terminal of the photodetector to either a node or a reference voltage; a second switch coupling the second terminal of the photodetector to either the node or a voltage rail; A control circuit comprising: generating, at least in part, a positive signed numeric output by configuring the first switch to couple the first terminal to the reference voltage and configuring the second switch to couple the second terminal to the node; and generating, at least in part, a negative signed numeric output by configuring the first switch to couple the first terminal to the node and configuring the second switch to couple the second terminal to the voltage rail. [Appendix 2] the photodetector includes a photodiode; the first terminal comprises an anode; and 2. The apparatus of claim 1, wherein the second terminal includes a cathode. [Appendix 3] 2. The apparatus of claim 1, wherein the first switch and the second switch each include a transistor switch. [Appendix 4] 2. The apparatus of claim 1, wherein the reference voltage is ground. [Appendix 5] 2. The apparatus of claim 1, wherein the control circuitry includes a logic gate configured to output a sign direction bit, the sign direction bit including information indicating whether a numeric output includes a positive signed value or a negative signed value. [Appendix 6] 6. The apparatus of claim 5, wherein the logic gate comprises an XOR gate. [Appendix 7] 6. The apparatus of claim 5, wherein the logic gate is configured to compare a sign of an input vector element value with a sign of an input matrix element value. [Appendix 8] 1. An optical processing system comprising: a first plurality of optical modulators each configured to receive an input optical signal, modulate the input optical signal, and output a first optical signal representing an element of a vector; a second plurality of optical modulators, each optically coupled to an optical modulator of the first plurality of optical modulators and configured to receive the first optical signals, modulate the first optical signals, and output second optical signals representing a portion of a matrix-vector multiplication between the vector and a matrix; a plurality of optical detectors respectively optically coupled to a plurality of optical modulators of the second plurality of optical modulators and configured to convert the second optical signal into an electrical signal representing the portion of the matrix-vector multiplication, each optical detector of the plurality of optical detectors including a first terminal and a second terminal; a first switch coupling the first terminal of the first photodetector to either an output node or a reference voltage; a second switch coupling the second terminal of the first photodetector to either the output node or a voltage rail; A control circuit comprising: generating, at least in part, a positive signed numeric output by configuring the first switch to couple the first terminal of the first photodetector to the reference voltage and configuring the second switch to couple the second terminal of the first photodetector to the output node; and generating, at least in part, a negative signed numerical output by setting the first switch of the first photodetector to couple the first terminal to the output node and setting the second switch of the first photodetector to couple the second terminal to the voltage rail. [Appendix 9] the photodetector includes a photodiode; the first terminal comprises an anode; and 9. The optical processing system of claim 8, wherein the second terminal includes a cathode. [Appendix 10] 10. The optical processing system of claim 9, wherein the first switch and the second switch each include a transistor switch. [Appendix 11] 10. The optical processing system of claim 9, wherein the reference voltage is grounded. [Appendix 12] Further comprising a plurality of electrical summing circuits; a first electrical summing circuit of the plurality is coupled to two or more output nodes, each output node of the two or more output nodes being coupled to a photodetector through the first switch or the second switch; 10. The optical processing system of claim 9, wherein the first electrical summing circuit is configured to output an electrical signal representing the sum of the portions of the matrix-vector operations output by the optical detectors coupled to the two or more output nodes. [Appendix 13] 9. The optical processing system of claim 8, wherein the control circuitry includes a logic gate configured to output a sign direction bit, the sign direction bit including information indicating whether the numeric output includes a positive signed value or a negative signed value. [Appendix 14] 14. The optical processing system of claim 13, wherein the logic gate comprises an XOR gate. [Appendix 15] 14. The optical processing system of claim 13, wherein the logic gate is configured to compare a sign of an input vector element value with a sign of an input matrix element value. [Appendix 16] 1. A method for implementing a signed numeric value output by a photodetector in an optical processor, comprising: converting an output optical signal into a first electrical signal using a photodetector having a first terminal and a second terminal, the output optical signal being output by a portion of the optical processor; determining, using at least one conventional processor coupled to the optical processor, whether the first electrical signal represents a positive signed value or a negative signed value; adjusting, in response to determining whether the first electrical signal represents the positive signed numerical value or the negative signed numerical value, a setting of a first switch coupled to the first terminal and a setting of a second switch coupled to the second terminal using control circuitry of the optical processor, wherein the control circuitry: generating a positive signed numeric output, at least in part, by configuring the first switch to couple the first terminal to a reference voltage and configuring the second switch to couple the second terminal to a node; generating a negative signed numeric output, at least in part, by configuring the first switch to couple the first terminal to the node and configuring the second switch to couple the second terminal to a voltage rail; and outputting the first electrical signal from the photodetector such that the first electrical signal passes through either the first switch or the second switch based on the determination of whether the first electrical signal represents a positive signed value or a negative signed value. [Appendix 17] 17. The method of claim 16, wherein adjusting settings of the first switch and the second switch includes sending one or more electrical signals from the control circuitry to the first switch and the second switch, and the first switch and the second switch each include a transistor switch. [Appendix 18] modulating an input optical signal using a first optical modulator to optically represent elements of a vector in a first optical signal; modulating the first optical signal using a second optical modulator to optically represent an augend in the output optical signal, the augend representing a product between the vector and a row of a matrix when summed with another augend; 17. The method of claim 16, further comprising the step of detecting the output optical signal using the optical detector. [Appendix 19] 17. The method of claim 16, wherein adjusting the setting of the first switch and the setting of the second switch further includes generating a sign direction bit using a logic gate, the sign direction bit including information indicating whether the first electrical signal includes a positive signed value or a negative signed value. [Appendix 20] 20. The method of claim 19, wherein using a logic gate includes using an XOR gate. [Appendix 21] 1. An optical processor for implementing a matrix-vector multiplication operation, comprising: a first plurality of optical modulators each configured to receive an input optical signal, modulate the input optical signal, and output a first optical signal representing an element of a vector; a second plurality of optical modulators optically coupled to an optical modulator of the first plurality of optical modulators and each configured to receive the first optical signal, modulate the first optical signal, and output a second optical signal representing a portion of a matrix-vector multiplication between the vector and a matrix; a plurality of optical detectors coupled to a plurality of optical modulators of the second plurality of optical modulators and each configured to convert the second optical signal into an electrical signal representing the portion of the matrix-vector multiplication; a plurality of switches configured to implement a zero value in the matrix-vector multiplication operation by preventing transmission of an optical or electrical signal if the vector and / or matrix values include zero, wherein a switch of the plurality of switches is coupled to an output of each of the first plurality of optical modulators or each of the plurality of optical detectors. [Appendix 22] 22. The optical processor of claim 21, wherein a switch of the plurality of switches comprises an electrical switch and is coupled to an output of one of the first plurality of optical detectors. [Appendix 23] 23. The optical processor of claim 22, wherein the switch is configured to prevent transmission of an electrical signal if a corresponding value of an element of the matrix is zero. [Appendix 24] 22. The optical processor of claim 21, wherein a switch of the plurality of switches includes an optical switch and is coupled to an output of one of the second plurality of optical modulators. [Appendix 25] 25. The optical processor of claim 24, wherein the switch is configured to prevent transmission of an optical signal if a corresponding value of an element of the vector is zero. [Appendix 26] 22. The optical processor of claim 21, further comprising a plurality of electrical summers coupled to outputs of two or more optical detectors of the plurality of optical detectors and each configured to output an electrical signal representing a sum of the portions of the matrix-vector operations output by the two or more optical detectors. [Appendix 27] 27. The optical processor of claim 26, further comprising another plurality of switches coupled to outputs of the plurality of electrical summers, the switch of the another plurality of switches configured to prevent transmission of the electrical signal representing the sum if all values of elements of a row of the matrix are equal to zero. [Appendix 28] 1. A method for performing a matrix-vector row multiplication operation using an optical processor, comprising: modulating an input optical signal using a first optical modulator to optically represent elements of a vector in a first optical signal; modulating the first optical signal using a second optical modulator to optically represent an augend in a second plurality of optical signals, the augends which, when summed, represent a product between the elements of the vector and a row of a matrix; converting the second plurality of optical signals into a plurality of summand electrical signals using a photodetector; causing a switch coupled to an output of the first optical modulator to prevent transmission of the first optical signal to the second optical modulator if the value of the element of the vector is equal to zero, and / or causing one or more switches coupled to an output of the optical detector to prevent transmission of the augend electrical signal if the value of one or more elements of a row of the matrix is equal to zero. [Appendix 29] summing the augend electrical signals using an electrical summation unit to obtain a product electrical signal; 29. The method of claim 28, further comprising the step of outputting the product electrical signal. [Appendix 30] 30. The method of claim 29, further comprising causing a switch coupled to an output of the electrical summer to prevent transmission of the product electrical signal if all element values of a row of the matrix are equal to zero. [Appendix 31] 29. The method of claim 28, wherein causing the switch coupled to the first optical modulator to prevent transmission of the first optical signal to the second optical modulator if the value of the element of the vector is equal to zero includes opening an optical switch. [Appendix 32] 29. The method of claim 28, wherein causing the one or more switches coupled to the output of the photodetector to prevent transmission of the augend electrical signal if the value of one or more elements of a row of the matrix is equal to zero comprises opening an electrical switch. [Appendix 33] 29. The method of claim 28, wherein modulating the input optical signal using the first optical modulator includes modulating an amplitude of the input optical signal. [Appendix 34] At least one non-transitory computer-readable medium that, when executed by at least one optical processor, causes the optical processor to: modulating an input optical signal using a first optical modulator to optically represent elements of a vector in a first optical signal; modulating the first optical signal using a second optical modulator to optically represent an augend in a second plurality of optical signals, the augends, when summed, representing a product between the vector and a row of a matrix; converting the second plurality of optical signals into a plurality of summand electrical signals using a photodetector; causing a switch coupled to an output of the first optical modulator to prevent transmission of the first optical signal to the second optical modulator if a value of the element of the vector is equal to zero, and / or causing one or more switches coupled to an output of the optical detector to prevent transmission of the augend electrical signal if a value of one or more elements of a row of the matrix is equal to zero. [Appendix 35] The method comprises: summing the augend electrical signals using an electrical summation unit to obtain a product electrical signal; and outputting the product electrical signal. [Appendix 36] 36. The at least one non-transitory computer-readable medium of claim 35, wherein the method further includes causing a switch coupled to an output of the electrical summer to prevent transmission of the product electrical signal if the values of all elements of a row of the matrix are equal to zero. [Appendix 37] 35. The at least one non-transitory computer-readable medium of claim 34, wherein causing the switch coupled to the first optical modulator to prevent transmission of the first optical signal to the second optical modulator if the value of the element of the vector is equal to zero includes opening an optical switch. [Appendix 38] 35. The at least one non-transitory computer-readable medium of claim 34, wherein causing the one or more switches coupled to the output of the photodetector to prevent transmission of the augend electrical signal if the value of one or more elements of a row of the matrix is equal to zero includes opening an electrical switch. [Appendix 39] 35. The at least one non-transitory computer-readable medium of claim 34, wherein modulating the input optical signal using the first optical modulator includes modulating an amplitude of the input optical signal. [Appendix 40] 1. A method for performing a matrix-matrix operation using an optical processor, comprising the steps of: programming a first matrix into a first optical sub-processor; programming a second matrix into a second optical sub-processor, the second optical sub-processor including an input coupled to an output of the first optical sub-processor; A method comprising the steps of inputting a plurality of one-hot vectors as an optical signal to the first optical sub-processor, and outputting from the second optical sub-processor an output vector representing a portion of a multiplication of the first matrix and the second matrix. [Appendix 41] 41. The method of claim 40, further comprising outputting from the first optical sub-processor a plurality of electrical signals representing the multiplication between the one-hot vector and the first matrix. [Appendix 42] receiving, in the second optical sub-processor, the plurality of electrical signals from the first optical sub-processor; 42. The method of claim 41, further comprising: modulating an input optical signal using the plurality of electrical signals received from the first optical sub-processor using an optical modulator of the second optical sub-processor. [Appendix 43] 42. The method of claim 41, further comprising, after outputting the plurality of electrical signals representing the multiplication between the one-hot vector and the first matrix, amplifying the plurality of electrical signals using one or more amplifiers. [Appendix 44] 1. An optical processor configured to perform a matrix-matrix operation, comprising: a first optical sub-processor configured to optically perform a matrix-vector multiplication of the one-hot vector and a first matrix to obtain the first vector; a second optical sub-processor configured to receive an output signal from the first optical sub-processor and to optically perform a matrix-vector multiplication of the first vector and a second matrix. [Appendix 45] 1. An optical processor configured to perform a matrix-matrix operation, comprising: a first optical sub-processor, a first plurality of optical modulators, each configured to receive an input optical signal, modulate the input optical signal, and output a first optical signal representing an element of a one-hot vector; a second plurality of optical modulators optically coupled to an optical modulator of the first plurality of optical modulators and each configured to receive the first optical signal, modulate the first optical signal, and output a second optical signal representing a portion of a matrix-vector multiplication between the one-hot vector and a first matrix; a first plurality of optical detectors coupled to an optical modulator of the second plurality of optical modulators and each configured to convert the second optical signal into an electrical signal representing the portion of the matrix-vector multiplication; the first optical sub-processor including a first plurality of electrical summers coupled to outputs of two or more optical detectors of the first plurality of optical detectors and each configured to output an electrical signal representative of an element of a vector resulting from summing a portion of the matrix-vector multiplication; a second optical sub-processor, a third plurality of optical modulators each receiving an output electrical signal from an electrical summing unit of the plurality of electrical summing units of the first optical sub-processor, the third plurality of optical modulators each configured to receive an input optical signal, modulate the input optical signal according to the received output electrical signal, and output a third optical signal representing the element of the vector; a fourth plurality of optical modulators optically coupled to an optical modulator of the third plurality of optical modulators and each configured to receive the third optical signal representing an element of the vector, modulate the third optical signal, and output a fourth optical signal representing a portion of a matrix-matrix multiplication between the first matrix and the second matrix; a second plurality of optical detectors, each coupled to an optical modulator of the third plurality of optical modulators and configured to convert the third optical signal into an electrical signal representing a portion of the matrix-matrix multiplication; and a second optical sub-processor including a second plurality of electrical summers coupled to outputs of two or more optical detectors of the second plurality of optical detectors and each configured to output an electrical signal representative of an element of a matrix resulting from summing a portion of the matrix-matrix multiplication.
Claims
1. 1. An apparatus for implementing signed numeric values, comprising: a photodetector including a first terminal and a second terminal; a first switch coupling the first terminal of the photodetector to either a node or a reference voltage; a second switch coupling the second terminal of the photodetector to either the node or a voltage rail; A control circuit comprising: generating, at least in part, a positive signed numeric output by configuring the first switch to couple the first terminal to the reference voltage and configuring the second switch to couple the second terminal to the node; and generating, at least in part, a negative signed numeric output by configuring the first switch to couple the first terminal to the node and configuring the second switch to couple the second terminal to the voltage rail.
2. the photodetector includes a photodiode; the first terminal comprises an anode; and The apparatus of claim 1 , wherein the second terminal comprises a cathode.
3. The apparatus of claim 1 , wherein the first switch and the second switch each comprise a transistor switch.
4. The apparatus of claim 1 , wherein the reference voltage is ground.
5. 2. The apparatus of claim 1 , wherein the control circuitry includes a logic gate configured to output a sign direction bit, the sign direction bit including information indicating whether a numeric output generated by the control circuitry includes a positive signed numerical value or a negative signed numerical value.
6. The apparatus of claim 5 , wherein the logic gate comprises an XOR gate.
7. The apparatus of claim 5 , wherein the logic gate is configured to compare a sign of an input vector element value with a sign of an input matrix element value.
8. 1. An optical processing system comprising: a first plurality of optical modulators each configured to receive an input optical signal, modulate the input optical signal, and output a first optical signal representing an element of a vector; a second plurality of optical modulators, each optically coupled to an optical modulator of the first plurality of optical modulators and configured to receive the first optical signals, modulate the first optical signals, and output second optical signals representing a portion of a matrix-vector multiplication between the vector and a matrix; a plurality of optical detectors respectively optically coupled to a plurality of optical modulators of the second plurality of optical modulators and configured to convert the second optical signal into an electrical signal representing the portion of the matrix-vector multiplication, each optical detector of the plurality of optical detectors including a first terminal and a second terminal; a first switch coupling the first terminal of the first photodetector to either an output node or a reference voltage; a second switch coupling the second terminal of the first photodetector to either the output node or a voltage rail; A control circuit comprising: generating, at least in part, a positive signed numeric output by configuring the first switch to couple the first terminal of the first photodetector to the reference voltage and configuring the second switch to couple the second terminal of the first photodetector to the output node; and generating, at least in part, a negative signed numerical output by setting the first switch of the first photodetector to couple the first terminal to the output node and setting the second switch of the first photodetector to couple the second terminal to the voltage rail.
9. the photodetector includes a photodiode; the first terminal comprises an anode; and The optical processing system of claim 8 , wherein the second terminal comprises a cathode.
10. 10. The optical processing system of claim 9, wherein the first switch and the second switch each comprise a transistor switch.
11. The optical processing system of claim 9 , wherein the reference voltage is ground.
12. Further comprising a plurality of electrical summing circuits; a first electrical summing circuit of the plurality is coupled to two or more output nodes, each output node of the two or more output nodes being coupled to a photodetector through the first switch or the second switch; 10. The optical processing system of claim 9, wherein the first electrical summing circuit is configured to output an electrical signal representing the sum of the portions of the matrix-vector multiplications output by the photodetectors coupled to the two or more output nodes.
13. 9. The optical processing system of claim 8, wherein the control circuitry includes a logic gate configured to output a sign direction bit, the sign direction bit including information indicating whether a numeric output generated by the control circuitry includes a positive signed numerical value or a negative signed numerical value.
14. The optical processing system of claim 13 , wherein the logic gate comprises an XOR gate.
15. 14. The optical processing system of claim 13, wherein the logic gate is configured to compare the sign of an input vector element value with the sign of an input matrix element value.
16. 1. A method for implementing a signed numeric value output by a photodetector in an optical processor, comprising: converting an output optical signal into a first electrical signal using a photodetector having a first terminal and a second terminal, the output optical signal being output by a portion of the optical processor; determining, using at least one conventional processor coupled to the optical processor, whether the first electrical signal represents a positive signed value or a negative signed value; adjusting, using control circuitry of the optical processor, a setting of a first switch coupled to the first terminal and a setting of a second switch coupled to the second terminal in response to determining whether the first electrical signal represents the positive signed numerical value or the negative signed numerical value, wherein the control circuitry: generating a positive signed numeric output, at least in part, by configuring the first switch to couple the first terminal to a reference voltage and configuring the second switch to couple the second terminal to a node; generating a negative signed numeric output, at least in part, by configuring the first switch to couple the first terminal to the node and the second switch to couple the second terminal to a voltage rail; and outputting the first electrical signal from the photodetector such that the first electrical signal passes through either the first switch or the second switch based on the determination of whether the first electrical signal represents a positive signed value or a negative signed value.
17. 17. The method of claim 16, wherein adjusting settings of the first switch and the second switch comprises sending one or more electrical signals from the control circuitry to the first switch and the second switch, and wherein the first switch and the second switch each comprise a transistor switch.
18. modulating an input optical signal using a first optical modulator to optically represent elements of a vector in a first optical signal; modulating the first optical signal using a second optical modulator to optically represent an augend in the output optical signal, the augend representing a product between the vector and a row of a matrix when summed with another augend; detecting the output optical signal using the optical detector; 20. The method of claim 16, further comprising:
19. 17. The method of claim 16, wherein adjusting the setting of the first switch and the setting of the second switch further comprises generating a sign direction bit using a logic gate, the sign direction bit comprising information indicating whether the first electrical signal comprises a positive signed value or a negative signed value.
20. 20. The method of claim 19, wherein using the logic gate comprises using an XOR gate.
Citation Information
Patent Citations
Optical neuro-computer
JP1990045815A
Optical computing method
JP1990059915A
Semiconductor-laser driving device, method, optical module and optical transmitter
JP2009194138A
Semiconductor device, driver and semiconductor relay
JP2015177204A
