Optical in-memory computing systems

Abstract

According to some embodiments of the present disclosure, optical computing systems are disclosed that employ optical sources, such as optical frequency combs, and optical memories to perform computations such as Matrix-Vector Multiplication (MVM) operations or Matrix-Matrix Multiplication (MMM) operations. According to some embodiments of the present disclosure, optical computing systems are disclosed that employ optical sources, such as optical frequency combs, optical memories, and components configured to perform hyperspectral operations to increase throughput.

Description

UNITED STATES PATENT APPLICATIONFOROPTICAL IN-MEMORY COMPUTING SYSTEMSINVENTOR[S]:Myoung-Gyun Suh, Monterey Park, CA, USA Byoung Jun Park, Sunnyvale, CA, USA Mostafa Honari, Sunnyvale, CA, USA Yoshihisa Yamamoto, Sunnyvale, CA, USAAttorney Docket No.: NTTR-2023005pctTITLEOPTICAL IN-MEMORY COMPUTING SYSTEMSCROSS-REFERENCE TO RELATED APPLICATION^]

[0001] This application claims priority to US Provisional Patent Application Serial No. 63 / 590,429, filed on October 14, 2023, which is incorporated herein by reference in its entirety.FIELD

[0002] The present disclosure generally relates to computing systems, and more particularly to optical systems for performing computational operations.BACKGROUND

[0003] The rapid advancements in machine learning across numerous industries have amplified the demand for extensive matrix- vector multiplication operations, thereby challenging the capacities of traditional von Neumann computing architectures. To address this, researchers are currently exploring alternatives such as in-memory computing systems to develop faster and more energy-efficient hardware. In particular, there is renewed interest in computing systems that are based on optics, which could potentially handle matrix- vector multiplication in a more energy-efficient way. Despite promising initial results, developing a highly parallel, programmable, and scalable optical computing system capable of rivaling electronic computing hardware remains a challenge.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] FIG. 1 illustrates a high-level process / system for performing hyperspectral inmemory computing according to some embodiments.

[0005] FIG. 2 illustrates an example optical computing system according to some embodiments.

[0006] FIG. 3 illustrates another example optical computing system according to some embodiments.

[0007] FIG. 4 illustrates another example optical computing system according to some embodiments.

[0008] FIG. 5 illustrates another example optical computing system according to some embodiments.

[0009] FIG. 6 is an example setup for an optical computing system that can perform an optical matrix-vector multiplication via mixed space-frequency multiplexing according to some embodiments.

[0010] FIG. 7 is an example setup for an optical computing system that may perform a matrix-matrix multiplication via a hyperspectral operation according to some embodiments.

[0011] FIG. 8 illustrates an example chip that includes SLMs and photodetectors according to some embodiments.

[0012] FIG. 9 illustrates a three-dimensional (3D) opto-electronic neural network according to some embodiments.

[0013] FIG. 10 illustrates differences between an example conventional electronic computing-in-memory (CIM), and an example opto-electronic hyperspectral CIM in accordance with some embodiments.

[0014] FIG. 11 illustrates two stacking approaches for stacking an example opto-electronic hyperspectral CIM onto a substrate with high-band memory (HBM) units according to some embodiments.DETAILED DESCRIPTION

[0015] According to various embodiments of the present disclosure, in-memory computing architectures (herein optical computing systems) are disclosed herein that uses spatial light modulators as programmable optical memory, thereby boosting the computational throughput and energy efficiency. In some embodiments, the optical computing systems may employ hyperspectral operations that integrate space multiplexing with frequency multiplexing of optical frequency combs to further boost computational throughput and energy efficiency. The optical computing systems to be disclosed herein have been experimentally demonstrated to perform multiply-accumulate operations with higher than 4- bit precision in both matrix- vector and matrix-matrix multiplications, which suggests the systems’ potential for a wide variety of deep learning and optimization tasks. These systems exhibit extraordinary modularity, scalability, and programmability, effectively transcending the traditional limitations of optics-based computing architectures. The optical computingsystems to be disclosed herein have the potential to scale beyond peta operations per second, marking a significant step towards achieving high-throughput cncrgy-cfficicnt optical computing.

[0016] The recent breakthroughs in machine learning, exemplified by Al models such as ChatGPT, have revolutionized a myriad of industries such as healthcare, finance, retail, automative, and manufacturing. These transformations have led to a surge in demand for extensive matrix- vector multiplication (MVM) operations that are essential for a wide range of algorithms, especially in deep learning and large-scale optimization. This growing computational requirement has presented challenges for traditional von Neumann computing architectures, driving researchers to explore alternate approaches, such as in-memory computing. By incorporating non-volatile memory elements within the processor, inmemory computing effectively addresses the performance bottleneck arising from the separation of memory and processing units in conventional architectures. This leads to more efficient data movement, reduced power consumption, and the capability for highly parallel computations.

[0017] In some embodiments, the systems to be described herein may employ hyperspectral in-memory computing, which may boost computational throughput by combining space and frequency multiplexing with intensity modulation at the computational clock frequency. In some embodiments, computations may be primarily executed on a two-dimensional spatial light modulator (SLM), functioning as a programmable optical memory, where the weight matrix either remains static or experiences slow updates. This setup may allow optics to predominantly handle energy-efficient and parallel data movement, while electronics ensure the systems’ programmability. Importantly, the system architectures leverages scalable optical computing methods and components to substantially increase computational throughput. This strategy may include the integration of hyperspectral operations using optical frequency combs (OFCs), that, with coherently-locked massively parallel laser comb lines, could facilitate both frequency and space multiplexing.

[0018] FIG. 1 illustrates a high-level concept of hyperspectral in-memory computing system in accordance with some embodiments. More particularly, FIG. 1 illustrates a high- level process / system for efficiently performing multiply-accumulate (MAC) operations by storing weight-matrix in a 2D optical memory. Computational throughput is boasted throughthe integration of space, frequency, and time multiplexing. FIG. 1 shows an example of hypcrspcctral operation, in which spatial and frequency multiplexing of the output from a light source, such as an optical frequency comb (OFC), was performed over time (e.g., time multiplexing) on a hyperspectral plane 10. In the example illustrated in FIG. 1, the hyperspectral plane 10 includes a two-dimensional array of hyperspectral plane (HP) pixels 12. In this example, five distinct optical frequency lines may be transmitted through each HP pixel 12 for concurrently transmitting or relaying five different information bits / packets.

[0019] The hyperspectral plane 10 is propagated through the hyperspectral computing system by first going through a 2D optical memory 20 that includes a 2D array of optical memory (OM) pixels 22 (e.g., pixel elements). As will be further described herein, in some embodiments, the 2D optical memory 20 may be implemented using one or more spatial light modulators (SLMs). Each OM pixel 22 may modulate (e.g., light phase modulation, light amplitude modulation, or light polarization modulation) to perform a multiplication operation on the five optical frequency lines of a corresponding HP pixel 12. That is, and as noted before, each HP pixel 12 includes five distinct frequency lines, so each of the OM pixel 22 may modulate five different frequency lines at any given moment in time.

[0020] After the multiplication operation is performed by the 2D optical memory 20, the different frequency lines may be sorted out to each frequency, and an accumulate operation may be performed by a 2D detector 30 to complete, for example, a multiply-accumulate operation. In essence, the frequency sorting for the accumulate operation may be performed between the 2D optical memory 20 and the 2D detector 30 using, for example, free-space optics (e.g., gratings, cylindrical lenses, and so forth).

[0021] As briefly noted above, embodiments of the present disclosure are directed to optical computing systems that can perform computational operations such as Matrix-Vector Multiplication (MVM) or Matrix-Matrix Multiplication (MMM) commonly performed in a variety of applications such as Al-deep learning (e.g., neural networking) and other optimization tasks. More particularly, the optical computing systems may perform computationally intensive operations using, for example, an optical source that generates multiple optical frequency lines such as optical frequency combs to increase throughput and perform the computational operations in an optical memory, which offers various advantages (e.g., energy as well as computational advantages). In some embodiments,hyperspectral operations may be performed in some embodiments of the systems to perform certain computational operations such as MMM.

[0022] According to some embodiments of the present disclosure, an optical computing system is disclosed herein that may be used to perform MVM operations. For the embodiments, such an optical computing system may include an optical source to generate a plurality of optical frequency lines having different light frequencies. The optical computing system may also include one or more optical modulators to modulate the plurality of optical frequency lines to encode the optical frequency lines with input vector information.

[0023] Embodiments of the optical computing system may also include an optical demultiplexer to spatially separate the optical frequency lines. Embodiments of the optical computing system may also include an optical element to fan out the spatially separated optical frequency lines in a direction perpendicular to the frequency separation direction of the optical demultiplexer. Embodiments of the optical computing system may also include a programmable optical memory device to encode a matrix and that includes an array of pixel elements to receive the separated optical frequency lines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate a respective optical frequency line directed to the respective pixel to output a respective attenuated optical frequency line.

[0024] Embodiments of the optical computing system may also include an optical multiplexer to spatially combine the attenuated optical frequency lines outputted by the programmable optical memory device in the direction of frequency separation. Embodiments of the optical computing system may also include a photodetector device including an array of photodetector elements to record the spatially combined attenuated optical frequency lines.

[0025] In some embodiments, the optical source comprises an array of coherent light sources. In some embodiments, the array of coherent light sources comprises an optical frequency comb. In some embodiments, the array of coherent light sources comprises a vertical cavity surface-emitting laser (VCSEL).

[0026] In some embodiments, the optical source to amplitude, phase, or frequency modulate the plurality of optical frequency lines. In some embodiments, the optical element is acylindrical lens. In some embodiments, the programmable optical memory device to amplitude, phase, or frequency modulate the separated optical frequency lines.

[0027] In some embodiments, the plurality of optical frequency lines are modulated by electro-optic modulators. In some embodiments, each optical frequency line generated by the optical source has a linewidth narrower than 100 MHz. In some embodiments, the optical frequency lines are evenly spaced apart at greater than 1 GHz.

[0028] In some embodiments, the optical demultiplexer and the optical multiplexer each includes at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIPA). In some embodiments, the programmable optical memory device comprises a spatial light modulator (SLM).

[0029] In some embodiments, the programmable optical memory device comprises a two- dimensional array of pixel elements. In some embodiments, the programmable optical memory device comprises pixel elements having pixel-to-pixel pitch size smaller than 100 microns. In some embodiments, the programmable optical memory device has an operational wavelength range wider than 10 nanometer (nm). In some embodiments, the array of photodetector elements of the photodetector device are characterized by pixel-to- pixel pitch size of less than 100 microns.

[0030] In some embodiments, the optical computing system is a closed loop system. For these embodiments, the array of photodetector elements of the photodetector device is electronically coupled to the array of pixel elements of the programmable optical memory device such that output of the array of photodetector elements are fed back to the array of pixel elements of the programmable optical memory device. In some embodiments, the photodetector elements of the photodetector device may be used to at least one of photodetection, nonlinear operation, or optical modulation.

[0031] In some embodiments, each pixel element of the programmable optical memory device is disposed on a first side of the photodetector device, and each photodetector element is disposed on a second side of the photodetector device directly opposite from a respective pixel element on the first side.

[0032] In some embodiments, the pixel elements of the programmable optical memory device are to photo detect and optically modulate, and to interface with optical signals, and are on same or different sides of the photodetector device. In some embodiments, the arrayof pixel elements of the programmable optical memory device comprises of a two- dimensional array of phase change material or a two-dimensional array of vertical cavity surface emitting laser (VCSEL) elements.

[0033] In some embodiments, the photodetector device includes an electrical nonlinearity and gain layer disposed between the array of optical modulator pixel elements of the programmable optical memory device and the array of photodetector elements. In some embodiments, the array of photodetector elements may be characterized by pixel-to-pixel pitch size of less than 100 microns.

[0034] According to various embodiments of the present disclosure a second optical computing system is disclosed herein that may be used to perform MMM operations. For the embodiments, the second optical computing system includes an optical source to generate a plurality of optical frequency lines having different light frequencies and to intensity modulate the plurality of optical frequency lines to form an optical pulse stream. Embodiments of the second optical computing system may also include a first optical demultiplexer to spatially separate the optical frequency lines in one spatial direction.

[0035] Embodiments of the second optical computing system may also include a first optical element to evenly fan-out the spatially separated optical frequency lines in a direction perpendicular to the frequency separation spatial direction of the first optical demultiplexer. Embodiments of the second optical computing system may also include a first programmable optical memory device to encode a first matrix and that includes an array of pixel elements to receive the separated optical frequency lines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate a respective optical frequency line directed to the respective pixel to output attenuated optical frequency line.

[0036] Embodiments of the second optical computing system may also include an optical multiplexer to spatially combine attenuated optical frequency lines outputted by the first programmable memory device. Embodiments of the second optical computing system may also include a second optical element to evenly fan out the combined optical frequency lines in the frequency separation direction of the first optical demultiplexer. Embodiments of the second optical computing system may also include a second programmable optical memory device to encode a second matrix and that includes an array of pixel elements to receive thecombined optical frequency lines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate respective optical frequency lines directed to the respective pixel to output respective attenuated optical frequency lines.

[0037] Embodiments of the second optical computing system may also include a third optical element to evenly fan-in the combined attenuated optical frequency lines in a direction perpendicular to the fan-out direction of the second optical element. Embodiments of the second optical computing system may also include a second optical demultiplexer to spatially separate the attenuated optical frequency lines in the fan-in direction of the third optical element. Embodiments of the second optical computing system may also include a photodetector device including an array of photodetector elements to record the separated attenuated optical frequency lines, the separated attenuated optical frequency lines indicating the multiplication result of the first matrix and the second matrix.

[0038] In some embodiments, the optical computing system may be a closed loop system in which the photodetector elements of the photodetector device is electronically coupled to the pixel elements of the first programmable optical memory device such that an output of the photodetector elements of the photodetector device is fed back to the pixel elements of the first programmable optical memory device. In some embodiments, the photodetector elements of the photodetector device to at least one of photodetection, nonlinear operation, or optical modulation.

[0039] In some embodiments, the photodetector elements of the photodetector device are to photo detect and to optically modulate, and to interface with optical signals, and is on same side or different side of the photodetector device. In some embodiments, the pixel elements of the first programmable optical memory device are optical modulator pixels of the first programmable optical memory device, wherein each optical modulator pixel is disposed on a first side of the photodetector device, and wherein each photodetector element is disposed on a second side of the photodetector device directly opposite from a respective optical modulator pixel on the first side. In some embodiments, the photodetector device includes an electrical nonlinearity and gain layer disposed between the optical modulator pixels and the array of photodetector elements.

[0040] In some embodiments, the optical source comprises an array of coherent light sources. In some implementations, the array of coherent light sources comprises an opticalfrequency comb. In some implementations, the array of coherent light sources comprises a vertical cavity surface-emitting laser (VCSEL). In some implementations, the optical source to amplitude, phase, or frequency modulate the plurality of optical frequency lines.

[0041] In some embodiments, the first programmable optical memory device to amplitude, phase, or frequency modulate the spatially separated optical frequency lines. In some implementations, the second programmable optical memory devices to amplitude, phase, or frequency modulate the fanned out combined optical frequency lines. In some implementations, the photodetector elements of the photodetector device are to photo detect and to optically modulate, and to interface with optical signals, and is on same side or different side of the photodetector device.

[0042] In some embodiments, the array of photodetector elements of the photodetector device are characterized by pixel-to-pixel pitch size of less than 100 microns.

[0043] In some embodiments, the optical source to amplitude, phase, or frequency modulate the plurality of optical frequency lines. In some embodiments, the first programmable optical memory device to amplitude, phase, or frequency modulate the spatially separated optical frequency lines. In some embodiments, the second programmable optical memory devices to amplitude, phase, or frequency modulate the combined optical frequency lines.

[0044] In some embodiments the optical source incudes electro-optic modulators to modulate the plurality of optical frequency lines. In some embodiments, each optical frequency line generated by the optical source has a linewidth narrower than 100 MHz. In some embodiments, the optical frequency lines are evenly spaced apart at greater than 1 GHz.

[0045] In some embodiments, the first optical demultiplexer, the second optical demultiplexer, and the optical multiplexer may include at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIPA). In some embodiments, the first and second programmable optical memory devices each may include a spatial light modulator (SLM).

[0046] In some embodiments, the first and second programmable optical memory devices each may include a two-dimensional array of pixel elements. In some embodiments, the first and second programmable optical memory devices each may include pixel elements having pixel-to-pixel size smaller than 100 microns. In some embodiments, the first and secondprogrammable optical memory devices each has operational wavelength range wider than 10 nanometers. In some embodiments, the first and second optical elements may be cylindrical lens.

[0047] FIG. 2 illustrates an example optical computing system according to various embodiments of the present disclosure. For the embodiments, the example optical computing system 100a is an open loop system that includes an optical source 102, one or more optical modulators 104, an optical demultiplexer 106, an optical element 108, a programable optical memory device 110 that includes an array of pixel elements 112, an optical multiplexer 114, and a photodetector device 116. In various embodiments, the optical computing system 100a may be employed to perform, among other operations, matrixvector multiplication (MVM) operations.

[0048] When the optical computing system 100a is in operation and in accordance with various embodiments, the optical source 102 may generate a plurality of optical frequency lines with different light frequencies. For these embodiments, the optical source 102 may amplitude, phase, or frequency modulate the plurality of optical frequency lines to encode information. In some embodiments, the optical source 102 may comprise of an array of coherent light sources. For these embodiments, the array of coherent light sources may comprise an optical frequency comb (OFC). In some cases, the array of coherent light sources may include at least a vertical cavity surface-emitting laser (VCSEL). In some cases, each optical frequency line generated by the optical source 102 has a linewidth narrower than 100 MHz. In some implementations, the optical frequency lines are evenly spaced apart at greater than 1 GHz.

[0049] In various embodiments, the one or more optical modulators 104 may modulate the plurality of optical frequency lines generated by the optical source to encode the optical frequency lines with input vector information. In some embodiments, the one or more optical modulators 104 may be one or more electro-optic modulators.

[0050] In various embodiments, the optical demultiplexer 106 may spatially separate the optical frequency lines. In some cases, the optical multiplexers may include one or more gratings.

[0051] In various embodiments, the optical element 108 may fan out the spatially separated optical frequency lines in a direction perpendicular to the frequency separation direction ofthe optical demultiplexer. In some implementations, the optical element may be a cylindrical lens.

[0052] In various embodiments, the programable optical memory device 110 that includes an array of pixel elements 112 may encode a matrix and the array of pixel elements may receive the separated optical frequency lines outputted by the optical element 108. In some embodiments, each pixel element of the array of pixel elements may have a respective attenuation weight to attenuate a respective optical frequency line directed to the respective pixel to output a respective attenuated optical frequency line. In some cases, the programmable optical memory device 110 is configured to amplitude, phase, or frequency modulate the separated optical frequency lines. In some implementations, the programmable optical memory device 110 may comprise a spatial light modulator (SLM).

[0053] In various embodiments, the programmable optical memory device 110 may comprise a two-dimensional array of pixel elements (e.g., OM pixels 22 of FIG. 1). In some implementations, the programmable optical memory device may comprise pixel elements having pixel-to-pixel pitch size smaller than 100 microns. In some implementations, the programmable optical memory device has an operational wavelength range wider than 10 nanometer (nm).

[0054] In various embodiments, the optical multiplexer 114 may spatially combine the attenuated optical frequency lines outputted by the programmable optical memory device 110 in the direction of frequency separation. In various embodiments, the optical demultiplexer 106 and the optical multiplexer 114 may each comprise at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIPA).

[0055] In various embodiments, the photodetector device 116 comprises an array of photodetector elements to record the spatially combined attenuated optical frequency lines. In doing so, an accumulation operation may be performed as illustrated, for example, in FIG. 1 by a 2D (two-dimensional) detector 30. In some implementations, the array of photodetector elements of the photodetector device 116 may be characterized by pixel-to- pixel pitch size of less than 100 microns

[0056] F illustrates an example of another optical computing system according to various embodiments of the present disclosure. For these embodiments, the example optical computing system 100b is, in essence, a closed-looped version of the optical computingsystem 100a of FIG. 2 that may be employed to perform MVM operations, similar to optical computing systcmlOOa. The optical computing system 100b includes an optical source 102’ (similar or the same as the optical source 102 of FIG. 2), one or more optical modulators 104’ (similar or the same as the one or more optical modulators 104 of FIG. 2), an optical demultiplexer 106’ (similar or the same as the optical demultiplexer 106 of FIG. 2), an optical element 108’ (similar or the same as the optical element 108 of FIG. 2) , a photodetector device 116’ (which includes an array of pixel elements 112’, an array of photodetector elements 118’, and an electrical nonlinearity / gain layer 120’), and an optical multiplexer 114’ (similar or the same as the optical multiplexer 114 of FIG. 2). Note that in various embodiments, the photoreactor device 116’ may be implemented in a chip (e.g., a substrate) as illustrated, for example, in FIG. 8.

[0057] In the closed-loop system of FIG. 3, output of the array of photodetector elements 118’ is looped back to the array of pixel elements 112’ (e.g., programmable optical memory device) via, for example, electrical nonlinearity / gain layer 120’. That is, and in accordance with various embodiments, the array of photodetector elements 118’ of photodetector device 116’ (which generally corresponds to the photodetector elements 118 of the photodetector device 116 of FIG. 2) is electronically coupled to the array of pixel elements 112’ (which generally corresponds to the pixel elements 112 of the programmable optical memory device 110 of FIG. 2) such that an output of the array of photodetector elements 118 is fed back to the array of pixel elements 112’. This feature is more clearly discussed below with respect to FIG. 8. That is, the photodetector device 116’ comprises a portion (e.g., photodetector elements 118') that performs the functions of a photodetector and another portion (e.g., pixel elements 112’) that performs the functions of a programmable optical memory device, such as the programmable optical memory device 110 of FIG. 2.

[0058] In some implementations, the photodetector elements 118’ of the photodetector device 116’ may be used to at least one of photodetection, nonlinear operation, or optical modulation.

[0059] In some embodiments, each pixel element 112’ of the programmable optical memory device note that the pixel elements 112’ constitutes the programmable optical memory device integrated into the photodetector device 116’ of FIG. 3) is disposed on a first side of the photodetector device 116’, and each photodetector element 118’ is disposed on a secondside of the photodetector device 1 16’ directly opposite from a respective pixel element 1 12’ on the first side of the photodetector device 116’. In some embodiments, the pixel elements 112’ (which may constitute a programmable optical memory device) may be configured to photo detect and optically modulate, and to interface with optical signals, and are on same or different sides of the photodetector device 116’.

[0060] In some embodiment, the array of pixel elements 112’ (as illustrated for example in FIG. 3) of the programmable optical memory device may comprise of a two-dimensional array of phase change material or a two-dimensional array of vertical cavity surface emitting laser (VCSEL) elements. In some embodiments, the photodetector device 116’ may include an electrical nonlinearity and gain layer disposed between the array of pixel elements 112’ of the programmable optical memory device and the array of photodetector elements 118’.

[0061] FIG. 4 illustrates an example of another optical computing system according to various embodiments of the present disclosure. For these embodiments, the example computing system 200a is an open loop system that includes an optical source 202, a first optical demultiplexer 206, a first optical element 208, a first programmable optical memory device 210, which includes a first array of pixel elements 212, an optical multiplexer 214, a second optical element 216, a second programmable optical memory 218, which includes a second array of pixel elements 220, a third optical element 222, a second optical demultiplexer 224, and a photodetector device 226. In various embodiments, the computing system 200a may be particularly suited for performing matrix-matrix multiplication (MMM) operations among other types of operations.

[0062] In various embodiments, when the computing system 200a is in operation, the optical source 202 may generate a plurality of optical frequency lines having different light frequencies and may intensity modulate the plurality of optical frequency lines to form an optical pulse stream. In various implementations, the optical source 202 may include an array of coherent light sources. In some implementations, the array of coherent light sources may comprise an optical frequency comb (OFC). In some implementations, the array of coherent light sources may comprise a vertical cavity surface-emitting laser (VCSEL).

[0063] In various embodiments, the optical source 202 may be configured to amplitude, phase, or frequency modulate the plurality of optical frequency lines. In some implementations, the optical source 202 may include electro-optic modulators to modulatethe plurality of optical frequency lines. In some embodiments, each optical frequency line generated by the optical source 202 may have a linewidth narrower than 100 MHz. In some embodiments, the optical frequency lines generated by the optical source 202 may be evenly spaced apart at greater than 1 GHz

[0064] Referring back to FIG. 4, the first optical demultiplexer 206 may be configured to spatially separate the optical frequency lines outputted by the optical source 202 in one spatial direction. In some implementations, the first optical demultiplexer may include at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIP A).

[0065] In various embodiments, the first optical element 208 may be configured to spatially separate the optical frequency lines in one spatial direction. In some implementations, the first optical element 208 may be a cylindrical lens.

[0066] In various embodiments the first programmable optical memory device 210 may be configured to encode a first matrix and that includes an array of pixel elements to receive the separated optical frequency lines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate a respective optical frequency line directed to the respective pixel to output attenuated optical frequency line. In some implementations, the first programmable optical memory device 210 is configured to amplitude, phase, or frequency modulate the spatially separated optical frequency lines. In some embodiments, the first programmable optical memory device 210 may comprise an SLM. In some embodiments, the first programmable optical memory device 210 may comprise a two- dimensional array of pixel elements.

[0067] In various embodiments, the optical multiplexer 214 may be configured to spatially combine attenuated optical frequency lines outputted by the first programmable memory device. In some implementations, the optical multiplexer 214 may include at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIP A).

[0068] In various embodiments, the second optical element 216 may be designed to evenly fan out the combined optical frequency lines in the frequency separation direction of the first optical demultiplexer. In some implementations, the second optical element 216 may be a cylindrical lens.

[0069] In various embodiments, the second programmable optical memory device 218 may be configured to encode a second matrix and that includes an array of pixel elements to receive the combined optical frequency lines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate respective optical frequency lines directed to the respective pixel to output respective attenuated optical frequency lines. In some embodiments, the second programmable optical memory device 218 may be configured to amplitude, phase, or frequency modulate the fanned out combined optical frequency lines outputted by the second optical element 216. In some implementations, the second programmable optical memory device 218 may comprise an SLM. In some implementations, the second programmable optical memory device 218 may comprise a two-dimensional array of pixel elements.

[0070] In various embodiments, the first and second programmable optical memory devices 210 and 218 may each comprise pixel elements having pixel-to-pixel size smaller than 100 microns. In some embodiments, In some embodiments, the first and second programmable optical memory devices each has operational wavelength range wider than 10 nanometers.

[0071] In various embodiments, the third optical element 222 may be configured to evenly fan-in the combined attenuated optical frequency lines in a direction perpendicular to the fan-out direction of the second optical element 216. In some implementations, the third optical element 222 may be a cylindrical lens.

[0072] In various embodiments, the second optical demultiplexer 224 may be configured to spatially separate the attenuated optical frequency lines in the fan-in direction of the third optical element 222. In some implementations, the second optical demultiplexer 224 may include at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIP A).

[0073] In various embodiments, the photodetector device 226 may comprise of an array of photodetector elements to record the separated attenuated optical frequency lines, the separated attenuated optical frequency lines indicating the multiplication result of the first matrix and the second matrix. In some implementations, the array of photodetector elements of the photodetector device may be characterized by pixel-to-pixel pitch size of less than 100 microns.

[0074] FIG. 5 illustrates an example of another optical computing system according to various embodiments of the present disclosure. For these embodiments, the example computing system 200b is, in essence, a closed-loop version of the optical computing system 200a of FIG. 4 that may be employed to perform MMM operations, similar to the optical computing system 200a. The optical computing system 200b includes an optical source 202’ (similar or the same functions as the optical source 202 FIG. 4), a first optical demultiplexer 206’ (similar or the same as the first optical demultiplexer 206 of FIG. 4), a first optical element 208’ (similar or the same as the first optical element 208 of FIG. 4), a photodetector device 226’ (which includes a first array of pixel elements 212’ that represents a first programmable optical memory device for the optical computing system 200b and that corresponds to the first array of pixel elements 212 of the programmable optical memory device 210 of FIG.4), an array of photodetector elements 228’ that corresponds to the array of photodetector elements 228 of FIG. 4, and an electrical nonlinearity / gain layer 230’), an optical multiplexer 214’ (similar or the same as the optical multiplexer 214 of FIG. 4)1, a second optical element 216’ (similar or the same as the second optical element 216), a programmable optical memory device 218’ (which includes a second array of pixel elements 220’), a third optical element 222’ (similar or the same as the third optical element 222 of FIG. 4), and a second optical demultiplexer 224’ (similar or the same as the second optical demultiplexer 224 of FIG. 4). Because the array of pixel elements 212’ of the photodetector device 226’ may correspond to a first programmable optical memory device for the optical computing device 200b, the programmable optical memory device 218’ may also be referred to herein as a “second” programmable optical memory device 218’. Note that in various embodiments, the photoreactor device 126’ may be implemented in a chip (e.g., a substrate). In various embodiments, the optical computing systems 200a and 200b of FIGS. 4 and 5 may be employed to perform MMM operations. In the closed-loop system of FIG. 5, an output of the photodetector elements 228’ is looped back to the pixel elements 112’ (e.g., a programmable optical memory device) via, for example, electrical nonlinearity / gain layer 230’.

[0075] In various embodiments, the photodetector elements 228’ of the photodetector device 226’ of the closed-loop optical computing system 200b is electronically coupled to the pixel elements 212’of the first programmable optical memory device such that an output of thephotodetector elements 228’ of the photodetector device 226’ is fed back to the pixel elements 212’. Note again that in FIG. 5, the first pixel elements 212’ included in the photodetector device 226’ represents the pixel elements of a first programmable optical memory device of the optical computing system 202’, whereas the second pixel elements 220’ in FIG. 5 is the pixel elements of the second programmable optical memory device (the programmable optical memory device 218’) of the optical computing system 202’.

[0076] In some implementations, the photodetector elements 228’of the photodetector device 226’ may be used to at least one of photodetection, nonlinear operation, or optical modulation. In some implementations, the photodetector elements 228’of the photodetector device 226’ are to photo detect and optically modulate, and to interface with optical signals, and is on same side or different side of the photodetector device 226’. More particularly, in some cases, the photodetector device 226’ may be integrated in a chip or substrate.

[0077] In various embodiments, the pixel elements 221’ (which corresponds to the first programmable optical memory device of the optical computing system 200b of FIG. 5) are optical modulator pixels of the first programmable optical memory device, wherein each optical modulator pixel is disposed on a first side of the photodetector device, 226’ and wherein each photodetector element is disposed on a second side of the photodetector device 226’ directly opposite from a respective optical modulator pixel on the first side of the photodetector device 226’. In some implementations, the photodetector device 226’ may include an electrical nonlinearity and gain layer disposed between the optical modulator pixels (e.g., first pixel elements 212’ of FIG. 5) and the array of photodetector elements 218’.

[0078] FIG. 6 is an example setup for an optical computing system that can perform an optical matrix-vector multiplication (MVM) via mixed space-frequency multiplexing according to various embodiments. For the embodiments, the optical computing system 600 includes an optical source 610 (which further includes a fiber optic frequency comb (OFC) 612, a waveshaper 614, an erbium-doped fiber amplifier (EDFA) 616, and a electro-optic (EO) modulator 618), a collimator 622, a photodetection (PD) 622, an optical spectral analyzer (OSA) 624, an oscilloscope (OSC) 626, beam expander 630 (which comprises a pair of lens), a pair of gratings 632, mirrors 634 and 652, a cylindrical lens 636, beam splitters 638 and 646, doublet lens 640, 654, and 660, a polarizer 642, a spatial lightmodulator (SLM) 644, pair of gratings 650, a two-dimensional (2D) camera 662, and a onedimensional (ID) camera 656.

[0079] In various embodiments, the optical source 610 may, among other things, generate optical frequency lines and, via the waveshaper 616, amplitude, phase, or frequency modulate the optical frequency lines to encode the optical frequency lines with input vector. In the embodiment illustrated in FIG. 6, the pair of gratings 632 function as a demultiplexer. The output of the pair of gratings 632 may be fanned out vertically by the cylindrical lens 636, which may be a cylindrical concave lens. The SLIM 644, which defines matrix elements Mij, may perform element-wise multiplication by attenuation. More particularly, the SLM 644 may encode the matrix elements (Mij) as attenuation weights for multiplication with the input vector. The doublet lenses 640, 654, and 660, and the beam expander 430 may be employed to minimize aberration and image plane distortion on the SLM surface. In FIG. 6, the polarizer 642 may be used to operate a phase-only SLM for intensity modulation. The ID camera 656 may be used to acquire accumulated data from optics such as the grating, while the 2D camera 662, as well as the OSA 624 and the OSC 626, may be used for monitoring system performance and may not be necessary for operational performances. Similarly, some of the optical components such as some of the mirrors and lenses may be optional in various alternative implementations.

[0080] FIG. 7 is an example setup for an optical computing system that may perform a matrix-matrix multiplication via a hyperspectral operation according to various embodiments. For the embodiments, the first portion of the optical computing system 700, outside of a hyperspectral portion 760, includes an optical source 710, a collimator 720, a beam expander 730 (which comprises a pair of lens), mirrors 732, 736, and 756, a grating 734, a concave cylindrical lens 738 (for vertical fan-out), a beam splitter (BS) 740, doublet lens 742 and 750, a polarizer 744, a first SLM 746, a flip mirror 748, a two-dimensional camera 752, and three gratings 754. These components of the first portion of the optical computing system 700 generally mirror the components of the optical computing system 600 of FIG. 6, performing similar functions as the corresponding components of the optical computing system 600. In contrast, the second portion 760 (hereinafter “hyperspectral portion 760”) in FIG. 7 includes components for hyperspectral operation that enables a batch processing of MVM multiplication.

[0081] As illustrated, the hyperspectral portion 760 includes a concave cylinder 760, a beam splitter 762 doublet lenses 764, 772, and 780, a polarizer 766, a second SLM 768, another flip mirror 770, a 2D camera 774, a convex cylinder 776, a grating 778, and another 2D camera 782. In various embodiments, concave cylinder 760 horizontally fan-out optical frequency lines, while concave cylinder 776 vertically fan-in optical frequency lines. The grating 778 may frequency sort vertically.

[0082] In various embodiments, the first portion of the optical computing system 700 on the left side of FIG. 7 and outside of the hyperspectral portion 760 generally corresponds to the MVM setup, similar to FIG. 6, and the hyperspectral portion 760 enables an MM operation through hyperspectral multiply-accumulate process. In various embodiments, matrices are encoded onto the first SLM 746 and the second SLM, and the resultant matrix is captured by the 2D camera782. The optical computing system 700 is able to perform matrix-matrix multiplication (MMM) operations by utilizing hyperspectral encoding, where each SLM pixel encodes a matrix weight across multiple comb lines. This is fundamentally a batch processing of matrix-vector multiplication using wavelength-division multiplexing.

[0083] FIG. 7 also shows a close-up view 760 of two optical lines 792 outputted by BS 762. As a result of a hyperspectral operation, each optical line 792 is actually made of multiple lines that have been spatially and frequency multiplexed as indicated by different shaded lines that are transmitted through the same spatial pixel.

[0084] FIG. 8 illustrates an example chip that includes SLMs and photodetectors to facilitate close-loop systems, such as those illustrated in FIGS. 3 and 5 according to some embodiments. More specifically, the chip 800 (which comprises a substrate) may be employed to implement the photodetector device 116’ or 226’ of FIGS. 3 or 5 according to some implementations. The chip 800 includes a spatial light modulator (SLM) layer 802, a photodetector layer 806, and an electrical nonlinearity / gain layer 804 disposed between the spatial light modulator layer 802 and the electrical nonlinearity / gain layer 804. The spatial light modulator layer 802 may embody a programmable optical memory that comprises a plurality of pixel elements. Each of the pixel elements may have a respective attenuation weight to attenuate a respective optical frequency line directed to the respective pixel to output a respective attenuated optical frequency line. In some cases, this may entail the pixel elements modulating the optical frequency lines that are directed to the pixel elements byamplitude, phase, or frequency modulation. Note that in FIG. 8, the arow above the chip 800 represents the frequency lines directed to the pixel elements of the spatial light modulator layer 802 being modulated and outputted by the pixel elements.

[0085] The photodetector layer 806 includes a plurality of photodetector elements that may be used to detect and record optical frequency lines directed to the photodetector elements. As illustrated, each photodetector element of the photodetector layer 806 is electrically connected to a respective pixel element of the SLM layer 802 through a respective via of the electrical nonlinearity / gain layer 804.

[0086] Referring to FIG. 9, which illustrates a three-dimensional (3D) opto-electronic neural network according to various embodiments. The 3D opto-electronic neural network 900, which is an open loop system, may comprise a plurality of neural network layers (hereinafter “layers”) including a first layer 904, a second layer 906, and a third layer 908, as illustrated in FIG. 9. As one of ordinary skill in the art will recognize, such a neural network may, in reality, comprise of many more layers than those depicted in FIG. 9. These layers 904, 906, and 906 represents a cascaded system.

[0087] As illustrated, each layer 904, 906, and 908 includes a first free-space optics 910, an optical memory 912, a second free-space optics 914, and an opto-electronic neuron 916. The first free-space optics 910 and the second free-space optics 914 may each include various optical components such as gratings, cylindrical lens, mirrors, and so forth as previously described. In various embodiments, the optical memory 912, which may include weight Matrix M, may be implemented with an array of pixel elements. The opto-electronic neuron 916 having two opposing sides and that includes, on one side of the opto-electronic neuron 916 that faces the second free-space optics 914, photodetector pixels or elements, and on the opposite side, optical light emitters or modulators.

[0088] In the example illustrated in FIG. 9, the first layer 904 may receive from an optical source 902 four optical lines of an optical frequency comb, each optical line includes four different frequency lines modulated by the optical ring modulators under each optical line. The optical memory 912 may attenuate the four frequency lines. In some embodiments, the optical memory 912 may optically modulate the four frequency lines. The outputs of the optical memory may be demultiplexed and sorted out to the opto-electronic neuron 916.

[0089] FIG. 10 illustrates differences between an example conventional electronic compute- in-memory (CIM), and an example opto-clcctronic hypcrspcctral CIM in accordance with some embodiments. As illustrated, the electronic CIM 1010, which may be implemented in a chip, includes a two-dimensional (2D) N x N array of memory cells (depicted as being a square or rectangular blocks). In an electronic CIM 1010, each memory cell is implemented with an electronic circuit. In the FIG. 10, all of the cells except for the cells in the far-right column are for O(N2) multiplication operations. The cells on the far-right column are for O(N) accumulate operations to accumulate the results of the multiplication operations performed by memory cells in the same row to the left of the respective accumulate cell in the far-right column.

[0090] In contrast to the electronic CIM 1010, the example opto-electronic hyperspectral CIM 1000 includes two chips, a first chip 1002 and a second chip 1004. In some embodiments, the first chip 1002 may be a synapse chip that includes a 2D array of optical memory pixels that may perform O(N3) hyperspectral multiply operations. That is, each optical memory pixel (e.g., as represented by the rectangular or square shapes on the first chip 1002) may concurrently perform multiple multiply operations on multiple optical frequency lines being concurrently transmitted through the same optical memory pixel, thus substantially increasing the throughput of the opto-electronic hyperspectral CIM 1000. In various embodiments, the second chip 1004 may be a neuron chip that includes a 2D optical-electrical-optical (OEO) pixel array, where each pixel may perform O(N2) accumulate operations. In various embodiments, the output of the first chip 1002 (e.g., a synapse chip with a 2D array of optical memory pixels) may be demultiplexed and sorted out for transmission to the appropriate pixels of the OEO pixel array of the second chip 1004 as illustrated by the thin lines between the first chip 1002 and the second chip 1004 in FIG.10.

[0091] The conventional electronic CIM 1010 may can be characterized as having certain features and limitations. For example, the electronic CIM 1010 can have “massive” scaling via nano-scale lithography. It normally requires physical wiring for MAC operation, which is energy inefficient due power dissipation during operation. 3D stacking is generally challenging due to issues with physical wiring and thermal design power (TDP) limits. Theelectronic CIM 1010 may also be limited to a single operation per matrix array at a time (which may be < 10 Tcra operations per second (T0PS) / mm2).

[0092] In contrast, the optoelectronic hyperspectral CIM 1000 may have certain characteristics and features that under at least certain circumstances may be more advantageous to those features / characteristics of electronic CIM 1010. For example, although the optoelectronic hyperspectral CIM 1000 is also able to be massively scaled via nano-scale lithography, just like the electronic CIM 1010, it can additionally operate with frequency multiplexing thus substantially increasing throughput. Unlike the electronic CIM 1010, the optoelectronic hyperspectral CIM 1000 does not require physical wiring for MAC operations and enables 3D architecture, thus minimizing its 2D chip footprint. In various embodiments, the use of a synapse chip to perform parallel multiply operations and the use of a neuron chip to perform, for example, accumulation and nonlinear operation may advantageously result in efficient use of chip area (> Peta OPS / mm2). For the embodiments, the optoelectronic hyperspectral CIM 1000 may also be able to perform hyperspectral operation per matrix array at a time. Finally, the optoelectronic hyperspectral CIM 1000 permits computations to be performed locally at each pixel, thus minimizing power dissipation that occurs in electrical wires.

[0093] Referring to FIG. 11, which illustrates two stacking approaches / structures for connecting the pixel array of the second chip 1004 of the optoelectronic hyperspectral CIM 1000 of FIG. 10 to a plurality of high-band memory (HBM) units according to various embodiments. In various embodiments, because of the high throughput of the optoelectronic hyperspectral CIM 1000, the use of HBM units may be necessary.

[0094] In the first structure 1100, the second chip 1004 of the optoelectronic hyperspectral CIM 1000 may be disposed atop a substrate 1102 (e.g., a chip). In various embodiments, the substrate 1102 may be a silicon interposer. The substrate 1102 includes a plurality of HBM units that surrounds the second chip 1004. Thus, the pixel array of the second chip 1004 may be electrically connected to HBM units laterally.

[0095] In the second structure 1110, the second chip 1104 is stacked atop the HBMs on the substrate 1102, thus the pixel array of the second chip 1004 is electrically connected to the HBM units vertically. As a result, the second structure has a smaller 2D footprint, lower latency, and lower power dissipation compared to structure 1100.

[0096] Structures 1 100 and 1110 may be compatible with TSMC SoIC-X, CoWoS, and SoW technologies. These systems may meet readout bandwidth of greater than 100 Tbps (e.g., 100 wavelengths x 1000 pixel counts in one axis x 1 GHz).

[0097] After reviewing the present disclosure, an individual of ordinary skill in the art will immediately appreciate that some details and features can be added, removed and / or changed without deviating from the spirit of the invention. Reference throughout this specification to "one embodiment," "an embodiment," “additional embodiment(s)” or “some embodiments,” “various embodiments,” “one implementation,” “some implementation,” “various implementations,” means that a particular feature, structure or characteristic described in connection with the embodiment(s) or implementation(s) is included in at least one or some embodiment(s) and / or implementation(s), but not necessarily all embodiments and / or implementations, such that the references do not necessarily refer to the same embodiment (s) / implementation(s). Note that the words embodiment(s) and implementation(s) may be used interchangeably, and therefore, are synonymous unless indicated otherwise. Furthermore, the particular features, steps, structures, or characteristics may be combined in any suitable manner in one or more embodiments and / or implementations. These and other changes can be made to the embodiments and / or implementations in light of the above- detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments and / or implementations disclosed in the specification and the claims, but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled.

Claims

CLAIMS1. An optical computing system, comprising: an optical source to generate a plurality of optical frequency lines having different light frequencies; one or more optical modulators to modulate the plurality of optical frequency lines to encode the optical frequency lines with input vector information; an optical demultiplexer to spatially separate the optical frequency lines; an optical element to fan out the spatially separated optical frequency lines in a direction perpendicular to the frequency separation direction of the optical demultiplexer; a programmable optical memory device to encode a matrix and that includes an array of pixel elements to receive the separated optical frequency lines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate a respective optical frequency line directed to the respective pixel to output a respective attenuated optical frequency line; an optical multiplexer to spatially combine the attenuated optical frequency lines outputted by the programmable optical memory device in the direction of frequency separation; and a photodetector device comprising an array of photodetector elements to record the spatially combined attenuated optical frequency lines.

2. The optical computing system of claim 1, wherein the optical source comprises an array of coherent light sources.

3. The optical computing system of claim 2, wherein the array of coherent light sources comprises an optical frequency comb.

4. The optical computing system of claim 2, wherein the array of coherent light sources comprises a vertical cavity surface-emitting laser (VCSEL).

5. The optical computing system of claim 1 , wherein the optical source to amplitude, phase, or frequency modulate the plurality of optical frequency lines.

6. The optical computing system of claim 1, wherein the plurality of optical frequency lines are modulated by electro-optic modulators.

7. The optical computing system of claim 1, wherein each optical frequency line generated by the optical source has a linewidth narrower than 100 MHz.

8. The optical computing system of claim 1, wherein the optical frequency lines are evenly spaced apart at greater than 1 GHz.

9. The optical computing system of claim 1, wherein the optical demultiplexer and the optical multiplexer each comprises at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIPA).

10. The optical computing system of claim 1 , wherein the optical element is a cylindrical lens.

11. The optical computing system of claim 1, wherein the programmable optical memory device is configured to amplitude, phase, or frequency modulate the separated optical frequency lines.

12. The optical computing system of claim 1, wherein the programmable optical memory device comprises a spatial light modulator (SLM).

13. The optical computing system of claim 1, wherein the programmable optical memory device comprises a two-dimensional array of pixel elements.

14. The optical computing system of claim 1, wherein the programmable optical memory device comprises pixel elements having pixel-to-pixel pitch size smaller than 100 microns.

15. The optical computing system of claim 1, wherein the programmable optical memory device has an operational wavelength range wider than 10 nanometer (nm).

16. The optical computing system of claim 1, wherein the array of photodetector elements of the photodetector device are characterized by pixel-to-pixel pitch size of less than 100 microns.

17. The optical computing system of claim 1, wherein the optical computing system is a closed loop system in which the array of photodetector elements of the photodetector device is electronically coupled to the array of pixel elements of the programmable optical memory device such that output of the array of photodetector elements are fed back to the array of pixel elements of the programmable optical memory device.

18. The computing system of claim 17, wherein the photodetector elements of the photodetector device are used to at least one of photodetection, nonlinear operation, or optical modulation.

19. The optical computing system of claim 17, wherein the photodetector elements of the photodetector device are used to photo detect and optically modulate, and to interface with optical signals, and is on same side or different side of the photodetector device.

20. The optical computing system of claim 17, wherein each pixel element of the programmable optical memory device is disposed on a first side of the photodetector device, and each photodetector element is disposed on a second side of the photodetector device directly opposite from a respective pixel element on the first side.

21. The optical computing system of claim 17, wherein the pixel elements of the programmable optical memory device are to photo detect and optically modulate, and to interface with optical signals, and are on same or different sides of the photodetector device.

22. The optical computing system of claim 17, wherein the array of pixel elements of the programmable optical memory device comprises of a two-dimensional array of phase change material or a two-dimensional array of vertical cavity surface emitting laser (VCSEL) elements.

23. The optical computing system of claim 17, wherein the photodetector device includes an electrical nonlinearity and gain layer disposed between the array of optical modulator pixel elements of the programmable optical memory device and the array of photodetector elements.

24. An optical computing system, comprising: an optical source to generate a plurality of optical frequency lines having different light frequencies and to intensity modulate the plurality of optical frequency lines to form an optical pulse stream; a first optical demultiplexer to spatially separate the optical frequency lines in one spatial direction; a first optical element to evenly fan-out the spatially separated optical frequency lines in a direction perpendicular to the frequency separation spatial direction of the first optical demultiplexer; a first programmable optical memory device to encode a first matrix and that includes an array of pixel elements to receive the separated optical frequency lines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate a respective optical frequency line directed to the respective pixel to output attenuated optical frequency line; an optical multiplexer to spatially combine attenuated optical frequency lines outputted by the first programmable memory device; a second optical element to evenly fan out the combined optical frequency lines in the frequency separation direction of the first optical demultiplexer; a second programmable optical memory device to encode a second matrix and that includes an array of pixel elements to receive the combined optical frequencylines, each pixel element of the array of pixel elements having a respective attenuation weight to attenuate respective optical frequency lines directed to the respective pixel to output respective attenuated optical frequency lines; a third optical element to evenly fan-in the combined attenuated optical frequency lines in a direction perpendicular to the fan-out direction of the second optical element; a second optical demultiplexer to spatially separate the attenuated optical frequency lines in the fan-in direction of the third optical element; and a photodetector device comprising an array of photodetector elements to record the separated attenuated optical frequency lines, the separated attenuated optical frequency lines indicating the multiplication result of the first matrix and the second matrix.

25. The optical computing system of claim 24, wherein the optical source comprises an array of coherent light sources.

26. The optical computing system of claim 25, wherein the array of coherent light sources comprises an optical frequency comb.

27. The optical computing system of claim 26, wherein the array of coherent light sources comprises a vertical cavity surface-emitting laser (VCSEL).

28. The optical computing system of claim 24, wherein the optical source to amplitude, phase, or frequency modulate the plurality of optical frequency lines.

29. The optical computing system of claim 24, wherein the optical source includes electro-optic modulators and the plurality of optical frequency lines are modulated by the electro-optic modulators.

30. The optical computing system of claim 24, wherein each optical frequency line generated by the optical source has a linewidth narrower than 100 MHz.31 . The optical computing system of claim 24, wherein the optical frequency lines are evenly spaced apart at greater than 1 GHz.

32. The optical computing system of claim 24, wherein the first optical demultiplexer, the second optical demultiplexer, and the optical multiplexer comprise at least one of one or more gratings, prism-grating-prism (PGP), or virtually imaged phase array (VIPA).

33. The optical computing system of claim 24, wherein the first programmable optical memory device to amplitude, phase, or frequency modulate the spatially separated optical frequency lines.

34. The optical computing system of claim 24, wherein the second programmable optical memory devices to amplitude, phase, or frequency modulate the fanned out combined optical frequency lines.

35. The optical computing system of claim 24, wherein the first and second programmable optical memory devices each comprise a spatial light modulator (SLM).

36. The optical computing system of claim 24, wherein the first and second programmable optical memory devices each comprise a two-dimensional array of pixel elements.

37. The optical computing system of claim 24, wherein the first and second programmable optical memory devices each comprise pixel elements having pixel-to-pixel size smaller than 100 microns.

38. The optical computing system of claim 24, wherein the first and second programmable optical memory devices each has operational wavelength range wider than 10 nanometers.

39. The optical computing system of claim 24, wherein the first and second optical elements are cylindrical lens.

40. The optical computing system of claim 24, wherein the array of photodetector elements of the photodetector device are characterized by pixel-to-pixel pitch size of less than 100 microns.

41. The optical computing system of claim 24, wherein the optical computing system is a closed loop system in which the array of photodetector elements of the photodetector device is electronically coupled to the pixel elements of the first programmable optical memory device such that an output of the photodetector elements of the photodetector device is fed back to the pixel elements of the first programmable optical memory device.

42. The optical computing system of claim 41, wherein the photodetector elements of the photodetector device to at least one of photodetection, nonlinear operation, or optical modulation.

43. The optical computing system of claim 41, wherein the photodetector elements of the photodetector device are to photo detect and to optically modulate, and to interface with optical signals, and is on same side or different side of the photodetector device.

44. The optical computing system of claim 41, wherein the pixel elements of the first programmable optical memory device are optical modulator pixels of the first programmable optical memory device, wherein each optical modulator pixel is disposed on a first side of the photodetector device, and wherein each photodetector element is disposed on a second side of the photodetector device directly opposite from a respective optical modulator pixel on the first side.

45. The optical computing system of claim 44, wherein the photodetector device includes an electrical nonlinearity and gain layer disposed between the optical modulator pixels and the array of photodetector elements.

Images