Metasurface-based optical vector-matrix multiplier systems and methods

Abstract

An optical vector-matrix multiplication system includes a transmitter subsystem, such as a silicon photonic transmitter subsystem or a VCSEL array, that encodes a digital input vector into a vector-encoded optical signal. One portion of a free-space optical subsystem fans out the vector-encoded optical signal along a first axis and directs it to a tunable optical metasurface. The tunable optical metasurface encodes a digital matrix as a two-dimensional matrix of optical modulation parameters and modulates the fanned-out vector-encoded optical signal to form a modulated optical signal. Another portion of the free-space optical subsystem condenses the modulated optical signal along the first axis to perform a summation and directs the optical radiation to a silicon photonic receiver subsystem, which detects and decodes the condensed optical signal.

Description

Metasurface-Based Optical Vector-Matrix Multiplier Systems and MethodsRELATED APPLICATIONS

[0001] This application claims priority and benefit to U.S. Provisional Patent Application No. 63 / 620,386, filed on January 12, 2024, titled “Metasurface-Based Optical Vector-Matrix Multiplier for Multi-Chip Modules,” which application is hereby incorporated by reference in its entirety.TECHNICAL FIELD

[0002] This application relates to metamaterial elements, optical computing architectures, and vector-matrix multiplication.BRIEF DESCRIPTION OF THE DRAWINGS

[0003] FIG. 1 illustrates a block diagram of optical vector-matrix multiplication, according to one embodiment.

[0004] FIG. 2 illustrates a block diagram of an analog processor device, according to one embodiment.

[0005] FIG. 3A illustrates a block diagram representation of an optical vector-matrix multiplication (OVMM) device, according to one embodiment.

[0006] FIG. 3B illustrates a block diagram representation of the stacked layers of the OVMM device of FIG. 3A, according to one embodiment.

[0007] FIG. 4 illustrates a flow chart of an example method for implementing OVMM using a reconfigurable metasurface, according to one embodiment.DETAILED DESCRIPTION

[0008] Traditional approaches for vector-matrix multiplication rely on electronic processors to handle the computational load, which can be limited by the speed and power consumption of electronic components. In the field of optical computing, various approaches have been developed to perform vector-matrix multiplication using optical systems. Early optical computing systems utilized bulky, linearly arranged optical1NEU0810components such as lenses, beam splitters, and spatial light modulators to perform matrix operations. These systems, while effective in demonstrating the principles of optical computation, were often large, complex, and difficult to integrate with existing electronic systems.

[0009] The systems and methods described herein for optical vector-matrix multiplication leverage tightly integrated silicon photonics, tunable metasurfaces, and free-space optics to achieve efficient, high-speed, and energy-efficient computation of vector-matrix multiplications. The presently described systems and methods are particularly suitable for applications in artificial intelligence (Al) inference, neural network operations, and other computational tasks requiring large-scale linear algebra operations.

[0010] According to various embodiments described herein, an optical vector-matrix multiplication system includes one or more of: a silicon photonic transmitter subsystem, a two-dimensionally tunable optical metasurface, a free-space optical subsystem, a silicon photonic receiver subsystem, a vector processing unit (VPU), an integrated control circuit, a monolithic SRAM module, and a high-bandwidth memory (HBM). In some embodiments, the integrated control circuit (e.g., an application-specific integrated circuit (ASIC) or electronic integrated circuit (EIC)) connects and controls communication with the various subsystems, VPU, SRAM module, and HBM.

[0011] The silicon photonic transmitter subsystem is configured to encode a digital input vector into a transmitted optical signal via phase modulation and / or amplitude modulation. The silicon photonic transmitter subsystem may utilize, for example, a Mach- Zehnder interferometer, an electro-optic modulator, and / or a ring resonator. The optical signal represents the vector data in a modulated optical form suitable for subsequent processing.

[0012] In various embodiments, the optical vector-matrix multiplication system can utilize incoherent optical sources, such as one or more arrays of vertical-cavity surfaceemitting lasers (VCSEL arrays), as an alternative approach to generate the optical signals. In such embodiments, the silicon photonic transmitter subsystem may be replaced (or supplemented) with one or more VCSEL arrays to serve as the light source for encoding the digital input vector as the vector-encoded optical signal. In some2NEU0810embodiments, each element or subset of elements of the VCEL array may correspond to an individual element of the vector. Thus, in some embodiments, the transmitter subsystem may operate to modulate light emitted from the VCSELs in phase and / or amplitude either at the source or through subsequent optical components, such as modulators or a tunable metasurface. Incoherent light from VCSEL sources allows for additional flexibility in designing the optical paths and reduces coherence-related artifacts in free-space propagation and metasurface interactions.

[0013] The two-dimensionally tunable optical metasurface encodes a digital matrix as a two-dimensional array of optical modulation parameters (e.g., “weights” to be applied to the input vector in the form of phase, amplitude, and / or polarization changes). The metasurface modulates a fanned-out optical signal with the encoded matrix to perform an unsummed optical vector-matrix multiplication. In various embodiments, the metasurface is a dynamically tunable metasurface (e.g., using a tunable phase-change material such as liquid crystal between optical resonators). The dynamically tunable metasurface allows for the independent modulation of the phase and / or amplitude of the optical signal at each pixel. In some embodiments, the metasurface may be dynamically reconfigured to perform multiple sequential vector-matrix multiplications with varying optical modulation parameters used for each successive vector-matrix multiplication.

[0014] The free-space optical subsystem performs several functions for directing and shaping the optical radiation between the silicon photonic transmitter subsystem, the two- dimensionally tunable optical metasurface, and the silicon photonic receiver subsystem. For example, the free-space optical subsystem fans out the vector-encoded optical signal along a first axis, directs the fanned-out signal to the metasurface, and condenses the modulated optical signal along the same, first axis. The condensing operation completes the vector-matrix multiplication by summing the contributions along the axis to form a condensed optical signal that represents the resultant vector.

[0015] The free-space optical subsystem may include various lenses, mirrors, static or reconfigurable metalenses, beam splitters, diffraction gratings, and / or anamorphic optical elements to achieve the fan-out and condensation operations. As described herein, free-space optical subsystem may be arranged and configured in a compact3NEU0810configuration that allows for the parallel planar placement of the silicon photonic transmitter subsystem, the two-dimensionally tunable optical metasurface, and the silicon photonic receiver subsystem on the same chip or substrate.

[0016] The silicon photonic receiver subsystem detects and decodes the condensed optical signal into electronic signals representing the final result of the vector-matrix multiplication. The silicon photonic receiver subsystem may, for example, include an array of photodetectors capable of measuring signal properties such as intensity, phase, and / or polarization. The silicon photonic receiver subsystem may also include or be connected to an analog-to-digital converter to convert the detected analog signals from the photodetectors into digital signals. The decoded electronic signals (analog or digital) can then be processed further and / or converted to various digital formats for processing, transmission, and / or storage.

[0017] In various embodiments, an architecture is used that provides for compact packaging, high-speed operations, and energy-efficient operation. For example, a three- dimensional packaging with through-silicon vias (TSVs) may be used to connect an integrated control circuit with the silicon photonic transmitter subsystem, the tunable metasurface, and the receiver subsystem. In various embodiments, the integrated control circuit may be implemented as an application-specific integrated circuit (ASIC) or an electronic integrated circuit (EIC). The integrated control circuit is configured to manage data flow and dynamically reconfigure the metasurface during operation.

[0018] In some embodiments, the integrated control circuit may include an integrated vector processing unit (VPU) to perform post-detection operations such as calibration corrections, vector transformations, normalization, and activation functions (e.g., a Rectified Linear Unit (ReLU) activation function). The VPU allows for highly efficient and high-speed post-processing operations that are specifically useful for artificial intelligence (Al) and machine learning tasks.

[0019] In various embodiments, the integrated control circuit includes an integrated monolithic SRAM for high-speed, low-latency storage of input vectors and matrix data. The integrated control circuit may also be connected to a high-bandwidth memory (HBM) via an interposer or other high-speed communication channel. The HBM may be used to4NEU0810store, for example, large two-dimensional weight arrays to be applied as two-dimensional matrices of modulation parameters on the metasurface. The SRAM and HBM memory modules interface seamlessly with the silicon photonic components and the metasurface via high-bandwidth connections to optimize data access and energy efficiency. The integrated control circuit allows for the use of physically short and low-capacitance interconnections to increase the speed and energy efficiency of the system.

[0020] In some embodiments, the silicon photonic transmitter subsystem, two- dimensionally tunable optical metasurface, the free-space optical subsystem, and the silicon photonic receiver subsystem are capable of transmitting, modulating, and receiving multiple optical wavelengths simultaneously. In such embodiments, the optical vector-matrix multiplication system is capable of parallel computations for different input vectors with a given matrix. In some embodiments, the parallel computations are accomplished through the use of multiple silicon photonic transmitter subsystems and multiple silicon photonic receiver subsystems that are used in conjunction with a common tunable optical metasurface and free-space optical subsystem.

[0021] For example, in various embodiments, the silicon photonic transmitter subsystem operates to encode multiple optical wavelengths within the same system. The two-dimensionally tunable optical metasurface is configured to process these multiple wavelengths concurrently. The free-space optical subsystem further facilitates this parallelism by maintaining precise alignment and fan-out for each wavelength, ensuring that the vector-encoded optical signals are directed to the appropriate regions of the metasurface and appropriate regions of the silicon photonic receiver subsystem. In such embodiments, the silicon photonic receiver subsystem may include an array of wavelength-sensitive photodetectors. The array or arrays of photodetectors decode the condensed optical signals for each wavelength independently, allowing parallel outputs for distinct computations.

[0022] Various methods to perform vector-matrix multiplication are possible using the various embodiments of vector-matrix multiplication systems described herein. For example, a method to perform vector-matrix multiplication may include encoding a digital input vector, encoding a digital matrix on the metasurface, performing optical modulation5NEU0810and summation, and detecting and decoding the condensed optical signal. In some embodiments, the metasurface may be dynamically reconfigured to allow for multiple sequential computations with different input vectors and / or different matrix values.

[0023] The variously described optical vector-matrix multiplication systems achieve high computational throughput while significantly reducing energy consumption compared to digital systems.

[0024] Many existing computing systems, methods, and devices may be used in combination with the presently described systems and methods. Some of the infrastructure that can be used with embodiments disclosed herein is already available, such as general-purpose computers, computer programming tools and techniques, digital storage media, and communication links. A computing device or controller may include a processor, such as a microprocessor, a microcontroller, logic circuitry, or the like. Various technologies, systems, architectures, and applications are relevant to the presently described embodiments. Examples of such technologies, systems, architectures, and applications include, but are not limited to, certain aspects of deep neural networks, image recognition, recommender systems, medical diagnosis, language processing, and the like.

[0025] A system, controller, or processor of a system may include a special-purpose processing device, such as application-specific integrated circuits (ASIC), programmable array logic (PAL), programmable logic array (PLA), programmable logic device (PLD), field programmable gate array (FPGA), or other customizable and / or programmable device. The computing device may also include a machine-readable storage device, such as non-volatile memory, static RAM, dynamic RAM, ROM, CD-ROM, disk, tape, magnetic, optical, flash memory, or other machine-readable storage medium. Various aspects of certain embodiments may be implemented using hardware, software, firmware, or a combination thereof.

[0026] The components of the disclosed embodiments, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Furthermore, the features, structures, and operations associated with one embodiment may be applicable to or combined with the features, structures, or6NEU0810operations described in conjunction with another embodiment. In many instances, well- known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of this disclosure. For example, many of the optical elements may be substituted by alternative optical elements and / or combined with additional optical elements, such as, but not limited to, mirrors, prisms, lenses, filters, wave plates, diffraction ratings, beam splitters, optical fibers, waveguides, metalenses, and the like.

[0027] Many of the embodiments are described in the context of a metasurface. It is appreciated that, in some cases, alternative two-dimensionally modulated arrays, such as spatial light modulators (SLMs) and other suitable devices, may be used instead of or in addition to metasurfaces. Moreover, various architectures can be adapted to interchangeably utilize reflective or transmissive (e.g., refractive) metasurfaces, SLMs, or other two-dimensional modulated optical arrays.

[0028] FIG. 1 illustrates a block diagram 100 of the operational flow and some components of an optical vector-matrix multiplication system, according to one embodiment. As described herein, the operations may be implemented via a combination of silicon photonic components, a two-dimensionally tunable optical metasurface, and a free-space optical subsystem. In the illustrated example, the system is computing the mathematical equivalent of Ax = b, where A is a matrix 145 of, for example, weightings to be applied via multiplication to the input vector, x, 115. The resultant vector, b, 165 is the product of the vector-multiplication.

[0029] The input vector 115 is encoded into a vector-encoded optical signal 110 via, for example, a digital-to-analog converter, an amplifier, and one or more optical radiation transmitters of a silicon photonic transmitter subsystem. The vector-encoded optical signal 110 comprises phase-modulated and / or amplitude-modulated optical radiation corresponding to the input vector 115.

[0030] The vector-encoded optical signal 110 is expanded into a fanned-out vector- encoded optical signal 130 via an optical fan-out operation 120 performed by the free- space optical subsystem. The fan-out operation 120 spreads the vector-encoded optical signal 110 along a first axis (e.g., in one dimension). The fanned-out vector-encoded optical signal 130 represents the data of the original input vector 115 distributed across7NEU0810the dimensions of the metasurface. The optical modulation parameters 140 are, for example, phase and / or amplitude modulation parameters to be applied to incident optical radiation. The optical modulation parameters 140 programmed into the metasurface correspond to the values (e.g., weightings) of the matrix 145. The fan-out operation 120 ensures that each element of the vector-encoded optical signal 110 aligns correctly with the corresponding optical modulation parameters 140 of the matrix applied to the tunable optical metasurface.

[0031] The fanned-out vector-encoded optical signal 130 interacts with the two- dimensional matrix of optical modulation parameters 140 on the tunable optical metasurface. Again, the optical modulation parameters 140 correspond directly to the elements of the matrix 145. As the fanned-out vector-encoded optical signal 130 passes through and / or is reflected by the metasurface, the optical radiation is modulated pixel- by-pixel to produce a modulated optical signal that represents the unsummed results of the vector-matrix multiplication 175. Effectively, the metasurface performs the element- wise multiplication of the vector elements with the matrix parameters.

[0032] The free-space optical subsystem performs a condensing operation 150 on the modulated optical signal. The condensing operation 150 condenses the modulated optical signal along the first axis, effectively summing the contributions of each element to form a condensed optical signal 160. The condensed optical signal 160 corresponds to the final resultant vector, b, 165 of the vector-multiplication, Ax = b. Although not illustrated in FIG. 1 , the silicon photonic receiver subsystem detects and decodes the condensed optical signal 160 into electronic signals for further processing or use.

[0033] FIG. 2 illustrates a block diagram 200 of an analog processor device 215, according to one embodiment. The analog processor device 215 is an example of a compute-in-memory processor system designed to perform vector-matrix multiplications. As illustrated, a weight memory 220 and an activation memory 230 are connected to the passive analog device 210 of the analog processor device 215 via digital-to-analog converters (DAOs). Additionally, the analog processor device 215 includes a series of analog-to-digital converters (ADCs) to provide a digital output of the vector-matrix multiplication. The weight memory 220 stores the matrix data, which corresponds to the8NEU0810coefficients used in the vector-matrix multiplication. The weights from the weight memory 220 are fed into the passive analog device 210 via the multiple digital-to-analog converters (DACs).

[0034] The activation memory 230 provides the input vector data. Similar to the weight memory 220, the activation data from the activation memory 230 is also converted into analog signals via DACs before being fed into the passive analog device 210. The passive analog device 210 performs an analog computation of the element-wise operations required for vector-matrix multiplication. The results of these operations are then converted back into digital form using analog-to-digital converters (ADCs). The energy cost associated with the transfer of data between the weight memory 220, activation memory 230, and the analog processor device 215 is directly related to the capacitance of the connections between the memory banks and the processor. Given the significant distances, energy consumption is relatively high and the system has a relatively low overall efficiency. The limitations of the analog processor device 215 demonstrate the potential benefits of the various embodiments of optical vector-matrix multiplication systems described herein.

[0035] FIG. 3A illustrates a block diagram 300 representation of an optical vectormatrix multiplication (OVMM) system 305, according to one embodiment. As illustrated, the OVMM system 305 includes a tight integration of various subsystems and components that enable high-speed, energy-efficient computational operations. The OVMM system 305 includes a silicon photonic transmitter subsystem 310 that operates to encode a digital input vector into a transmitted optical signal. The encoding is achieved using phase modulation and / or amplitude modulation through integrated photonic devices such as Mach-Zehnder interferometers, electro-optic modulators, and / or ring resonators. The silicon photonic transmitter subsystem 310 converts the digital input vector into an optical format (e.g., as a vector-encoded optical signal) to be received by the two- dimensionally tunable optical metasurface 320 via the free-space optical subsystem 330.

[0036] The free-space optical subsystem 330 receives the vector-encoded optical signal and performs a fan-out operation. In various embodiments, the free-space optical subsystem 330 operates to fan out the vector-encoded optical signal along a first axis,9NEU0810distributing the vector-encoded optical signal across the dimensions of the metasurface 320. The free-space optical subsystem 330 incorporates lenses, mirrors, and / or anamorphic optics to ensure precise alignment and transformation of the vector-encoded optical signal into a fanned-out vector-encoded optical signal.

[0037] The fanned-out vector-encoded optical signal is directed for incidence on the metasurface 320. The metasurface 320 is dynamically tuned to encode a digital matrix as a two-dimensional array of optical modulation parameters. Each pixel on the metasurface 320 independently modulates the phase and / or amplitude of the incident optical radiation, enabling precise element-wise operations for the vector-matrix multiplication. In various embodiments, the metasurface 320 utilizes phase-change materials, such as liquid crystals, that are dynamically adjustable to allow for rapid adjustment of the modulation parameters during operation. The use of a dynamically reconfigurable metasurface allows for sequential computations of multiple vector-matrix multiplications with varying input vectors or weighting matrices, without requiring physical hardware changes.

[0038] The metasurface 320 modulates the fanned-out vector-encoded optical signal to form a modulated optical signal. The modulated optical signal represents an unsummed optical vector-matrix multiplication of the input vector and the matrix. The free- space optical subsystem 330 operates to condense the modulated optical signal along the same axis that was used to fan out the vector-encoded optical signal. The condensing operation performed by the free-space optical subsystem 330 implements a summation, which completes the vector-matrix multiplication. As previously noted, the free-space optical subsystem 330 may incorporate various lenses, mirrors, and anamorphic optics to implement the condensing function and direct the condensed optical signal to be received by the silicon photonic receiver subsystem 340.

[0039] The silicon photonic receiver subsystem 340 detects and decodes the condensed optical signal into electronic signals. The silicon photonic receiver subsystem 340 may include, for example, an array of photodetectors capable of measuring intensity, phase, and / or polarization. The silicon photonic receiver subsystem 340 operates to detect and convert the condensed optical signal into digital form. The output digital signal, corresponding to the resultant vector of the vector-matrix multiplication, is ready for further10NEU0810processing by other digital subsystems, transmission to a host via the host interface 390, and / or digital storage.

[0040] As described herein, an integrated control circuit 350 manages data flow between the photonic subsystems and dynamically reconfigures the metasurface during operation. The integrated control circuit 350 may comprise, for example, an applicationspecific integrated circuit (ASIC) or an electronic integrated circuit (EIC). The integrated control circuit 350 may include an integrated vector processing unit (VPU) 360. The VPU 360 may perform useful post-detection operations such as accumulation, Rectified Linear Unit (ReLU) activation, and / or normalization. The post-detection operations performed by the VPU 360 are particularly useful for applications in artificial intelligence and neural network inference.

[0041] In various embodiments, the integrated control circuit 350 includes tightly integrated memory modules to support high-speed, low-latency data storage and retrieval. For example, a monolithic SRAM 370 is co-located on the integrated control circuit 350. The monolithic SRAM 370 may, for example, store weights and activation data, while also facilitating input operations and / or output operations. In addition, a high- bandwidth memory (HBM) module 380 may be connected to the integrated control circuit 350 via an interposer. The HBM module 380 provides storage for large matrices and other data required for computation. The combination of integrated SRAM and high-speed connected HBM reduces latency and energy consumption during memory access operations.

[0042] In various embodiments, the HBM module 380, the SRAM 370, and the VPU 360 are accessible via a host interface 390 (e.g., via the integrated control circuit 350). The host interface 390 connects the OVMM system 305 to external computational frameworks, enabling the upload of input vectors and matrices and the retrieval of computed results. The host interface 390 supports seamless integration with larger systems, allowing the OVMM system 305 to function as a modular and versatile computational unit.

[0043] The illustrated architecture of the OVMM system 305 provides a balance of photonic, optical, and digital components that enable high-throughput and energy-11NEU0810efficient computations. The free-space optical subsystem 330 allows for coplanar colocation of the silicon photonic transmitter subsystem 310, metasurface 320, and silicon photonic receiver subsystem 340 proximate to the integrated control circuit 350.

[0044] FIG. 3B illustrates a block diagram 301 representation of the stacked layers of the OVMM system 305 of FIG. 3A, according to one embodiment. The illustrated integration of key components, including photonic, optical, and digital elements, within a compact packaging structure enables high-performance and energy-efficient computation. As illustrated, the system may include a printed circuit board (PCB) 395 that provides foundational physical and electrical support. A silicon interposer 385 is mounted on the PCB 395. The silicon interposer 385 provides a communication interface between various photonic and electronic components of the system. The silicon interposer 385 may, for example, include through-silicon vias (TSVs) to enable high-bandwidth connections between the various layers of the system.

[0045] An integrated control circuit 350 is positioned on the silicon interposer 385. The integrated control circuit 350 may, as previously described, comprise an ASIC or EIC with integrated SRAM. The integrated control circuit 350 serves as the central control hub for the system, managing data flow between the silicon photonic transmitter subsystem 310, the metasurface 320, and the silicon photonic receiver subsystem 340. The integrated control circuit 350 interfaces with the high-bandwidth memory (HBM) module 380 via the silicon interposer 385. As previously described, the HBM module 380 may be used to store large two-dimensional matrices and / or other data used for vector-matrix multiplication operations. The integration of the HBM module 380 and integrated control circuit 350 on the silicon interposer 385 minimizes physical distances, which reduces latency and energy consumption during memory access.

[0046] As illustrated and described herein, the layered architecture of the OVMM system is used to achieve efficient integration of photonic, optical, and electronic components. The layered architecture includes co-located and / or co-planar positioning of the integrated control circuit 350, the silicon photonic transmitter subsystem 310, the metasurface 320, and the silicon photonic receiver subsystem 340. The illustrated and described architecture reduces latency and energy costs, which is especially useful in12NEU0810high-performance computing applications. The combination of advanced packaging techniques, such as TSVs and silicon interposers, with high-performance components ensures that the system is capable of executing complex computations efficiently. The architecture is well-suited for applications in artificial intelligence, neural network inference, and other tasks requiring large-scale vector-matrix operations and / or other linear algebra operations.

[0047] FIG. 4 illustrates a flow chart 400 of an example method for implementing OVMM using a reconfigurable metasurface, according to one embodiment. As illustrated, the method may include encoding, at 410, a digital input vector into a transmitted, vector- encoded optical signal using at least one of phase modulation and amplitude modulation via a silicon photonic transmitter subsystem.

[0048] The method continues, at 420, by encoding a digital matrix as a two- dimensional matrix of optical modulation parameters on a two-dimensionally tunable optical metasurface. The metasurface is configured to modulate incoming optical signals based on the encoded modulation parameters. Each pixel or set of pixels of the metasurface operates to modulate (e.g., independently modulate) the phase and / or amplitude of the optical signal, thereby encoding the matrix data into the optical domain.

[0049] At 430, the method continues by fanning out the vector-encoded optical signal along a first axis to form a fanned-out vector-encoded optical signal via a free-space optical subsystem. The fan-out operation distributes the input vector components across the spatial dimensions of the metasurface. The free-space optical subsystem may include optical elements such as lenses, mirrors, or anamorphic optical components to achieve precise fan-out and alignment of the optical signal. The fan-out ensures that each element of the vector is directed to the corresponding row or column of the metasurface.

[0050] At 440, the fanned-out vector-encoded optical signal is directed to the tunable optical metasurface by the free-space optical subsystem. At 450, the fanned-out vector- encoded optical signal is modulated with the two-dimensional matrix of optical modulation parameters on the tunable optical metasurface. This process produces a modulated optical signal representing an unsummed optical vector-matrix multiplication. The modulated optical signal encodes the element-wise product of the input vector and the13NEU0810matrix, but the summation step necessary to complete the vector-matrix multiplication is not performed by the metasurface.

[0051] At 460, the modulated optical signal is condensed along the first axis to perform a summation, forming a condensed optical signal representing a completed optical vectormatrix multiplication. This summation is achieved via optical condensing performed by the free-space optical subsystem. The resulting condensed optical signal represents the resultant vector of the vector-matrix multiplication in the optical domain.

[0052] At 470, the method concludes by detecting and decoding the condensed optical signal into electronic signals representing the resultant vector of the vector-matrix multiplication using a silicon photonic receiver subsystem. The receiver subsystem employs an array of photodetectors to measure properties of the condensed optical signal, such as intensity, phase, and / or polarization. The optical measurements are converted into electronic signals that can be further processed or used in downstream computational tasks.

[0053] This disclosure has been made with reference to various exemplary embodiments, including the best mode. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of the present disclosure. While the principles of this disclosure have been shown in various embodiments, many modifications of structure, arrangements, proportions, elements, materials, and components may be adapted for a specific environment and / or operating requirements without departing from the principles and scope of this disclosure. These and other changes or modifications are intended to be included within the scope of the present disclosure.

[0054] This disclosure is to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope thereof. Likewise, benefits, other advantages, and solutions to problems have been described above with regard to various embodiments. However, benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or element.14NEU0810

Claims

CLAIMSWhat is claimed is:

1. An optical vector-matrix multiplication system, comprising: a transmitter subsystem to encode a digital input vector into a transmitted, vector-encoded optical signal using at least one of phase modulation and amplitude modulation; a two-dimensionally tunable optical metasurface to: encode a digital matrix as a two-dimensional matrix of optical modulation parameters, and modulate a fanned-out vector-encoded optical signal with the two- dimensional matrix of optical modulation parameters to form a modulated optical signal representing an unsummed optical vector-matrix multiplication of the input vector and the matrix; a free-space optical subsystem to: fan-out the vector-encoded optical signal along a first axis to form the fanned-out vector-encoded optical signal, direct the fanned-out vector-encoded optical signal to the tunable optical metasurface, and condense the modulated optical signal along the first axis to perform a summation to form a condensed optical signal representing a completed optical vector-matrix multiplication; and a silicon photonic receiver subsystem to detect and decode the condensed optical signal from the free-space optical subsystem into electronic signals representing a resultant vector of the vector-matrix multiplication.

2. The system of claim 1 , wherein the transmitter subsystem comprises a silicon photonic transmitter subsystem.15NEU08103. The system of claim 2, wherein the silicon photonic transmitter subsystem includes one or more modulators selected from a group of modulators consisting of: a Mach-Zehnder interferometer, an electro-optic modulator, and a ring resonator.

4. The system of claim 1 , wherein the transmitter subsystem comprises one or more arrays of vertical-cavity surface-emitting lasers (VCSEL arrays).

5. The system of claim 1 , wherein the two-dimensionally tunable optical metasurface is configured to independently modulate at least one of a phase and an amplitude of the fanned-out vector-encoded optical signal at each pixel.

6. The system of claim 1 , wherein the system is integrated using a three- dimensional packaging architecture with through-silicon vias (TSVs) connecting an integrated control circuit to the transmitter subsystem, the tunable optical metasurface, and the silicon photonic receiver subsystem.

7. The system of claim 1 , wherein the two-dimensionally tunable optical metasurface comprises a dynamically reconfigurable metasurface implemented using a phase-change material to allow for dynamic reconfiguration of the optical modulation parameters.

8. The system of claim 1 , further comprising: a vector processing unit (VPU) to perform post-detection operations on the electronic signals representing the resultant vector of the vector-matrix multiplication.

9. The system of claim 7, wherein the system is configured to perform multiple vector-matrix multiplications sequentially with different optical modulation parameters being used for each successive vector-matrix multiplication.

10. The system of any of claims 1 -7, wherein the free-space optical subsystem includes one multiple optical elements selected from a group of optical elements consisting of: a lens, a mirror, a beam splitter, and a diffraction grating.16NEU081011 . The system of any of claims 1 -7, wherein the free-space optical subsystem comprises anamorphic optical elements.

12. The system of any of claims 1 -7, wherein the silicon photonic receiver subsystem includes an array of photodetectors to detect the condensed optical signal.

13. The system of claim 12, wherein the photodetectors are configured to detect at least one of: intensity, phase, and polarization of the condensed optical signal.

14. The system of any of claims 1 -7, wherein the metasurface is configured to support multiple optical wavelengths simultaneously, enabling parallel vector-matrix multiplications for different input vectors.

15. The system of any of claims 1 -7, wherein the transmitter subsystem encodes the input vector using a digital-to-analog converter.

16. The system of any of claims 1 -7, wherein the system is configured to perform multiple vector-matrix multiplications sequentially by reusing the optical modulation parameters of the metasurface.

17. The system of claim 1 , wherein the system is configured for use in artificial intelligence inference.

18. The system of claim 8, wherein the VPU is configured to perform at least one post-detection operation from a group of post-detection operations consisting of: a calibration correction operation, a vector operation, a normalization operation, and a rectified linear unit (ReLLI) activation function operation.

19. The system of claim 8, wherein the integrated control circuit comprises one of an application-specific integrated circuit (ASIC) and an electronic integrated circuit (EIC), wherein the integrated control circuit is configured to control data flow between17NEU0810the transmitter subsystem, the tunable optical metasurface, and the silicon photonic receiver subsystem.

20. The system of claim 19, further comprising: a vector processing unit (VPU) to perform post-detection operations on the electronic signals representing the resultant vector of the vector-matrix multiplication, wherein the integrated control circuit is further connected to and configured to control data flow to and from the VPU.21 . The system of claim 8, wherein the integrated control circuit includes control logic to dynamically reconfigure the optical modulation parameters of the tunable optical metasurface during operation according to received data for different digital matrices.

22. The system of claim 8, further comprising a monolithic SRAM module integrated with the integrated control circuit, wherein the monolithic SRAM module is configured to provide high-speed, low-latency storage of input vectors and matrix data for the optical vector-matrix multiplication.

23. The system of claim 22, wherein the SRAM module is configured to provide data directly to the transmitter subsystem and the tunable optical metasurface through high-bandwidth memory interfaces.

24. The system of claim 22, further comprising a high-bandwidth memory (HBM) connected to the integrated control circuit via an interposer, wherein the HBM is configured to store two-dimensional matrix arrays used by the tunable optical metasurface.18NEU081025. A method for performing vector-matrix multiplication using an optical system, comprising: encoding a digital input vector into a transmitted, vector-encoded optical signal using at least one of phase modulation and amplitude modulation via a transmitter subsystem; encoding a digital matrix as a two-dimensional matrix of optical modulation parameters on a two-dimensionally tunable optical metasurface; fanning out the vector-encoded optical signal along a first axis to form a fanned- out vector-encoded optical signal via a free-space optical subsystem; directing the fanned-out vector-encoded optical signal to the tunable optical metasurface; modulating the fanned-out vector-encoded optical signal with the two- dimensional matrix of optical modulation parameters to form a modulated optical signal representing an unsummed optical vector-matrix multiplication; condensing the modulated optical signal along the first axis to perform a summation and form a condensed optical signal representing a completed optical vector-matrix multiplication; and detecting and decoding the condensed optical signal into electronic signals representing a resultant vector of the vector-matrix multiplication using a silicon photonic receiver subsystem.

26. The method of claim 25, wherein modulating the fanned-out vector-encoded optical signal comprises independently modulating at least one of a phase and an amplitude of the fanned-out vector-encoded optical signal at each pixel of the tunable optical metasurface.

27. The method of claim 25, further comprising: dynamically reconfiguring the optical modulation parameters of the tunable optical metasurface by tuning a phase-change material.19NEU081028. The method of claim 27, further comprising: performing multiple vector-matrix multiplications sequentially by dynamically reconfiguring the optical modulation parameters of the metasurface for each of a plurality of successive vector-matrix multiplications.

29. The method of claim 25, further comprising: performing post-detection operations on the electronic signals representing the resultant vector using a vector processing unit (VPU).20NEU0810

Images