Multi-mode photon convolution accelerator based on lossless mode fan-in

Abstract

The invention discloses a multimode photon convolution accelerator based on lossless mode fan-in, and belongs to the field of integrated optical calculation. The accelerator utilizes multi-order orthogonal modes (TE0, TE1... TEn) in a multi-mode waveguide to realize multi-mode lossless fan-in of signals. By cascading the multi-mode 3-dB coupler and the multi-mode weight modulator array, the device executes convolution operation in parallel in two dimensions of mode and wavelength. According to the framework, M * N times of multiplication and addition operation can be completed only through M + N electro-optical modulators, and the theoretical calculation force density reaches 125.14 TOPS / mm < 2 >. Experimental results show that in the C wave band, the accelerator can achieve the optical domain calculation precision of 6-7 bits, and the recognition accuracy rates on an MNIST data set and a Fast-MNIST data set are 95.2% and 87.9% respectively. By eliminating 3-dB loss of a traditional single-mode system, the integration level and the hardware utilization rate are improved, and a foundation is laid for constructing a multi-layer cascaded all-optical neural network.

Description

Technical Field

[0001] The present invention belongs to the field of integrated optical computing, and more specifically, relates to a multi-mode photon convolution accelerator based on lossless mode fan-in. Background Art

[0002] Convolutional neural networks (CNNs) play a central role in tasks such as image recognition and natural language processing. However, traditional electronic devices face problems such as high power consumption, limited speed, and insufficient interconnection bandwidth when performing convolution operations. Optical computing, with its advantages of high-dimensional parallelism and ultra-high-speed transmission of photons, provides new possibilities for breaking through these bottlenecks. Current optical convolution schemes inevitably produce a 3-dB loss when fanning in single-mode signals, resulting in low energy efficiency. Convolution calculations require frequent signal merging, and multimode waveguides support TE0, TE1…TE n Lossless transmission of multiple orthogonal modes provides ideal conditions for efficient signal fan-in. Mode multiplexing allows for multi-channel signal merging without introducing additional loss, making mode multiplexing a preferred solution for efficient optical convolution. However, achieving large-scale, multi-wavelength, and multi-mode parallel convolution without loss remains a pressing technical challenge. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a multi-mode photon convolution accelerator based on lossless mode fan-in, aiming to solve the problems of large energy loss, low integration and insufficient hardware utilization of traditional single-mode systems in existing optical convolution technology.

[0004] To achieve the above objectives, the present invention provides a multimode photon convolution accelerator based on lossless mode fan-in, including an intensity modulation module, a cascaded multimode 3-dB coupler, a multimode weight modulator array, and a multimode waveguide router. The accelerator utilizes the orthogonal mode characteristics of the multimode waveguide to achieve lossless signal merging and performs convolution operations in parallel in both the mode and wavelength dimensions, significantly improving computational throughput and hardware utilization.

[0005] The intensity modulation module is used to perform intensity modulation on the input M mode signals to adapt to different convolution kernel calculation requirements. The M mode signals after intensity modulation are equally divided into N parts by the cascaded multimode 3-dB coupler, which are the first to Nth multimode signals, respectively. Each multimode signal includes the first to Mth modes;

[0006] The multimode weight modulator array includes N parallel multimode electro-optical modulators. By changing the electrical drive signal of each multimode electro-optical modulator to map it to a convolution kernel in a convolution calculation, intensity modulation of each mode signal is achieved, thereby accurately encoding different information of the convolution kernel into the optical domain. The electrical drive signal of each multimode electro-optical modulator corresponds one-to-one to the coefficient in the convolution kernel. The first to Nth multimode signals are respectively input into the N parallel multimode electro-optical modulator units, and the products of the first to Nth multimode signals and the corresponding weight coefficients are respectively obtained to perform an efficient convolution operation.

[0007] The multimode waveguide routing unit includes N multimode waveguide units, each of which includes a straight-through waveguide and several coupled waveguides. The several coupled waveguides are coupled with the straight-through waveguide of the corresponding multimode waveguide unit at the head end, and the several coupled waveguides are coupled with the straight-through waveguide of the adjacent multimode waveguide unit at the tail end. The products of the first to N-th multimode signals and the corresponding weight coefficients are sequentially input into the N multimode waveguide units. The products of the straight-through mode multimode signals supported by the straight-through waveguide and the corresponding weight coefficients are output from the corresponding straight-through waveguide output ports. At the same time, the products of the coupled mode multimode signals supported by the several coupled waveguides of the adjacent multimode waveguide units and the corresponding weight coefficients are coupled and output. The coupled waveguides at both ends respectively output the products of the first and N-th multimode signals and the corresponding weight coefficients. The final convolution result is output through M+N-1 output ports, meeting the requirements of large-scale parallel processing.

[0008] Furthermore, the multimode weight modulator and multimode waveguide routing adopt the reverse design method to compress the device size to achieve high performance and large tolerance.

[0009] Furthermore, the intensity modulation module includes M intensity modulators. The accelerator only needs M intensity modulators to encode an input vector of size M and N multi-mode electro-optical modulators to encode a convolution kernel of size N. It can complete the calculation within a single clock cycle. The total number of multiplication and accumulation (MAC) operations is M×N, which is equivalent to performing 2×M×N arithmetic operations.

[0010] Furthermore, the convolution operation uses wavelength multiplexing technology to integrate K wavelength channels into the same straight-through waveguide. Each channel processes 2×M×N operations in parallel, and the total number of multiplication and addition operations reaches 2×K×M×N, significantly improving the computing throughput and accelerating the optical convolution process.

[0011] Furthermore, the multimode electro-optical modulator utilizes intensity modulation technology; both the intensity modulator and the multimode electro-optical modulator are electro-optical modulators. This accelerator requires only M+N electro-optical modulators to encode the input vector and convolution kernel, performing M×N multiplication-addition operations, theoretically maximizing hardware utilization. The input vector serves as rapidly updated data, while the convolution kernel remains relatively stable and refreshes at a relatively low rate. Therefore, a high-speed modulator encodes the input vector, while a low-speed modulator adjusts the convolution kernel. Compared to traditional systems, this design avoids hardware redundancy, reduces power consumption, and improves computational efficiency, making the system more compact and efficient.

[0012] Furthermore, the accelerator, operating in the C-band, achieves optical-domain computational accuracy exceeding 6 bits, achieving classification accuracy comparable to that achieved in the electrical domain on the MNIST and Fashion-MNIST datasets. This maximizes hardware utilization without sacrificing accuracy, further demonstrating its efficiency and reliability in practical applications.

[0013] Furthermore, the device is compatible with standard silicon on insulating substrate (SOI) manufacturing processes, and the waveguide core material includes but is not limited to Si, and the cladding material includes but is not limited to SiO2.

[0014] Compared with the prior art, the above technical solutions proposed by the present invention can achieve the following:

[0015] Beneficial effects:

[0016] 1. The multimode photonic convolution accelerator based on lossless mode fan-in, provided by this invention, leverages the orthogonality of multimode waveguides and parallel computing technology to avoid the inherent 3dB loss bottleneck of traditional single-mode systems. By enabling lossless signal merging and highly parallel computing through multimode fan-in, this invention improves the computational throughput of optical computing systems, reduces hardware redundancy, lowers power consumption, and enhances scalability.

[0017] 2. The present invention realizes dual-dimensional parallel computing of mode and wavelength, thereby significantly improving the computing power of the system, and enhancing the system's energy efficiency and computing throughput;

[0018] 3. The multi-mode photon convolution accelerator of the present invention has higher hardware utilization and can complete M×N multiplication and addition operations while using only M+N electro-optical modulators, significantly reducing hardware redundancy and power consumption.

[0019] 4. The efficient photon convolution accelerator constructed by this invention can support high-precision optical domain computing and demonstrate excellent classification performance on standard datasets, demonstrating the efficiency and reliability of the accelerator in practical applications;

[0020] 5. The technical solution of the present invention has good scalability. It can flexibly cope with convolution operation tasks of different scales by adjusting the number of modes, the number of wavelength channels and the number of modulators, and is suitable for the construction of large-scale optical neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the multi-mode photon convolution accelerator of the present invention;

[0022] Figure 2 The experimental comparison diagram of single-mode and multi-mode fan-in; (a) and (b) are scanning electron microscope images of single-mode and multi-mode fan-in, respectively; (c) is the loss comparison of single-mode and multi-mode fan-in;

[0023] Figure 3 This is an optical microscope image and packaging diagram of the multi-mode photon convolution accelerator;

[0024] Figure 4 (a) is the single-wavelength experimental result of the multi-mode photon convolution accelerator; (b) is the structure of the convolutional neural network; (c) and (d) are the experimental and calculated confusion matrix results respectively;

[0025] Figure 5 These are the multi-wavelength experimental results of the multi-mode photon convolution accelerator; (a) and (b) are the accuracy results of the multi-wavelength; (c) and (d) are the confusion matrix results of the multi-wavelength experiment and calculation, respectively. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0027] The present invention provides a multimode photon convolution accelerator based on lossless mode fan-in, which includes an intensity modulation module, a cascaded multimode 3-dB coupler, a multimode weight modulator array, and a multimode waveguide router. The accelerator utilizes the orthogonal mode characteristics of the multimode waveguide to achieve lossless signal merging and performs convolution operations in parallel in both the mode and wavelength dimensions, significantly improving computational throughput and hardware utilization.

[0028] The intensity modulation module is used to perform intensity modulation on the input M mode signals to adapt to different convolution kernel calculation requirements. The M mode signals after intensity modulation are equally divided into N parts by the cascaded multimode 3-dB coupler, which are the first to Nth multimode signals, respectively. Each multimode signal includes the first to Mth modes;

[0029] The multimode weight modulator array includes N parallel multimode electro-optical modulators. By changing the electrical drive signal of each multimode electro-optical modulator, the signal is mapped into a convolution kernel in the convolution calculation to achieve intensity modulation of each mode signal, thereby accurately encoding different information of the convolution kernel into the optical domain. The electrical drive signal of each multimode electro-optical modulator corresponds one-to-one to the coefficient in the convolution kernel. The first to Nth multimode signals are respectively input to the N parallel multimode electro-optical modulator units. By adjusting the drive signal of each modulator, the product of the first to Nth multimode signals and the corresponding weight coefficient is obtained to perform an efficient convolution operation.

[0030] The multimode waveguide routing unit includes N multimode waveguide units, each of which includes a straight-through waveguide and several coupled waveguides. The several coupled waveguides are coupled with the straight-through waveguide of the corresponding multimode waveguide unit at the head end, and the several coupled waveguides are coupled with the straight-through waveguide of the adjacent multimode waveguide unit at the tail end. The products of the first to N-th multimode signals and the corresponding weight coefficients are sequentially input into the N multimode waveguide units. The products of the straight-through mode multimode signals supported by the straight-through waveguide and the corresponding weight coefficients are output from the corresponding straight-through waveguide output ports. At the same time, the products of the coupled mode multimode signals supported by the several coupled waveguides of the adjacent multimode waveguide units and the corresponding weight coefficients are coupled and output. The coupled waveguides at both ends respectively output the products of the first and N-th multimode signals and the corresponding weight coefficients. The final convolution result is output through M+N-1 output ports, meeting the requirements of large-scale parallel processing.

[0031] Specifically, the multimode weight modulator and multimode waveguide routing adopt the inverse design method to compress the device size to achieve high performance and large tolerance.

[0032] Specifically, the intensity modulation module includes M intensity modulators. The accelerator only needs M intensity modulators to encode an input vector of size M and N multi-mode electro-optical modulators to encode a convolution kernel of size N. It can complete the calculation within a single clock cycle. The total number of multiplication and accumulation (MAC) operations is M×N, which is equivalent to performing 2×M×N arithmetic operations.

[0033] Specifically, the convolution operation uses wavelength multiplexing technology to integrate K wavelength channels into the same straight waveguide. Each channel processes 2×M×N operations in parallel, and the total number of multiplication and addition operations reaches 2×K×M×N, significantly improving the computing throughput and accelerating the optical convolution process.

[0034] Specifically, the multimode electro-optical modulator (EMM) employs intensity modulation technology, with both the intensity modulator and the MEM being electro-optical modulators. This accelerator requires only M+N EOMs to encode the input vector and convolution kernel, performing M×N multiplication-addition operations, theoretically maximizing hardware utilization. The input vector serves as rapidly updated data, while the convolution kernel remains relatively stable and refreshes at a relatively low rate. Therefore, a high-speed modulator encodes the input vector, while a low-speed modulator adjusts the convolution kernel. Compared to traditional systems, this design avoids hardware redundancy, reduces power consumption, and improves computational efficiency, making the system more compact and efficient.

[0035] Specifically, the accelerator, operating in the C-band, achieves optical-domain computational accuracy exceeding 6 bits, achieving classification accuracy comparable to that achieved in the electrical domain on the MNIST and Fashion-MNIST datasets. This maximizes hardware utilization without sacrificing accuracy, further demonstrating its efficiency and reliability in practical applications.

[0036] Specifically, the device is compatible with a standard silicon on insulating substrate (SOI) manufacturing process, and the waveguide core material includes but is not limited to Si, and the cladding material includes but is not limited to SiO2.

[0037] like Figure 1 As shown, the multimode photon convolution accelerator of the present invention achieves lossless signal merging through multimode fan-in, solving the energy loss problem of traditional single-mode systems. The system uses multiple orthogonal waveguide modes to combine signals, and each beam of input light is mapped to a different waveguide mode, reducing interference and reflection between channels and improving equipment utilization. The architecture of the multimode photon accelerator includes a multimode 3-dB coupler, a multimode weight modulator array, and a multimode waveguide router. To process a single image, the image is first flattened into a one-dimensional vector of length P. Then, the M mode signals are divided into N equal parts through a cascaded multimode 3-dB coupler. Each signal is used as a convolution kernel through N multimode weight modulators. Finally, the sliding output of the convolution is generated by the multimode waveguide router, which contains a mode demultiplexer and a multiplexer and has M+N-1 output ports.

[0038] Figure 2 This paper presents a comparison of experimental results for single-mode and multimode fan-in achieved through inverse design. (a) and (b) are scanning electron microscope images of a 14.1μm-long single-mode fan-in and a multimode fan-in in TE2 and TE1 modes, respectively. The inverse design ensures compactness and excellent performance. (c) shows insertion loss across a broadband wavelength range. The average loss for a single-mode fan-in in the C-band is approximately 3.53dB (dashed line), while the average loss for a multimode fan-in in the TE2-TE0 modes is only 0.32, 0.2, and 0.21dB (solid lines).

[0039] Figure 3(a) shows an optical microscope image of the fabricated multimode photon convolution accelerator chip. The chip is fabricated on a 180nm commercial SOI process platform and measures 0.6×0.7mm. 2 , integrating a tri-mode 3-dB coupler, a tri-mode core modulator, and tri-mode routing. (b) shows the chip's optoelectronic packaging, complete with vertical grating coupling and wire bonding, with a total insertion loss of less than 2.73dB, validating its excellent C-band performance.

[0040] Figure 4 The experimental results under a single wavelength are shown. (a) 1000 sets of random input vectors and convolution kernels were tested. Each convolution produces an output vector of length 6, generating a total of 6000 data points. The experimental data shows that the calculation results are closely aligned along the diagonal, indicating that the system has high accuracy and reliability. Bit accuracy (N b ) is fitted with a Gaussian standard deviation of 0.0072, with a calculated precision of 7 digits. In (b), a convolutional neural network (CNN) is used to classify handwritten digits (0-9) into ten categories, where the convolution layer is implemented using optical convolution (shown in orange). The image is first resized to 12×12 pixels and flattened into a 144×1 vector. Then, the photon convolution layer uses four 4×1 convolution kernels to generate four 146×1 feature maps. After applying the ReLU nonlinearity, the four 146×1 feature maps are reshaped into 584×1 vectors and then input to a fully connected layer with 10 neurons via SoftMax activation. We trained the MNIST dataset (70,000 images in total, 60,000 of which were used for training, with a training to validation ratio of 5:1) for 20 iterations. In the optical domain test, loading 1,000 images, the classification accuracy was 95.2%, close to the theoretical electrical domain accuracy of 95.8%. The confusion matrices in (c) and (d) show that optical noise has minimal impact on the classification accuracy, indicating stable performance of the system.

[0041] Figure 5 Experimental results using multiple wavelengths are presented. We tested at 1530nm and 1565nm wavelengths, achieving 6-bit and 7-bit computational accuracy, respectively. Recognition accuracy on the Fashion-MNIST dataset was 87.9%. This wavelength multiplexing technique supports multi-channel parallel processing, significantly improving computational throughput. In electrical testing, recognition accuracy was 89.2%, a 1.3% difference (compared to 0.6% for MNIST) likely due to differences in accuracy at different wavelengths.

[0042] The multimode photon convolution accelerator provided by the present invention integrates a multimode 3-dB coupler, a core modulator and a waveguide router, occupying only 0.6×0.7mm 2, compatible with commercial standard manufacturing processes. In the C-band, the average insertion loss of the TE2-TE0 mode beam synthesis is 0.32, 0.2 and 0.21dB, and the total insertion loss is less than 2.73dB at a wavelength of 1.55μm. The accelerator provides 7-bit computing accuracy and a classification accuracy of 95.2% on the MNIST dataset, verifying its high efficiency. Wavelength division multiplexing and mode division multiplexing technologies significantly improve throughput, and multi-wavelength channels are loaded in parallel, achieving 6-bit and 7-bit accuracy at 1530nm and 1565nm respectively, and the Fashion-MNIST optical domain accuracy is 87.9%. The accelerator provides 4.38THz available bandwidth and a computing density of up to 125.14TOPS / mm 2 By using a lossless multimode fan-in approach, the insertion loss bottleneck of traditional single-mode systems is circumvented, and a new method for improving the efficiency of convolution and other linear operations is proposed. This architecture demonstrates the broad application prospects of photon accelerators and provides important support for low-energy, high-throughput photonic computing platforms.

[0043] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-mode photon convolution accelerator based on lossless mode fan-in, characterized in that: Including intensity modulation module, cascaded multimode 3-dB coupler, multimode weight modulator array and multimode waveguide routing, The intensity modulation module is used to perform intensity modulation on the input M mode signals, and the M mode signals after intensity modulation are divided into N parts by the cascaded multimode 3-dB coupler, which are the first to Nth multimode signals, respectively, and each multimode signal includes the first to Mth modes; The multimode weight modulator array includes N parallel multimode electro-optical modulators. By changing the electrical drive signal of each multimode electro-optical modulator and mapping it to a convolution kernel in a convolution calculation, intensity modulation of each multimode signal is achieved, and different information of the convolution kernel is encoded into the optical domain. The electrical drive signal of each multimode electro-optical modulator corresponds one-to-one to a weight coefficient in the convolution kernel. The first to Nth multimode signals are respectively input into the N parallel multimode electro-optical modulator units to obtain the products of the first to Nth multimode signals and the corresponding weight coefficients. The multimode waveguide route includes N multimode waveguide units, each multimode waveguide unit includes a straight waveguide and several coupled waveguides, the several coupled waveguides are coupled with the straight waveguide of the corresponding multimode waveguide unit at the head end, and the several coupled waveguides are coupled with the straight waveguide of the adjacent multimode waveguide unit at the tail end; the products of the first to Nth multimode signals and the corresponding weight coefficients are sequentially input into the N multimode waveguide units, the products of the straight-through mode multimode signals supported by the straight-through waveguide and the corresponding weight coefficients are output from the corresponding straight-through waveguide output ports, and at the same time, the products of the coupled mode multimode signals supported by the several coupled waveguides of the adjacent multimode waveguide units and the corresponding weight coefficients are coupled and output, and the coupled waveguides at both ends respectively output the products of the first and Nth multimode signals and the corresponding weight coefficients.

2. The multimode photon convolution accelerator according to claim 1, characterized in that: The multimode weight modulator and the multimode waveguide routing are designed using an inverse design method.

3. The multimode photon convolution accelerator according to claim 1, characterized in that: The intensity modulation module includes M intensity modulators, which are used to encode input signals, and N multi-mode electro-optical modulators to encode convolution kernels of size N, and perform M×N multiplication and accumulation operations in a single clock cycle.

4. The multimode photon convolution accelerator according to claim 3, characterized in that: Through wavelength multiplexing technology, K wavelength channels are integrated into the same straight waveguide; 2×M×N multiplication and accumulation operations are performed in each wavelength channel.

5. The multimode photon convolution accelerator according to claim 1, characterized in that: The multi-mode electro-optical modulator adopts intensity modulation technology.

6. The multimode photon convolution accelerator according to claim 1, characterized in that: The multimode photonic convolution accelerator operates in the C-band.

7. The multi-mode photon convolution accelerator according to claim 1, characterized in that: The core material of all waveguides includes Si and the cladding material includes SiO2.

Images