A dual microring non-volatile photonic convolution accelerator

Abstract

This invention relates to a dual-microring non-volatile photonic convolution accelerator, comprising a multi-wavelength light source, a power amplifier, a 1:N power divider, a modulator array, a dual-microring array based on phase change materials, and a photodetector array; input data X1~X N The signal intensity is converted into N composite optical signals by a modulator and a 1:N power divider; the composite optical signals are then converted into Y1~Y2 by a dual micro-ring array based on phase change materials and a photodetector array. M Electrical signal output; the dual micro-rings based on phase change materials are used to select, train, store, and maintain photonic convolution kernel weight data for channel resonant wavelengths, completing dot product calculations; the dual micro-ring array based on phase change materials is arranged in an N×M matrix, where the resonant wavelength of any matrix element is different from that of all other matrix elements in its row / column; realizing input data X1~X N The convolution calculation is performed by multiplying a 1×N vector with an N×M matrix of photon convolution kernel weights. The advantages are a compact structure, high computational density, and non-volatile data.

Description

[Technical Field]

[0001] This invention relates to the field of photonic convolution accelerator technology, and more specifically to a dual-micro-ring non-volatile photonic convolution accelerator. [Background Technology]

[0002] As Moore's Law gradually approaches its limits, computing chips based on the von Neumann architecture face significant challenges in improving computing power and reducing energy consumption per unit of computing power. Convolutional neural networks (CNNs), by digitally simulating the brain's architecture and utilizing multi-layered convolutional kernels for image feature extraction, play a crucial role in numerous fields such as image recognition and classification. Traditional electrical von Neumann architecture computing chips can no longer meet the massive computing power and high energy efficiency requirements of applications like CNNs, which involve numerous convolutional computations. Photonic convolution accelerators, leveraging the advantages of high bandwidth, low latency, and low power consumption of optics, employ a non-von Neumann architecture, significantly improving the performance of CNN chips in terms of energy efficiency, computing power, and computational latency. Using a non-von Neumann architecture-based photonic convolution accelerator to perform the core matrix-vector multiplication function in neural networks can greatly enhance the overall computing power of the chip, achieving higher energy efficiency and higher computational density per unit. Photonic convolution accelerators for CNN image recognition have been extensively studied, and various implementation schemes have been proposed.

[0003] Among them, photonic convolution accelerators based on wavelength division multiplexing (WDM) technology can realize the core matrix-vector multiplication calculation in convolution calculation by utilizing the high concurrency of WDM (Tait AN, De Lima TF, Zhou E, et al. Neuromorphic photonic networks using silicon photonic weight banks[J]. Scientific reports, 2017, 7(1): 1-10.). However, most current schemes use a single micro-ring structure and use electro-optic or thermo-optic methods to change the intensity of the optical signal by changing the resonant wavelength of the single micro-ring structure, thereby realizing the multiplication function of the convolution kernel weights. The use of thermo-optic or electro-optic modulation has data volatility and is accompanied by large energy consumption and thermal crosstalk between adjacent channels. Secondly, in order to reduce the dynamic crosstalk between channels caused by the resonant wavelength shift during micro-ring modulation, a large channel spacing is often required, which limits the number of channels in the free spectrum range of the micro-ring and the further improvement of chip computing density. Finally, high-precision calculation requires the assistance of high-precision micro-ring phase modulation technology.

[0004] Phase change materials (PCMs) are a class of materials whose physical properties can be altered by optical or electrical pulses. They are typically composed of elements such as germanium (Ge), antimony (Sb), selenium (Se), and tellurium (Te). By controlling the intensity and width of the input pulse, PCMs can undergo different degrees of physical state changes, exhibiting state retention characteristics. The output light from PCMs in different physical states can exhibit varying degrees of phase and intensity changes, thus achieving multi-state phase and amplitude modulation.

[0005] This invention addresses the data volatility of existing photonic convolution accelerators, as well as the technical problems of insufficient channel capacity and decreased accuracy caused by dynamic crosstalk between channels due to the resonant wavelength drift caused by traditional single micro-ring weight adjustment. It provides technical improvements to the micro-ring array photonic convolution accelerator. [Summary of the Invention]

[0006] The purpose of this invention is to provide a dual-micro-ring array photonic convolution accelerator with a compact structure, high computational density, low dynamic crosstalk between channels, and non-volatile data.

[0007] To achieve the above objectives, the technical solution adopted by this invention is a dual-micro-ring non-volatile photonic convolution accelerator, comprising a multi-wavelength light source with M output wavelengths, a power amplifier, a 1:N power divider, an array of N modulators, a dual-micro-ring array based on phase change materials arranged in an N×M configuration, and an array of M photodetectors, where N≤M; the multi-wavelength light source generates wavelengths from λ1 to λ2. M A composite optical signal with wavelength number M is amplified, then divided into N composite optical signals with wavelength number M by a 1:N power divider before being input to the modulator array. The input data are X1 to X2. N The signal intensity is modulated into N composite optical signals with M wavelengths. These N composite optical signals then pass through an N×M array of phase-change material-based dual microrings. The dual microrings, connected in series via waveguides, are used to select and train the channel resonant wavelengths, store and maintain photonic convolution kernel weight data, and perform dot product calculations between the input optical signal and the photonic convolution kernel weight data. The dual microring array is arranged in an N×M matrix, where the resonant wavelength of any matrix element is different from all other matrix elements in its row / column. Laterally, the modulated N composite optical signals with M wavelengths pass through each of the M dual microrings and change their propagation direction to longitudinal, completing the dot product calculation X. i W ij Among them, X i The amplitude of the input optical signal (i.e., the output optical signal after passing through the modulator array) based on the dual micro-rings of the phase change material, W ijThe weighted data is based on the dual micro-ring photonic convolution kernel of phase change material, i = 1…N, j = 1…M; the N optical signals output from the dual micro-rings based on phase change material in the same column in the vertical direction are merged again into one composite optical signal through a bus waveguide, thus obtaining M composite optical signals; the M composite optical signals are converted into Y1~Y1~Y2~Y3~Y4~Y5~Y6~Y7~Y8~Y9~Y1~Y1~Y1~Y1~Y2 ... M Electrical signals, with inputs of X1 to X N The matrix multiplication of the 1×N matrix vector and the N×M matrix vector of the photon convolution kernel weights is performed to achieve convolution calculation, and the output electrical signal is: Where R is the photodetector responsivity.

[0008] Further, the dual microring based on phase change material includes a first microring resonator covered with phase change material PCM1 on a microring waveguide, a connecting waveguide between microrings covered with phase change material PCM2 on the waveguide, a second microring resonator covered with phase change material PCM3 on the microring waveguide, and a waveguide crossover connecting the first and second microring resonators; the photonic convolution accelerator uses electrical or optical pulse excitation to change the state of the phase change material on the dual microring based on phase change material, thereby changing the effective refractive index of the waveguide and adjusting the resonant wavelength of the dual microring based on phase change material; the relationship between the resonant wavelength of the dual microring based on phase change material and the effective refractive index is expressed as: n eff L=mλ m , where n eff The effective refractive index of the waveguide at the resonant wavelength is represented by L, the circumference of the microring is represented by m, and λ is an integer. m This is the resonant wavelength.

[0009] Preferably, the optical signal input from the lateral direction passes through the first microring resonator of the dual microrings based on phase change materials and reaches the waveguide connecting the microrings. Finally, it is output in the longitudinal direction through the second microring resonator. The phase change materials PCM1 and PCM3 adjust the effective refractive index so that the corresponding microrings are at the designed resonant wavelength. PCM2 introduces waveguide loss to attenuate the amplitude of the input optical signal. The whole process is equivalent to realizing XW dot product calculation, where X is the input optical signal based on the dual microrings of phase change materials, and W is the weight data of the photonic convolution kernel of the dual microrings based on phase change materials.

[0010] Preferably, the first microring resonator and the second microring resonator adopt the same design. The PCM1 and PCM3 are phase change materials with low loss in both crystalline and amorphous states. The PCM2 can be a phase change material with significantly increased loss before and after phase change, so as to adjust the optical transmission power.

[0011] Preferably, the phase change materials that can be used for PCM1 and PCM3 are low-loss phase change materials such as SbSe or SbS, and the phase change material that can be used for PCM2 is a phase change material such as Ge2Sb2Te5 (GST) that has a significant change in loss after phase change.

[0012] Preferably, the N×M matrix used for resonant wavelength selection selects the resonant wavelength according to the wavelength routing arrangement shown in the following N×M matrix, where the resonant wavelength of any matrix element is different from that of all other matrix elements in its row / column, where N≤M:

[0013]

[0014] Preferably, the intensity of the N-wavelength composite optical signal after passing through the 1:N power divider is normalized to 1, the intensity of the modulated N-wavelength composite optical signal is normalized to a value between 0 and 1, and the normalized value of the photonic convolution kernel weight data is normalized to a value between 0 and 1.

[0015] Preferably, the multi-wavelength light source includes an optical frequency comb and a multi-wavelength laser array.

[0016] Preferably, the 1:N power divider includes a 1:N MMI, a 1:2MMI tree structure, a Y-branch tree structure, and cascaded directional optical couplers.

[0017] Preferably, the modulator is a silicon-based thermo-optic modulator, a silicon-based PIN electro-optic modulator, or a silicon-based PN electro-optic modulator; the photodetector is a waveguide-type germanium-silicon detector or a germanium detector.

[0018] Compared with existing technologies, the dual-micro-ring non-volatile photonic convolution accelerator of this invention has the following advantages: 1. It has data non-volatility. Phase change materials can change their state characteristics through optical / electric pulse excitation, exhibiting good state preservation characteristics and data non-volatility. 2. It has low power consumption. Once the convolution kernel parameters are trained, no additional electrical or optical energy consumption is required to maintain them, exhibiting low power consumption. 3. It has high computational density. The dual-micro-ring unit array structure completes wavelength division multiplexing and demultiplexing functions, exhibiting high computational concurrency. At the same time, the channel resonant wavelength remains unchanged. The scheme of using light intensity changes to realize optical computation can reduce dynamic crosstalk between channels, increase channel density, and ultimately achieve higher computational density. 4. The device is CMOS compatible. All devices can be integrated onto the same chip through monolithic integration technology. The chip has good stability, robustness, and large-scale scalability. [Attached Image Description]

[0019] Figure 1 This is a diagram of a dual-micro-ring non-volatile photonic convolution accelerator architecture.

[0020] Figure 2 This is a diagram of the convolution kernel weight unit structure.

[0021] Figure 3 This is a schematic diagram of changes in light signal intensity.

[0022] The reference numerals and components involved in the attached figures are as follows: 1. Multi-wavelength light source, 2. Power amplifier, 3. 1:N power divider, 4. Input modulator array, 5. Convolution kernel weighting unit, 51. First microring resonator, 52. Microring interconnecting waveguide, 53. Second microring resonator, 54. Waveguide crossover, 6. Photodetector array.

Detailed Implementation Methods

[0023] The present invention will now be further described with reference to the embodiments and the accompanying drawings.

[0024] Example

[0025] This embodiment implements a dual-micro-ring non-volatile photon convolution accelerator.

[0026] This embodiment presents a dual-micro-ring non-volatile photonic convolution accelerator that utilizes phase-change materials to achieve light intensity modulation and exhibits data non-volatility. The dual-micro-ring device structure is used to select the channel wavelength, and its resonant wavelength remains unchanged during the adjustment of the convolution kernel weight value, reducing dynamic crosstalk between channels and increasing channel density, which is beneficial for achieving higher chip computing density. Figure 1 This is a diagram of a dual-micro-ring non-volatile photonic convolution accelerator architecture. (See attached diagram.) Figure 1 As shown, this embodiment of a dual-micro-ring non-volatile photonic convolution accelerator includes:

[0027] 1. Multi-wavelength light source with M wavelengths. Optional multi-wavelength light source systems include optical frequency combs, multi-wavelength laser arrays, etc.

[0028] 2. Power amplifier, used to amplify the output light from multi-wavelength light sources to meet the needs of subsequent optical signal processing and detection.

[0029] 3. 1:N power divider, the optional 1:N power dividers include 1:N MMI, 1:2MMI tree structure, Y branch tree structure, cascaded directional optical coupler, etc.

[0030] 4. Input modulator array, the number of modulators is N, and the selectable modulators include silicon-based thermo-optic modulators, silicon-based PIN electro-optic modulators, and silicon-based PN electro-optic modulators; the photodetector is a waveguide-type germanium-silicon detector or a germanium detector.

[0031] 5. Convolutional kernel weight units arranged in an array, with the array of convolutional kernel weight units arranged in an N×M pattern, where N≤M.

[0032] 6. A photodetector array, numbered M, with selectable detectors including waveguide-type germanium-silicon detectors or germanium detectors.

[0033] Figure 2 This is a diagram of the convolution kernel weight unit structure. (See attached diagram.) Figure 2 As shown in the figure, this embodiment discloses a dual-micro-ring non-volatile photonic convolution accelerator, wherein the convolution kernel weight unit includes:

[0034] 1. The first microring resonator and the phase change material PCM1 covering the microring waveguide.

[0035] 2. Micro-ring connecting waveguides and the phase change material PCM2 covering the waveguides.

[0036] 3. The second microring resonator and the phase change material PCM3 covering the microring waveguide.

[0037] 4. Waveguide crossover.

[0038] The relationship between the resonant wavelength of a microring resonator and the effective refractive index of the microring can be expressed as: n eff L=mλ m , where n eff The effective refractive index of the waveguide at the resonant wavelength is represented by L, the circumference of the microring is represented by m, and λ is an integer. m The resonant wavelength is determined by the waveguide's effective refractive index. The resonant wavelength of the microring can be shifted by changing the effective refractive index of the waveguide. This embodiment describes a dual-microring non-volatile photonic convolution accelerator, which uses electrical or optical pulse excitation to change the state of the phase change material on the microring, thereby altering the waveguide's effective refractive index and adjusting the microring's resonant wavelength to the designed wavelength of the photonic convolution accelerator.

[0039] As attached Figure 2 As shown by the black arrow, this embodiment describes a dual-micro-ring non-volatile photonic convolution accelerator. The propagation path of the optical signal at the resonant wavelength of the convolution kernel weight unit is as follows: The input optical signal in the lateral direction passes through the first micro-ring resonator and reaches the connecting waveguide covered by phase change material PCM2. The phase change material introduces waveguide loss, which attenuates the optical amplitude. Finally, it is output in the longitudinal direction through the second micro-ring resonator. The whole process is equivalent to realizing XW dot product calculation, where X is the input optical signal of the convolution kernel weight unit and W is the weight data of the convolution kernel weight unit. Figure 3 This is a schematic diagram illustrating changes in optical signal intensity. (See attached diagram) Figure 3 As shown, X represents the amplitude of the optical signal after modulation (i.e., the input optical signal to the convolution kernel weight unit), with a normalized amplitude value ranging from 0 to 1; W represents the attenuation degree of the optical amplitude after output from the convolution kernel weight unit (weight data), with a normalized weight data value ranging from 0 to 1. The two microring resonators can employ the same design, and selectable phase change materials include GST, SbSe, SbS, SST, etc.

[0040] This embodiment describes a dual-micro-ring non-volatile photon convolution accelerator, the working principle of which is as follows:

[0041] 1. Multi-wavelength light sources produce wavelengths from λ1 to λ2. M A composite optical signal with wavelength number M;

[0042] 2. The composite optical signal with wavelength number M is output through an optical power amplifier to meet the energy requirements of subsequent optical processing;

[0043] 3. A composite optical signal with M wavelengths is divided into N composite optical signals with M wavelengths each by a 1:N power divider;

[0044] 4. An N-channel composite optical signal input modulator array with M wavelengths modulates the intensity of the composite optical signal, thereby converting the input signal X1 to X... N This is represented as the input optical amplitude information (i.e., the intensity of a composite optical signal with N wavelengths and M wavelengths);

[0045] 5. The modulated N-wavelength composite optical signal passes through an N×M convolution kernel weight unit matrix. By adjusting the phase change material on the microring resonator, the resonant wavelengths of the convolution kernel weight units are arranged according to the following wavelength routing method, where N≤M. That is, an N (row) × M (column) dimensional (square) matrix is ​​used to select the resonant wavelengths, and the resonant wavelength of any matrix element is different from that of all other matrix elements in its row / column.

[0046]

[0047] 6. In the horizontal direction of the convolution kernel weight unit, the modulated optical signal changes its propagation direction to the vertical direction after passing through the convolution kernel weight unit at the corresponding resonant wavelength, and completes the dot product calculation X. i W ij , where X i W is the input optical signal to the weight unit of the convolution kernel in the i-th row. ij This is the weight data of the convolution kernel weight unit in the i-th row and j-th column, where i = 1…N, j = 1…M. It carries the input signal X1-X. N The N composite optical signals of the information enter the vertical direction after passing through M convolution kernel weighting units at corresponding resonant wavelengths in the horizontal direction. The N optical signals in the same column in the vertical direction are merged into one composite optical signal again through the bus waveguide, and the M composite optical signals are output in the vertical direction.

[0048] 7. The composite optical signal carrying the X input information and W weight information in the longitudinal direction is accumulated by a photodetector, thereby completing the matrix-vector product calculation of a 1×N row vector and an N×M two-dimensional vector: Among them, Y jThis is the output of the j-th detector, where R is the photodetector responsivity, and X... i W is the input optical signal to the weight unit of the convolution kernel in the i-th row. ij It is the weight data of the convolution kernel weight unit in the i-th row and j-th column, i = 1…N, j = 1…M.

[0049] This embodiment presents a dual-micro-ring non-volatile photonic convolution accelerator, which solves the technical problem of data volatility in photonic convolution kernel accelerators, as well as the technical problems of insufficient channel capacity and decreased accuracy caused by dynamic crosstalk between channels due to the resonant wavelength drift caused by traditional single-micro-ring weight adjustment. Its technical effects are as follows:

[0050] 1. It exhibits data non-volatility. Phase change materials can change their state properties through optical / electric pulse excitation, exhibiting good state retention characteristics and data non-volatility.

[0051] 2. Low power consumption. The photonic convolution accelerator in this embodiment does not require additional electrical or optical energy to maintain the convolution kernel parameters after training, thus exhibiting low power consumption.

[0052] 3. High computational density. The photonic convolution accelerator in this embodiment uses a dual-micro-ring unit array structure to complete wavelength division multiplexing and demultiplexing functions, which has high computational concurrency. At the same time, the channel resonant wavelength remains unchanged. The scheme of realizing optical computing by utilizing light intensity changes can reduce dynamic crosstalk between channels, increase channel density, and ultimately achieve higher computational density.

[0053] 4. The photonic convolution accelerator in this embodiment is CMOS compatible and can integrate all devices onto the same chip through monolithic integration technology. The chip has good stability and large-scale scalability.

[0054] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and additions without departing from the principle of the present invention, and these improvements and additions should also be considered within the scope of protection of the present invention.

Claims

1. A dual-micro-ring non-volatile photon convolution accelerator, characterized in that: The system includes a multi-wavelength light source with M output wavelengths, a power amplifier, a 1:N power divider, an array of N modulators, an N×M dual micro-ring array based on phase change materials, and an array of M photodetectors, where N≤M; the multi-wavelength light source generates wavelengths of... A composite optical signal with wavelength number M is amplified, then divided into N composite optical signals with wavelength number M by a 1:N power divider before being input to the modulator array. The intensity of the composite optical signal with N wavelengths is loaded by the modulator. The modulated N-wavelength composite optical signals pass through an N×M array of dual micro-rings based on phase change materials. The phase change material-based dual microrings employ a waveguide-connected structure to select, train, store, and maintain photonic convolution kernel weight data for the channel resonant wavelengths, completing the dot product calculation between the input optical signal and the photonic convolution kernel weight data. The phase change material-based dual microring array is arranged in an N×M matrix, where the resonant wavelength of any matrix element is different from all other matrix elements in its row / column. In the transverse direction of the phase change material-based dual microring array, N modulated composite optical signals with M wavelengths pass through M phase change material-based dual microrings, changing their propagation direction to longitudinal and completing the dot product calculation. ,in It is based on the amplitude of the dual micro-ring input optical signal of the phase change material. It is based on the weight data of the dual micro-ring photonic convolution kernel of phase change material, i=1…N, j=1…M; In the longitudinal direction, the N optical signals output from the dual micro-ring structure based on phase change material in the same column are merged again into a single composite optical signal through the bus waveguide, thereby obtaining M composite optical signals. M-channel composite optical signals are converted by a photodetector array into... Electrical signal, to realize input as The matrix multiplication of the 1×N matrix vector and the N×M matrix vector of the photon convolution kernel weights is performed to achieve convolution calculation, and the output electrical signal is: , where R is the photodetector responsivity; The dual microring based on phase change material includes a first microring resonator covered with phase change material PCM1 on a microring waveguide, a microring connecting waveguide covered with phase change material PCM2 on the waveguide, a second microring resonator covered with phase change material PCM3 on the microring waveguide, and a waveguide crossover connecting the first and second microring resonators; the photonic convolution accelerator uses electrical pulse or optical pulse excitation to change the state of the phase change material on the dual microring based on phase change material, thereby changing the effective refractive index of the waveguide and adjusting the resonant wavelength of the dual microring based on phase change material. The relationship between the resonant wavelength and effective refractive index of the dual microring based on phase change materials is expressed as follows: ,in, The effective refractive index of the waveguide at the resonant wavelength is represented by L, the circumference of the microring is represented by m, and m is an integer. This is the resonant wavelength.

2. The dual-micro-ring non-volatile photon convolution accelerator according to claim 1, characterized in that: The optical signal input from the lateral direction passes through the first microring resonator of the dual microrings based on phase change materials and reaches the waveguide connecting the microrings. Finally, it is output in the longitudinal direction through the second microring resonator. The phase change materials PCM1 and PCM3 adjust the effective refractive index to make the corresponding microrings reach the designed resonant wavelength. PCM2 introduces waveguide loss to attenuate the amplitude of the input optical signal. The whole process is equivalent to realizing the XW dot product calculation, where X is the input optical signal based on the dual microrings of phase change materials, and W is the weight data of the photonic convolution kernel based on the dual microrings of phase change materials.

3. The dual-micro-ring non-volatile photon convolution accelerator according to claim 2, characterized in that: The first microring resonator and the second microring resonator adopt the same design. PCM1 and PCM3 are phase change materials with low loss in both crystalline and amorphous states, while PCM2 is a phase change material with significantly increased loss before and after phase change, in order to adjust the optical transmission power.

4. The dual-micro-ring non-volatile photon convolution accelerator according to claim 3, characterized in that: The phase change materials selected for PCM1 and PCM3 are SbSe or SbS phase change materials, while the phase change material selected for PCM2 is Ge2Sb2Te5 phase change material.

5. The dual-micro-ring non-volatile photon convolution accelerator according to claim 1, characterized in that: The N×M matrix used for resonant wavelength selection selects the resonant wavelength according to the wavelength routing arrangement shown in the following N×M matrix. The resonant wavelength of any given matrix element is different from that of all other matrix elements in its row / column, where N... M: 。 6. The dual-micro-ring non-volatile photon convolution accelerator according to claim 1, characterized in that: The intensity of the N-wavelength composite optical signal after passing through the 1:N power divider is normalized to 1, the intensity of the modulated N-wavelength composite optical signal is normalized to a value between 0 and 1, and the normalized value of the photonic convolution kernel weight data is between 0 and 1.

7. The dual-micro-ring non-volatile photon convolution accelerator according to claim 1, characterized in that: The multi-wavelength light source includes an optical frequency comb and a multi-wavelength laser array.

8. The dual-micro-ring non-volatile photon convolution accelerator according to claim 1, characterized in that: The 1:N power divider includes a 1:N MMI, a 1:2 MMI tree structure, a Y-branch tree structure, and cascaded directional optical couplers.

9. A dual-micro-ring non-volatile photon convolution accelerator according to claim 1, characterized in that: The modulator is a silicon-based thermo-optic modulator, a silicon-based PIN electro-optic modulator, or a silicon-based PN electro-optic modulator; the photodetector is a waveguide-type germanium-silicon detector or a germanium detector.

Images