A non-volatile photonic convolution accelerator

Abstract

This invention relates to a non-volatile photonic convolution accelerator, comprising a continuous-wave laser, a first-stage 1:N beam splitter, a modulator array, a second-stage 1:M beam splitter, a beam splitter array based on phase change materials, and a photoelectric conversion unit; the continuous-wave laser outputs a single-wavelength light source which is evenly split into N optical signals by the first-stage 1:N beam splitter, with input data X1 to X... N The signal intensity is modulated into N optical signals; each of the N optical signals passes through a second-stage 1:M beam splitter; each of the N×M optical signals is connected to a beam splitter based on a phase change material; the beam splitter based on the phase change material adopts an MZI device structure, used for training and storing photonic convolution kernel weight data, and performing dot product between the input data and the photonic convolution kernel weight data; the N×M optical signals are converted into electrical signals by a photoelectric conversion unit, performing matrix-vector product between the input data and the convolution kernel weight data. The advantages are simple inter-channel phase modulation, no need for an external multi-wavelength light source system, and non-volatile data.

Description

[Technical Field]

[0001] This invention relates to the field of photonic convolution accelerator technology, and more specifically to a non-volatile photonic convolution accelerator. [Background Technology]

[0002] With the explosive growth of information in modern times, traditional von Neumann architecture computing, due to its independent information processing and storage units, suffers from a mismatch between memory access speed and information processing speed, limiting its computing bandwidth. Frequent data storage access leads to high computing energy consumption, creating a bottleneck between information "computation" and "storage." Given the massive computing power demands of applications such as machine learning, artificial intelligence, and smart healthcare, researching next-generation, more efficient and energy-saving computing paradigms is particularly important.

[0003] Photonic computing has a natural advantage in parallelism, enabling parallel computation in multiple dimensions such as wavelength, mode, polarization, phase, and amplitude. Compared to electrical computing, photonic computing has achieved significant advantages in energy efficiency, speed, and information throughput. The problem of high-speed data transmission in electrical computing can be easily solved in optics through low-power, high-bandwidth waveguides.

[0004] The numerous convolutional computations in convolutional neural networks (CNNs) limit their data processing speed and consume significant energy due to their high computational power requirements. Employing photonic convolutional accelerators based on non-von Neumann architectures to perform the core matrix-vector multiplication function in neural networks can greatly improve the overall computing power of the chip, achieving higher energy efficiency and higher computational density per unit area. Photonic convolutional accelerators for CNNs have been extensively studied, and various implementation schemes have been proposed.

[0005] Photonic convolution accelerators based on spatial light utilize multi-level diffraction lenses to achieve large-scale matrix computations, which have a large computational capacity, but their systems are relatively large and difficult to integrate.

[0006] Photonic convolution accelerators based on Mach-Zehnder interferometer (MZI) thermal phase shifter arrays achieve arbitrary matrix-vector multiplication through matrix decomposition. However, they require a large number of MZI units and electrodes for thermal phase control. The challenges they face include thermal crosstalk between adjacent devices, complex and high-precision phase control, and large-scale matrix scalability.

[0007] The micro-ring array photonic convolution accelerator based on wavelength division multiplexing (WDM) utilizes the high concurrency of WDM to achieve matrix-vector multiplication by controlling the resonant wavelength of the micro-ring through electro-optic and thermo-optic methods. However, the micro-ring control requires high precision and a complex and stable external multi-wavelength light source system.

[0008] Furthermore, modulation via thermo-optical or electro-optical methods requires additional energy to maintain the convolution weight data, resulting in data volatility.

[0009] Phase change materials (PCMs) are substances whose physical properties can be transformed between crystalline, semi-crystalline, and amorphous states through light / electric pulse excitation, and which maintain this transformation at room temperature. They are typically composed of elements such as germanium (Ge), antimony (Sb), selenium (Se), and tellurium (Te). The process of these physical property transformations is called a phase change process, and the output light energy can be multi-state regulated by controlling the degree of crystallization and amorphization of the material.

[0010] This invention addresses the technical problems of existing photonic convolution accelerators, such as data volatility, the need for complex phase modulation between channels, and the need for complex peripheral multi-wavelength light source systems, and makes technical improvements to photonic convolution accelerators. [Summary of the Invention]

[0011] The purpose of this invention is to provide a photonic convolution accelerator with simple inter-channel phase modulation, no need for an external multi-wavelength light source system, and non-volatile data.

[0012] To achieve the above objectives, the technical solution adopted by this invention is a non-volatile photonic convolution accelerator, comprising a continuous-wave laser for outputting a single-wavelength light source, a first-stage 1:N beam splitter, an N-level modulator array, a second-stage 1:M beam splitter, an N×M phase-change material-based beam splitter array, and a photoelectric conversion unit; the continuous-wave laser outputs a single-wavelength optical signal, which, after passing through the first-stage 1:N beam splitter, is evenly divided into N optical signals and input to the modulator array, with input data X1 to X... N The input data is loaded into N optical signals via a modulator; these N optical signals are then further divided into M optical signals by N second-stage 1:M splitters; the N×M optical signals after passing through the second-stage 1:M splitters are each connected to a phase change material-based splitter; the phase change material-based splitter uses an MZI device structure for training, storing, and maintaining photonic convolution kernel weight data, and performing dot product calculations between the input optical signal and the photonic convolution kernel weight data; the N×M optical signals after passing through the phase change material-based splitter are converted into electrical signals by a photoelectric conversion unit, realizing the 1×N vector input data X1~X... N The information convolution calculation is completed by multiplying the matrix vectors of the photon convolution kernel weight data and the N×M vector.

[0013] Furthermore, the phase change material-based beam splitter is an MZI device with upper and lower arms covered by phase change material. The phase change material state of the upper and lower arms is changed by optical / electric pulse driving, thereby changing the effective refractive index of the covered waveguide. This introduces a phase difference between the upper and lower arms of the MZI device, causing the output light intensity of the upper and lower output ports of the MZI device to change.

[0014] Preferably, the output optical signal intensities of the upper and lower output ports of the MZI device are represented as P1 and P2, respectively. The weight W of the beam splitter based on the phase change material is normalized. When P1 = P2, W is 0; when P1 is the maximum output light intensity, W is 1; and when P2 is the maximum output light intensity, W is -1. By adjusting the state of the phase change material, the weight W of the beam splitter array based on the phase change material can be varied in the range of -1 to 1.

[0015] Furthermore, the photoelectric conversion unit includes an array of N×(2M) photodetectors and an array of M subtractors; the optical signals output from the upper and lower output ports of the phase change material-based beam splitter are detected by the left and right photodetectors respectively to complete the photoelectric signal conversion; the output electrical signals of the left and right photodetectors arranged in the same column are connected together and accumulated in the electrical domain, and the subtractors arranged in the same column subtract the accumulated electrical signals from the left and right channels: X i ·W ij =R(P1) ij -P2 ij ), where X i It is the input data, W ij The photon convolution kernel weights, P1, are based on phase change material beam splitters. ij It is the output optical signal intensity at the output port of the MZI device, P2 ij The output light signal intensity is given by the MZI device, where R is the photodetector responsivity, i = 1…N, j = 1…M; thus, matrix-vector multiplication is performed between the input matrix vector X and the convolution kernel weight matrix vector W, resulting in the output signal Y. i for:

[0016]

[0017] Preferably, the photodetector is a waveguide-type germanium-silicon or germanium detector.

[0018] Preferably, the 1:N beam splitter is a 1:N MMI, a 1:2MMI tree structure, a Y-branch tree structure, or a cascaded directional optical coupler.

[0019] Preferably, the modulator is a silicon-based thermo-optic modulator, a silicon-based PIN electro-optic modulator, or a silicon-based PN electro-optic modulator.

[0020] Preferably, the 1:M splitter is a 1:M MMI, a 1:2MMI tree structure, a Y-branch tree structure, or a cascaded directional optical coupler.

[0021] Preferably, the phase change material is Ge2Sb2Te5 (GST), Ge2Sb2Se4Te1 (GSST), SbS, SbSe, or other low-loss non-volatile phase change materials.

[0022] Compared with existing technologies, the non-volatile photonic convolution accelerator of this invention has the following advantages: 1. It uses phase-change materials, which can maintain photonic convolution kernel weight data without additional photoelectric energy consumption, and has the characteristics of low energy consumption and non-volatile data; 2. It adopts a single-wavelength modulation scheme, which eliminates the need for a complex external multi-wavelength light source system, greatly reducing the complexity of the system and avoiding the accuracy degradation problem caused by the instability of the light source in the wavelength division multiplexing system; 3. It adopts an incoherent direct detection scheme between different channels, which eliminates the need for complex phase modulation between different channels, and greatly reduces the difficulty of phase modulation as the channel scale increases; 4. It can concurrently realize the calculation of M convolution kernel matrices of size N, and has high concurrent computing capabilities; 5. It can realize the numerical calculation of convolution kernel weights in the real number domain; 6. The system calibration difficulty is low, and it can realize fast system correction; 7. The unit device is CMOS compatible, and it can realize larger-scale matrix-vector multiplication calculation through monolithic integration technology, which has good system scalability; 8. It has low wavelength sensitivity, is not sensitive to process, and is not sensitive to optical phase between different channels, so the system has good robustness. [Attached Image Description]

[0023] Figure 1 This is a schematic diagram of a non-volatile photonic convolution accelerator.

[0024] Figure 2 This is a schematic diagram of a spectrometer based on phase change materials.

[0025] Figure 3 This is a diagram showing the relationship between the phase difference between the two arms of a beam splitter based on phase change materials and the output light intensity of the upper and lower output ports.

[0026] The reference numerals and components involved in the attached figures are as follows: 1. Continuous wave laser, 2. First-stage 1:N beam splitter, 3. Modulator array, 4. Second-stage 1:M beam splitter, 5. Beam splitter based on phase change material, 6. Photodetector, 7. Subtractor.

Detailed Implementation Methods

[0027] The present invention will now be further described with reference to the embodiments and the accompanying drawings.

[0028] Example

[0029] This embodiment implements a non-volatile photonic convolution accelerator.

[0030] This embodiment presents a non-volatile photonic convolution accelerator, which aims to address the data volatility problem in photonic convolution accelerators, reduce the complexity of phase modulation in existing technical solutions, and meet the requirements of complex peripheral multi-wavelength light source systems.

[0031] This embodiment presents a non-volatile photonic convolution accelerator that utilizes the non-volatility of phase change materials and employs a single-wavelength light source modulation scheme. It does not require a complex multi-wavelength light source system in the periphery, nor does it require complex phase modulation techniques between channels. It has good system scalability, thereby realizing a large-scale, high-efficiency, and low-latency photonic convolution computing accelerator.

[0032] Figure 1 This is a schematic diagram of a non-volatile photonic convolution accelerator. (See attached diagram.) Figure 1 As shown in the figure, this embodiment of a non-volatile photonic convolution accelerator includes the following structure:

[0033] 1. Continuous wave laser, used to output a single wavelength light source.

[0034] 2. First-stage 1:N beam splitter. Optional 1:N beam splitters include 1:N MMI, 1:2MMI tree structure, Y-branch tree structure, cascaded directional optical couplers, and other power splitting devices.

[0035] 3. Modulator array, with a quantity of N, that is, including N modulators. The modulators can be silicon-based thermo-optic modulators, silicon-based PIN electro-optic modulators, or silicon-based PN electro-optic modulators, etc.

[0036] 4. Second-stage 1:M splitter. Optional 1:M splitters include 1:M MMI, 1:2MMI tree structure, Y-branch tree structure, cascaded directional optical couplers, and other power splitting devices.

[0037] 5. Spectrometers based on phase change materials; Figure 2 This is a schematic diagram of a spectrometer based on phase change materials. (See attached diagram) Figure 2 As shown, the beam splitter structure based on phase change materials can be selected from Ge2Sb2Te5 (GST), Ge2Sb2Se4Te1 (GSST), SbS, SbSe or other low-loss non-volatile phase change materials, etc.; the beam splitter is arranged in an N×M matrix, with a quantity of N×M.

[0038] 6. Photodetector array, the photodetectors can be waveguide-type germanium-silicon or germanium detectors, etc.; each pair of detectors appears in pairs, and the overall array is arranged in a matrix, with a quantity of N×(2M).

[0039] 7. Subtractors, arranged in an array, number M.

[0040] This embodiment describes a non-volatile photonic convolution accelerator. The specific implementation principle of the chip architecture is as follows:

[0041] 1. The continuous wave laser outputs a single-wavelength optical signal, which is then split into N optical signals by a first-stage 1:N splitter and input (connected) to a modulator array, i.e., N modulators.

[0042] II. Input data X1~X N The optical signal intensity is applied to the corresponding optical path (1-N optical paths) by the modulator.

[0043] Third, the N optical signals with the input data loaded are divided into M optical signals again by N second-stage 1:M splitters.

[0044] IV. The optical signals from the second-stage 1:M splitter are each connected to a phase change material-based splitter. (See attached diagram) Figure 2 As shown, a beam splitter structure based on phase change materials can cover both arms of an MZI (Mach-Zehnder interferometer) device with phase change materials. Figure 3 This is a diagram showing the relationship between the phase difference between the two arms of a beam splitter based on phase change materials and the output light intensity of the upper and lower output ports. (See attached diagram.) Figure 3 As shown, this embodiment discloses a non-volatile photonic convolution accelerator. The upper and lower arms of the MZI device are driven by optical or electrical pulses, causing a change in the state of the phase change material and thus altering the effective refractive index of the covered waveguide. This introduces a phase difference between the two arms of the MZI, which leads to changes in the output light intensity at the upper and lower output ports. The output light signal intensities at the upper and lower output ports are denoted as P1 and P2, respectively. The weights are normalized: a weight of 0 when P1 = P2, a weight of 1 when P1 is the maximum output light intensity, and a weight of -1 when P2 is the maximum output light intensity. By adjusting the state of the phase change material, the weights of the phase change material-based beam splitter corresponding to the N×M matrix beam splitter can be varied from -1 to 1.

[0045] V. The two output beams of the beam splitter based on phase change materials are detected by different detectors. The output signals of the same detector array are accumulated in the electrical domain, and the corresponding two electrical signals are subtracted by a subtractor, thus completing the core computational unit in the convolution calculation: matrix-vector multiplication.

[0046]

[0047]

[0048] Among them, Y i It is the calculated output, X i It is the input data, W ij P1 is the photon convolution kernel weight of a beam splitter based on phase change materials. ijThe output optical signal intensity at the output port of the beam splitter based on phase change materials, P2 ij R is the output optical signal intensity at the lower port of the beam splitter based on phase change material, where R is the photodetector responsivity, i = 1…N, j = 1…M.

[0049] This embodiment presents a non-volatile photonic convolution accelerator, addressing the data volatility problem of photonic convolution accelerators and the issue that existing solutions require complex phase modulation between channels or complex external multi-wavelength light source systems; its technical advantages are as follows:

[0050] a. Using phase change materials, the photon convolution kernel weight data can be maintained without additional photoelectric energy consumption, and it has the characteristics of low energy consumption and non-volatile data.

[0051] b. By adopting a single-wavelength modulation scheme, there is no need for a complex external multi-wavelength light source system, which greatly reduces the complexity of the system and avoids the decrease in accuracy caused by the instability of the light source in the wavelength division multiplexing system.

[0052] c. The non-coherent direct detection scheme is used between different channels, which eliminates the need for complex phase modulation between different channels. As the channel scale increases, the difficulty of phase modulation is greatly reduced.

[0053] d. The accelerator system can concurrently perform calculations of M convolution kernel matrices of size N, and has high concurrent computing capabilities.

[0054] e. The accelerator system can perform numerical calculations of convolution kernel weights in the real number field.

[0055] f. The accelerator system is easy to calibrate and can achieve rapid system correction.

[0056] g. The accelerator system described above has CMOS-compatible unit devices and can achieve larger-scale matrix-vector multiplication calculations through monolithic integration technology, thus exhibiting good system scalability.

[0057] h. The accelerator system has low wavelength sensitivity, is not sensitive to process, and is not sensitive to optical phase between different channels, thus exhibiting good robustness.

[0058] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and additions without departing from the principle of the present invention, and these improvements and additions should also be considered within the scope of protection of the present invention.

Claims

1. A non-volatile photon convolution accelerator, characterized in that: The system includes a continuous-wave laser for outputting a single-wavelength light source, a first-stage 1:N beam splitter, an N-level modulator array, a second-stage 1:M beam splitter (also numbered N), an N×M phase-change material-based beam splitter array, and a photoelectric conversion unit. The continuous-wave laser outputs a single-wavelength optical signal, which, after passing through the first-stage 1:N beam splitter, is evenly divided into N optical signals and input to the modulator array, with input data X1 to X... N The modulator loads the N optical signal intensities; the N optical signals loaded with input data are then divided into M optical signals again by N second-stage 1:M splitters. The N×M optical signals after passing through the second-stage 1:M splitter are each connected to a phase change material-based splitter. The phase change material-based splitter employs an MZI device structure for training, storing, and maintaining photonic convolution kernel weight data, and performs dot product calculations between the input optical signal and the photonic convolution kernel weight data. The N×M optical signals after passing through the phase change material-based splitter are converted into electrical signals by a photoelectric conversion unit, realizing the input data X1 to X2 of a 1×N vector. N The information convolution calculation is completed by multiplying the matrix vectors of the photon convolution kernel weight data and the N×M vector.

2. The non-volatile photon convolution accelerator according to claim 1, characterized in that: The phase change material-based beam splitter is an MZI device comprising upper and lower arms covered with phase change material. Optical / electrical pulses drive the phase change material state of the upper and lower arms, altering the effective refractive index of the covered waveguide. This introduces a phase difference between the upper and lower arms of the MZI device, causing changes in the light intensity at the upper and lower output ports. The output light signal intensities at the upper and lower output ports of the MZI device are denoted as P1 and P2, respectively. The weight W of the phase change material-based beam splitter is normalized: W is 0 when P1 = P2, W is 1 when P1 is the maximum output light intensity, and W is -1 when P2 is the maximum output light intensity. By adjusting the phase change material state, the weight W of the phase change material-based beam splitter array varies from -1 to 1.

3. A non-volatile photon convolution accelerator according to claim 1, characterized in that: The photoelectric conversion unit includes an array of N×(2M) photodetectors and an array of M subtractors. The optical signals output from the upper and lower output ports of the phase change material-based beam splitter are detected by the left and right photodetectors to complete the photoelectric signal conversion. The output electrical signals of the left and right photodetectors arranged in the same column are connected together and accumulated in the electrical domain. The subtractors arranged in the same column subtract the accumulated electrical signals from the left and right channels: X i ·W ij =R(P1) ij -P2 ij ), where X i It is the input data, W ij P1 is the photon convolution kernel weight of a beam splitter based on phase change materials. ij It is the optical signal intensity at the output port of the MZI device, P2 ij The output port optical signal intensity is given by the MZI device, R is the photodetector responsivity, i = 1…N, j = 1…M; thus, matrix-vector multiplication is performed between the input matrix vector X and the convolution kernel weight matrix vector W, and the output signal Y is obtained. i for:

4. A non-volatile photon convolution accelerator according to claim 3, characterized in that: The photodetector is a waveguide-type germanium-silicon or germanium detector.

5. A non-volatile photon convolution accelerator according to claim 1, characterized in that: The 1:N beam splitter is a 1:N MMI, a 1:2MMI tree structure, a Y-branch tree structure, or a cascaded directional optical coupler.

6. A non-volatile photon convolution accelerator according to claim 1, characterized in that: The modulator is a silicon-based thermo-optic modulator, a silicon-based PIN electro-optic modulator, or a silicon-based PN electro-optic modulator.

7. A non-volatile photon convolution accelerator according to claim 1, characterized in that: The 1:M splitter is a 1:M MMI, 1:2MMI tree structure, Y-branch tree structure, or cascaded directional optical coupler.

8. A non-volatile photon convolution accelerator according to claim 1, characterized in that: The phase change material is a low-loss, non-volatile phase change material.

9. A non-volatile photon convolution accelerator according to claim 8, characterized in that: The low-loss non-volatile phase change material is Ge2Sb2Te5, Ge2Sb2Se4Te1, SbS, or SbSe phase change material.

Images