Neuromorphic Device

Abstract

A neuromorphic device (10) according to the present invention includes an M×1 input array (11) configured to convert an electrical input signal into an optical signal, an M×N processing unit (12) configured to execute photonic vector-matrix multiplication on the optical signal, an N×1 spike unit (13) configured to generate an optical spike signal based on an output signal from the M×N processing unit, an N×1 readout unit (14) configured to weight the optical spike signal, and optical waveguides (102) optically coupled to the M×1 input array, the M×N processing unit, the N×1 spike unit, and the N×1 readout unit, respectively.　With this arrangement, this invention can provide a neuromorphic device capable of executing signal processing with high accuracy.

Description

Neuromorphic Device

[0001] The present invention relates to a neuromorphic device using a photonic integrated circuit.

[0002] In recent years, the applications of modern AI (artificial intelligence) and machine learning have been increasing. Along with this, the demand for high-speed information processing and low power consumption is continuously increasing. For this reason, neuromorphic computing has been attracting attention.

[0003] A spiking neural network and reservoir computing are novel neuromorphic architectures. The former includes "spike" neurons. The spike neuron integrates input signals and short spike pulses soon after an energy threshold has been reached, in a similar manner to an actual biological brain.

[0004] The latter is a hardware-friendly architecture. In reservoir computing, inputs are mapped into a higher-dimensional space via a reservoir which is composed of random values. Then, a nonlinear function is applied to the outputs of the reservoir, together with a weighted-summation targeting desired result.

[0005] One of the most promising platforms for neuromorphic computing is neuromorphic photonics. This platform utilizes the remarkably advancing Photonic Integrated Circuit (PIC) and its tremendous parallelization and high-bandwidth capability.

[0006] Several advances in the field of neuromorphic photonics have been achieved, including photonic vector-matrix multiplication (VMM) (For example, non-patent literature 1), various types of optical reservoir and spiking neural network computing architectures (for example, non-patent literature 2), and a nonlinear optical activation function including heterogeneously-integrated spike lasers (for example, non-patent literatures 3 and 4).

[0007] [NPL 1] W.R. Clements, et al., "Optimal design for universal multiport interferometers," Optica, vol. 3, no. 12, pp. 1460-1465 (2016), https: / / doi.org / 10.1364 / OPTICA.3.001460. [NPL 2] D. Owen-Newns, et al., "GHz Rate Neuromorphic Photonic Spiking neural network With a Single Vertical-Cavity Surface-Emitting Laser (VCSEL)," J. Sel. Top. Quantum Electon., vol. 29, no. 2, paper 1500110 (2023), https: / / doi.org / 10.1109 / JSTQE.2022.3205716. [NPL 3] N.P. Diamantopoulos, et al., "Ultrafast Spike Membrane III-V Laser Neuron on Si," in Proc. ECOC’22, 18-22 September 2022, Basel, Switzerland, paper Mo3G.2. [NPL 4] N.P. Diamantopoulos, et al., "All-Optical Spike Membrane III-V Laser on Si," to be presented at CLEO’23, 07-12 May 2023, paper STu4P.3.

[0008] However, in the conventional neuromorphic photonics platform, only a part of the system such as the VMM or spike laser, or assembling of several components in a table-top system has been presented, and a fully functional chip has not been implemented.

[0009] To solve the problem as described above, a neuromorphic device according to the present invention includes an M×1 input array configured to convert an electrical input signal into an optical signal, an M×N processing unit configured to execute photonic vector-matrix multiplication on the optical signal, an N×1 spike unit configured to generate an optical spike signal based on an output signal from the M×N processing unit, an N×1 readout unit configured to weight the optical spike signal, and optical waveguides optically coupled to the M×1 input array, the M×N processing unit, the N×1 spike unit, and the N×1 readout unit, respectively.

[0010] According to the present invention, a neuromorphic device capable of executing signal processing with high accuracy can be provided.

[0011] Fig. 1 is a schematic view showing the arrangement of a neuromorphic device according to the first embodiment of the present invention;Fig. 2 is a schematic sectional side view showing the arrangement of the neuromorphic device according to the first embodiment of the present invention;Fig. 3A is a schematic view showing an example of the arrangement of an input array in the neuromorphic device according to the first embodiment of the present invention;Fig. 3B is a schematic view showing an example of the arrangement of the input array in the neuromorphic device according to the first embodiment of the present invention;Fig. 4 is a schematic sectional front view showing an example of the arrangement of the input array in the neuromorphic device according to the first embodiment of the present invention;Fig. 5 is a schematic view showing the arrangement of a processing unit in the neuromorphic device according to the first embodiment of the present invention;Fig. 6A is a schematic view showing an example of the arrangement of a spike unit in the neuromorphic device according to the first embodiment of the present invention;Fig. 6B is a schematic view showing an example of the arrangement of the spike unit in the neuromorphic device according to the first embodiment of the present invention;Fig. 7 is a view for explaining the operation of the spike unit in the neuromorphic device according to the first embodiment of the present invention;Fig. 8A is a schematic view showing an example of the arrangement of a readout unit in the neuromorphic device according to the first embodiment of the present invention;Fig. 8B is a schematic view showing an example of the arrangement of the readout unit in the neuromorphic device according to the first embodiment of the present invention;Fig. 9A is a view for explaining the effect of the neuromorphic device according to the first embodiment of the present invention;Fig. 9B is a view for explaining the effect of the neuromorphic device according to the first embodiment of the present invention;Fig. 9C is a view for explaining the effect of the neuromorphic device according to the first embodiment of the present invention; andFig. 9D is a view for explaining the effect of the neuromorphic device according to the first embodiment of the present invention.

[0012] <First Embodiment> A neuromorphic device according to the first embodiment of the present invention will be described with reference to Figs. 1 to 9D.

[0013] <Arrangement of Neuromorphic Device> As shown in Fig. 1, a neuromorphic device 10 according to this embodiment includes an M×1 EO input array 11, a processing unit that executes photonic vector-matrix multiplication (to be referred to as VMM hereinafter) 12, an N×1 spike unit 13, and an N×1 readout unit 14.

[0014] In the neuromorphic device 10, optical waveguides 102 connect the EO input array 11, the VMM 12, the spike unit 13, and the readout unit 14.

[0015] M and N are numbers of outputs from the EO input array 11 and the photonic VMM 12, respectively, and arranged in parallel. For example, they are formed like spatial multiplexing using different waveguides.

[0016] The number N of outputs of the spike unit 13 are arranged in serial. For example, they are formed like time-multiplexing within the same waveguide.

[0017] The readout unit 14 outputs the decision result of a classification task.

[0018] M is defined by inputs required for the task, while N defines the nodes of the reservoir. In principle, a larger number of N improves accuracy but increases the system complexity and power consumption.

[0019] Fig. 2 is a view schematically showing the layer arrangement in the light guiding direction in the neuromorphic device.

[0020] In the neuromorphic device, the above-described components are heterogeneously-integrated in a membrane III-V semiconductor structure on SiN / Si.

[0021] In this arrangement, a cladding layer 101 is formed of a low refractive index material based on SiO2, SiOx, SiON, a polymer, or the like.

[0022] One end of an input waveguide 105 is integrated to be optically coupled to the SiN waveguide 102 through means of evanescent coupling via a SiN taper. The other end of the input waveguide 105 is integrated to be optically coupled to an external fiber or another PIC through means of edge coupling. Here, the length of the SiN taper is 200 μm or less, and the width of the SiN taper tip is about 0.1 μm. The material of the core of the input waveguide 105 is SiOx, and the section of the core of the input waveguide 105 is of 3.5 × 3.5 mm2or less.

[0023] The optical waveguides 102 are used for main light propagation in the neuromorphic device, and formed of SiN. The SiN optical waveguide 102 has a width of 0.8 to 1.2 μm and a thickness of 450 to 550 ±20 nm for operation in a communication band such as the O-band or C-band.

[0024] The optical waveguides 102 may be formed of Si. In this case, the optical waveguide 102 has a width of 0.4 to 1.2 μm and a thickness of 220 nm for operation in a communication band such as the O-band or C-band.

[0025] A membrane III-V semiconductor structure 103 including an active layer is heterogeneously integrated, and can be used for an active element such as a semiconductor laser, a modulator, or an SOA (to be described later). The total thickness of the Membrane III-V semiconductor structure 103 is 350 nm or less.

[0026] A metallic heater 104 is used for, for example, the photonic VMM 12. The metallic heater 104 is formed based on tantalum metal or another material, and integrated with a width of 10 to 20 μm above the surface of the SiN optical waveguide 102 via the cladding layer 101.

[0027] The M×1 EO input array 11 converts an input electrical signal into an optical signal.

[0028] As shown in Fig. 3A, an array formed by integrating M directly-modulated lasers (DMLs) 111 is used for the M×1 EO input array 11.

[0029] Alternatively, as shown in Fig. 3B, the M×1 EO input array 11 may use an array of M external modulators 113 such as EA modulators or Mach-Zehnder modulators to convert input parameters from an electrical domain into an optical domain (E / O conversion). In the case of external modulation, the output of an optical source 112 such as a DFB laser is split and fed to all the modulators 113.

[0030] As shown in Fig. 4, the active element such as the laser or modulator has a lateral p-i-n membrane structure including an active core layer 1031 formed of an InGaAsP- or InGaAlAs-MQW, p-type InP 1032, and n-type InP 1033. The width of the active core layer 1031 is 0.8 μm or less. A vertical gap 1034 between the MQW core 1031 and the SiN optical waveguide 102 is 100 ±50 nm.

[0031] As shown in Fig. 5, the M×N photonic VMM 12 is formed by integrating N rectangular arrays of M×M photonic Mach-Zehnder interferometer (MZI) meshes 121. Each MZI mesh 121 has M stages and is composed of a plurality of MZI elements 122.

[0032] The MZI element 122 includes minimum two phase shifters (for example, a phase shifter φ and a phase shifter θ in Fig. 5). Each phase shifter includes the heater 104.

[0033] M optical signals propagated from the outputs of the EO input array 11 are each split into N portions by a splitter 123, and fed to M input ports of each M×M MZI mesh 121.

[0034] Outputs 124 of the M×N photonic VMM 12 are formed by N arrays composed of one output from each of the N MZI meshes 121.

[0035] The N×1 spike unit 13 is implemented either in an optic-electronic-optic (O / E / O) configuration or an all-optical (O / O) configuration as shown in Figs. 6A and 6B.

[0036] In a case of the O / E / O method, as shown in Fig. 6A, the N input signals are time-multiplexed by an optical serializer (parallel-to-serial converter, Ser) 131. The optical serializer 131 includes optical couplers and optical delay lines.

[0037] A photodetector (PD) 132 converts the signals into the electrical domain. The PD 132 is formed of an SiGe material or membrane III-V semiconductor structure, similar to the laser 111 or 112 and the modulator 113.

[0038] In accordance with the PD 132, an electronic integrated circuit (EIC) driver 133 electrically drives a spike laser 134 with an appropriate current for a spiking operation.

[0039] The spike laser 134 is formed by a membrane III-V semiconductor structure, similar to the laser 111 or 112. The spike effect is generated based on an integrated optical feedback method.

[0040] In a case of the all-optical method, as shown in Fig. 6B, the N input signals are time-multiplexed by an optical serializer (parallel-to-serial converter, Ser) 135, as in the case of the O / E / O method.

[0041] The time-multiplexed optical signal from the optical serializer 135 optically excites a spike laser 136. The spike laser 136 is formed by a membrane III-V semiconductor structure, similar to the laser 111, 112, or 134. The spike effect is generated based on integrated optical feedback, as in the spike laser 134.

[0042] In the O / E / O method and the all-optical method, an optical spike is emitted from the spike laser 134 or the spike laser 136 if the electrical or optical input signal exceeds a spiking threshold as shown in Fig. 7.

[0043] More specifically, if the input energy based on an electrical signal 1341 from the PD 132 in the O / E / O method or an optical signal 1342 from the optical serializer 135 in the all-optical method and a feedback 1343 exceeds the spiking threshold, an optical spike 1344 is generated.

[0044] The N×1 readout unit 14 is executed in an electronic circuit-based configuration or a photonic circuit-based configuration as shown in Figs. 8A and 8B, respectively.

[0045] In the electronic circuit-based configuration, as shown in Fig. 8A, the input signal is first converted into the electronic domain by a PD 141. The PD 141 has an arrangement similar to that of the PD 132.

[0046] Then, an electronic de-serializer (serial-to-parallel converter, Des) 142 feeds signals to an array of N adjustable electronic weight elements (weight array) 143.

[0047] The output of the weight array 143 is composed of the summation of N products of inputs × weights.

[0048] A decision unit 144 makes a decision based on the target application and final result (for example, the summation of N products of inputs × weights). For example, the decision unit 114 decides the difference between the target application (data) and the output (data) of the weight array 143, and outputs the output signal of the weight array 143 if the difference is below a predetermined range. That is, training ends.

[0049] If the difference between the target application (data) and the output (data) of the weight array 143 is equal to or larger than the predetermined range, the decision unit 144 changes the weight in the weight array 143, and repeats the process (training) between the weight array 143 and the decision unit 144.

[0050] In the photonic circuit-based configuration, as shown in Fig. 8B, an optical de-serializer (serial-to-parallel converter, Des) 145 feeds signals to an array of N adjustable photonic weight elements (weight array) 146.

[0051] The optical de-serializer 145 includes optical couplers and optical delay lines, similar to the optical serializer 131 or 135.

[0052] The photonic weight array 146 are formed by a heterogeneously integrated array of active elements based on membrane III-V semiconductor structure EAMs or SOAs. The output of the weight array 146 is composed of the summation of N products of inputs × weights.

[0053] A PD 147 converts the signal output from the photonic weight array 146 into the electronic domain. The PD 147 may function as a summation element for the outputs of the N photonic weight array 146.

[0054] As in the electronic circuit-based configuration, a decision unit 148 makes a decision based on the target application and final result.

[0055] In this manner, in the neuromorphic device 10, optical signals converted from electrical signals by the input array 11 are processed by the photonic VMM 12 in parallel, for example, in a spatially-multiplexed manner. In accordance with the output signals of the VMM 12, optical spike signals are generated by the spike unit 13, and output in serial, for example, in a time-multiplexed manner. In the readout unit 14, the time-series signals are weighted, and training is executed by comparing the weighted signal (data) with target data.

[0056] In the neuromorphic device 10, reservoir computing where readout is performed using a spiking neural network as a reservoir can be implemented on one photonic integrated chip.

[0057] <Effects> The effects of the neuromorphic device 10 according to this embodiment will be described with reference to Figs. 9A to 9D.

[0058] We simulate the performance of the neuromorphic device based on a simplified model of a leaky-integrate-and-fire (LIF) neuron model which is a spiking neuron model. This neuron model can simulate the spiking dynamics of a semiconductor laser with optical feedback. The LIF model executes a script using the MATLAB software.

[0059] For simulations, N = 512 nodes are used, and training of the readout weights is executed by the steepest-decent method using a least mean square training algorithm.

[0060] For training, inputs are randomly selected.

[0061] For evaluating the performance of the neuromorphic device, a well-known iris flower classification task is used. This has M = 4 inputs and is composed of 150 iris flower data samples in total. The 4×512 VMM matrix is generated randomly.

[0062] Figs. 9A to 9D show the simulation results.

[0063] Fig. 9A depicts the waveform of one iris flower data sample which outputs from the serializer 135 (or the serializer 131) and inputs to the spike laser 136 (or the spike laser 134). The total length of this waveform corresponds to N = 512 samples.

[0064] Fig. 9B depicts the output (spike) from the spike laser 136 (or the spike laser 134).

[0065] Fig. 9C depicts the mean square error generated by least mean square training algorism. As can be seen, zero MSE is achieved after training the weights with 13,000 or more random iris flower data samples.

[0066] Fig. 9D shows a confusion matrix indicating the performance of the neuromorphic device. The performance of the neuromorphic device is evaluated based on 150 randomly-generated iris flower data sets while using the trained weights. It can be seen from Fig. 9D that 150 iris flower data are correctly classified into groups of 50 data. In this manner, in the neuromorphic device, signal processing with the accuracy of 100% is implemented.

[0067] According to this embodiment, in the neuromorphic device, signal processing with high accuracy is achieved by the arrangement integrated on a single chip.

[0068] The example using the III-V semiconductor heterogeneous integration technology has been shown in the embodiment of the present invention, but the present invention is not limited to this. An InP-based semiconductor monolithic or hybrid integration technology may be used.

[0069] In the embodiment of the present invention, each of the input array and VMM may be formed from a single or several PICs. For example, the input array may be formed from a single PIC, and the 4×4 VMM may be formed from several PICs. The readout unit may be formed from a single or several heterogeneous PIC or EIC chips. For example, the readout unit may be formed from weight elements composed of PICs or EICs, a PD, and a decision unit formed from a single EIC.

[0070] In the embodiment of the present invention, examples of the structure, dimensions, material, and the like of the constituent parts have been described concerning the arrangement of the neuromorphic device, but the present invention is not limited to these. The neuromorphic device need only exhibit its functions and provide effects.

[0071] Note that the present invention is not limited to the above-described embodiments, and it is obvious that a person who has normal knowledge in this field can make many modifications and combinations within the technical scope of the present invention.

[0072] Some or all of the above-described exemplary embodiments can also be described as in the following supplementary notes but are not limited to the followings.

[0073] (Supplementary Note 1) There is provided a neuromorphic device comprising an M×1 input array configured to convert an electrical input signal into an optical signal, an M×N processing unit configured to execute photonic vector-matrix multiplication on the optical signal, an N×1 spike unit configured to generate an optical spike signal based on an output signal from the M×N processing unit, an N×1 readout unit configured to weight the optical spike signal, and optical waveguides optically coupled to the M×1 input array, the M×N processing unit, the N×1 spike unit, and the N×1 readout unit, respectively.

[0074] (Supplementary Note 2) In the neuromorphic device according to supplementary note 1, the M×1 input array outputs the optical signals in parallel, the M×N processing unit outputs the output signals in parallel, and the N×1 spike unit outputs the optical spike signals in serial.

[0075] (Supplementary Note 3) In the neuromorphic device according to supplementary note 1 or 2, the M×N processing unit includes N photonic Mach-Zehnder interferometer meshes in parallel, and in each of the N photonic Mach-Zehnder interferometer meshes, M signals are input in parallel and one signal is output.

[0076] (Supplementary Note 4) In the neuromorphic device according to supplementary note 3, the photonic Mach-Zehnder interferometer mesh includes a plurality of photonic Mach-Zehnder interferometers, and the photonic Mach-Zehnder interferometer includes at least two phase shifters.

[0077] (Supplementary Note 5) In the neuromorphic device according to any one of supplementary notes 1 to 4, the N×1 spike unit includes an optical serializer configured to time-multiplexes a plurality of optical signals input from the M×N processing unit, a photodetector configured to convert the time-multiplexed optical signal into an electrical signal, a driver configured to output a current in accordance with the electrical signal, and a spike laser configured to receive the current and generate the optical spike signal if the current exceeds a threshold.

[0078] (Supplementary Note 6) In the neuromorphic device according to any one of supplementary notes 1 to 4, the N×1 spike unit includes an optical serializer configured to time-multiplexes a plurality of optical signals input from the M×N processing unit, and a spike laser configured to receive the optical signal and generate the optical spike signal if an energy of the optical signal exceeds a threshold.

[0079] (Supplementary Note 7) In the neuromorphic device according to any one of supplementary notes 1 to 6, the N×1 readout unit includes a photodetector configured to convert the optical spike signal into an electrical spike signal, an electronic de-serializer configured to perform serial-to-parallel conversion on the electrical spike signal, a weight array configured to apply a weight on an output signal of the electronic de-serializer, and a decision unit configured to compare the weighted signal with target data.

[0080] (Supplementary Note 8) In the neuromorphic device according to any one of supplementary notes 1 to 6, the N×1 readout unit includes an optical de-serializer configured to perform serial-to-parallel conversion on the optical spike signal, a weight array configured to apply a weight on an output optical signal of the optical de-serializer, a photodetector configured to convert the weighted optical signal into an electrical signal, and a decision unit configured to compare the weighted electrical signal with target data.

[0081] (Supplementary Note 9) In the neuromorphic device according to any one of supplementary notes 1 to 8, the M×1 input array, the M×N processing unit, the N×1 spike unit, the N×1 readout unit, and the optical waveguides are photonic-integrated on a single chip.

[0082] (Supplementary Note 10) In the neuromorphic device according to any one of supplementary notes 1 to 9, the M×1 input array includes a plurality of active elements and the optical waveguides optically coupled to the active elements, and the active element has a membrane structure including a semiconductor active layer and a p-type semiconductor layer and an n-type semiconductor layer arranged on both sides of the semiconductor active layer in a horizontal direction.

[0083] (Supplementary Note 11) In the neuromorphic device according to any one of supplementary notes 1 to 10, an operation is performed in a reservoir computing method in which the M×1 input array, the M×N processing unit, and the N×1 spike unit are used as a reservoir and the readout unit executes training by weighting a signal from the reservoir.

[0084] The present invention is applicable to neuromorphic computing.

[0085] 10...neuromorphic device 102...optical waveguide 11...input array 12...processing unit (VMM) 13...spike unit 14...readout unit

Claims

1. A neuromorphic device comprising: 　　　　an M×1 input array configured to convert an electrical input signal into an optical signal; 　　　　an M×N processing unit configured to execute photonic vector-matrix multiplication on the optical signal; 　　　　an N×1 spike unit configured to generate an optical spike signal based on an output signal from the M×N processing unit; 　　　　an N×1 readout unit configured to weight the optical spike signal; and 　　　　optical waveguides optically coupled to the M×1 input array, the M×N processing unit, the N×1 spike unit, and the N×1 readout unit, respectively.

2. The neuromorphic device according to claim 1, wherein the M×1 input array outputs the optical signals in parallel, the M×N processing unit outputs the output signals in parallel, and the N×1 spike unit outputs the optical spike signals in serial.

3. The neuromorphic device according to claim 1 or 2, wherein 　　　　the M×N processing unit includes N photonic Mach-Zehnder interferometer meshes in parallel, and 　　　　in each of the N photonic Mach-Zehnder interferometer meshes, M signals are input in parallel and one signal is output.

4. The neuromorphic device according to claim 3, wherein 　　　　the photonic Mach-Zehnder interferometer mesh includes a plurality of photonic Mach-Zehnder interferometers, and 　　　　the photonic Mach-Zehnder interferometer includes at least two phase shifters.

5. The neuromorphic device according to claim 1 or 2, wherein 　　　　the N×1 spike unit includes 　　　　an optical serializer configured to time-multiplexes a plurality of optical signals input from the M×N processing unit, 　　　　a photodetector configured to convert the time-multiplexed optical signal into an electrical signal, 　　　　a driver configured to output a current in accordance with the electrical signal, and 　　　　a spike laser configured to receive the current and generate the optical spike signal if the current exceeds a threshold.

6. The neuromorphic device according to claim 1 or 2, wherein 　　　　the N×1 spike unit includes 　　　　an optical serializer configured to time-multiplexes a plurality of optical signals input from the M×N processing unit, and 　　　　a spike laser configured to receive the optical signal and generate the optical spike signal if an energy of the optical signal exceeds a threshold.

7. The neuromorphic device according to claim 1 or 2, wherein 　　　　the N×1 readout unit includes 　　　　a photodetector configured to convert the optical spike signal into an electrical spike signal, 　　　　an electronic de-serializer configured to perform serial-to-parallel conversion on the electrical spike signal, 　　　　a weight array configured to apply a weight on an output signal of the electronic de-serializer, and 　　　　a decision unit configured to compare the weighted signal with target data.

8. The neuromorphic device according to claim 1 or 2, wherein 　　　　the N×1 readout unit includes 　　　　an optical de-serializer configured to perform serial-to-parallel conversion on the optical spike signal, 　　　　a weight array configured to apply a weight on an output optical signal of the optical de-serializer, 　　　　a photodetector configured to convert the weighted optical signal into an electrical signal, and 　　　　a decision unit configured to compare the weighted electrical signal with target data.

Images