A photonic convolutional neural network chip

Abstract

The application provides a photonic convolutional neural network chip, comprising a light pulse transmitter, an input data mapper and a convolution kernel weight mapper, the light pulse transmitter is used for generating m rows of light pulses, each row of light pulses comprises n light pulses of different wavelengths, and the light intensities of all the light pulses are the same; the input data mapper comprises a first array, which is used for mapping input image data and light pulses into a mapping relationship, so that the light pulses carry image data; the convolution kernel weight mapper comprises a second array, which is used for mapping the light pulses carrying image data and convolution kernel weights into a mapping relationship, and performing convolution operation on the image data carried by the light pulses; the second array is connected to a balanced detector, and the balanced detector is connected to a transimpedance amplifier. The application expands the convolution kernel weight value of the convolution layer in the convolutional neural network algorithm from [-1, 1] to the entire real number domain interval on the photonic computing chip, and improves the classification accuracy of the photonic hybrid neural network on the picture data set.

Description

Technical Field

[0001] This invention relates to the field of photonic computing technology, and in particular to a photonic convolutional neural network chip. Background Technology

[0002] Artificial Neural Networks (ANNs) are mathematical models inspired by the simulation of the biological brain, and their computational power is determined by the underlying hardware of the computing device. In recent years, with the rise of ANNs, the types, scale, and number of ANNs have been continuously increasing, demonstrating excellent performance in fields such as image classification, object detection, autonomous driving, and pedestrian detection. However, due to the gradual slowdown of Moore's Law, the rate at which the number and size of transistors in the underlying hardware are increasing cannot keep pace with the increasing scale and number of ANNs. Therefore, a new method is needed to improve the operating speed of convolutional neural networks to meet the real-time requirements of tasks while reducing system power consumption.

[0003] Due to its ultra-wide bandwidth, ultra-high speed, and low loss characteristics, and its natural suitability for matrix operations, light has been increasingly combined with neural networks in recent years. In 2018, Armin of George Washington University proposed the concept of using micro-ring arrays as accelerators for photonic convolutional neural networks to accelerate neural network convolution operations. In 2021, Shaofu Xu implemented a photonic convolutional neural network using optical restoration and micro-ring weight arrays, and validated it on the MNIST dataset. Experimental results showed that the photonic convolutional neural network could achieve an accuracy of 97% and a computation speed of 100 TMAC / s. However, the above research results only allow the photonic convolution weights to take values ​​between [-1, 1], which narrows the range of weight values ​​for the neural network, and the performance on some more complex tasks may not meet expectations. Summary of the Invention

[0004] To address the technical problem of limited weight value range in existing photonic convolutional neural networks, one objective of this invention is to provide a photonic convolutional neural network chip, comprising a light pulse emitter, an input data mapper, and a convolutional kernel weight mapper.

[0005] The light pulse emitter is used to generate m rows of light pulses, wherein each row of light pulses includes n light pulses of different wavelengths, and all light pulses have the same light intensity;

[0006] The input data mapper includes a first array for modulating input image data and light pulses into a mapping relationship, so that the light pulses carry the image data.

[0007] The convolution kernel weight mapper includes a second array, used to modulate the light pulses carrying image data to the convolution kernel weights to form a mapping relationship, and to perform convolution operations on the image data carried by the light pulses;

[0008] The second array is connected to a balanced detector, which is connected to a transimpedance amplifier.

[0009] Preferably, the optical pulse emitter includes a frequency comb and a beam splitter;

[0010] The frequency comb is used to generate m×n light pulses of different wavelengths, and the beam splitter is used to generate m rows of light pulses from the m×n light pulses of different wavelengths, wherein each row of light pulses includes n light pulses of different wavelengths, and all light pulses have the same light intensity.

[0011] Preferably, the transimpedance amplifier is used to connect a 1×1 convolutional kernel to the convolutional layer of the convolutional neural network.

[0012] Preferably, the first array comprises m rows × n columns of all-through microring resonators.

[0013] Preferably, the second array comprises m rows × n columns of bifurcated multiplexed microring resonators.

[0014] This invention provides a photonic convolutional neural network chip that extends the weight values ​​of the convolutional kernels in the convolutional layers of the convolutional neural network algorithm from [-1,1] to the entire real number domain on a photonic computing chip. This expands the parameter value space of the photonic neural network and the range of parameters that can be taken during training, which is beneficial to improving the performance of the photonic convolutional neural network, increasing the classification accuracy of the photonic hybrid neural network for image datasets, and improving the computational speed of the convolutional network while reducing the power consumption of the system.

[0015] The present invention provides a photonic convolutional neural network chip that can improve task accuracy, reduce computation time, and lower system power consumption. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 The schematic diagram illustrates a photonic convolutional neural network chip according to the present invention.

[0018] Figure 2A schematic diagram of the all-through microring resonator of the present invention is shown.

[0019] Figure 3 A schematic diagram of the modulation of input image data and optical pulses by the first array of the present invention is shown.

[0020] Figure 4 A schematic diagram of the bifurcation multiplexed microring resonator of the present invention is shown.

[0021] Figure 5 This diagram illustrates the modulation of light pulses carrying image data with convolution kernel weights using the second array of the present invention. Detailed Implementation

[0022] To make the above and other features and advantages of the present invention clearer, the invention will be further described below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explanation to those skilled in the art and are exemplary only, not restrictive.

[0023] To address the technical problem of high computational load in existing image recognition algorithms, particularly in the detection of power insulator target images, this invention provides a photoelectric hybrid computing chip based on a T-shaped microring resonator.

[0024] like Figure 1 The diagram shown is a schematic of a photonic convolutional neural network chip according to the present invention. According to an embodiment of the present invention, a photonic convolutional neural network chip is provided, including a light pulse emitter 100, an input data mapper 200, and a convolutional kernel weight mapper 300.

[0025] The light pulse emitter 100 is used to generate m rows of light pulses, each row of light pulses including n light pulses of different wavelengths, and all light pulses have the same light intensity.

[0026] According to an embodiment of the present invention, the optical pulse emitter 100 includes a frequency comb 101 and a beam splitter 102. The frequency comb 101 is used to generate m×n optical pulses of different wavelengths, and the beam splitter 102 is used to generate m rows of optical pulses from the m×n optical pulses of different wavelengths, each row of optical pulses including n optical pulses of different wavelengths, and all optical pulses have the same light intensity.

[0027] The generated m rows of light pulses are transmitted to the input data mapper 200 to modulate the image data. The input data mapper 200 corresponds to the input layer of the convolutional neural network algorithm, and the image data is input to the input data mapper 200.

[0028] like Figure 1 As shown, the input data mapper 200 includes a first array for modulating input image data with light pulses to form a mapping relationship, so that the light pulses carry the image data.

[0029] The first array comprises m rows × n columns of all-pass micro-ring resonators, such as Figure 2 The schematic diagram shown is of the all-through microring resonator of the present invention. The all-through microring resonator 201 includes an input terminal and a through terminal.

[0030] According to an embodiment of the present invention, a frequency comb 101 is used to generate n light pulses of different wavelengths, with the wavelength of the light pulses ranging from 1500nm to 1600nm.

[0031] The material of frequency comb 101 uses silicon as a substrate, and the effective refractive index of silicon at different wavelengths is obtained using wavelength:

[0032] Where neff is the effective refractive index of silicon, and λ is the wavelength of the incident light.

[0033] The radius of the microring resonator at different wavelengths is determined using the effective refractive index, and its dimensions are set accordingly. R is the radius of the all-through micro-ring resonator, and p is the resonant order.

[0034] The light pulse emitter 100 generates m rows of light pulses, each row of light pulses including n light pulses of different wavelengths, and all light pulses have the same light intensity. In this embodiment, the intensity of the light pulse is set to 1.

[0035] m rows of optical pulses and image data are input to the input data mapper 200. An external voltage is applied to the m-row × n-column all-pass micro-ring resonator (first array) to change the effective refractive index of the silicon material, thereby changing the phase shift of the light and realizing the change of the output light intensity. The output light intensity is then modulated into normalized image data.

[0036] Specifically, the m-row × n-column all-pass micro-ring resonator (first array) utilizes the wavelength division multiplexing principle to simultaneously process input image data in m dimensions in parallel, thereby improving the system's operating speed.

[0037] The phase shift of the all-through microring resonator 201 changes with the effective refractive index as follows: Where Δφ is the phase shift, R is the all-through micro-ring resonator, and Δn is the change in the effective refractive index of silicon.

[0038] The optical transfer function of the all-pass micro-ring resonator 201 changes with phase shift as follows: Where α is the attenuation coefficient of the incident light, r is the transmission coefficient, and Tn is the transfer function of the incident light.

[0039] like Figure 3The diagram shown illustrates the modulation of input image data and optical pulses by the first array of the present invention. After normalization of the image data, it is segmented and expanded. The data format of each segmented image is (m, ), where m is the dimension of the data, Let be the length and width of the cropped image. Unwrap the cropped image to obtain a vector of dimension m and length n.

[0040] By applying an external voltage to the m-row × n-column all-pass micro-ring resonator (first array) of the input data mapper 200, the effective refractive index of the silicon material is changed, thereby changing the phase shift of the light. The output light intensity is modulated into normalized image data. After normalization processing, the image data is cut and unfolded to achieve a mapping relationship between the input image data and the light pulse, so that the light pulse carries the image data.

[0041] After passing through the input data mapper 200, the light pulse is input to the convolution kernel weight mapper 300 for convolution calculation. The convolution kernel weight mapper 300 corresponds to the convolutional layer of the convolutional neural network algorithm, and the convolution calculation is performed by the convolution kernel weight mapper 300.

[0042] like Figure 1 As shown, the convolution kernel weight mapper 300 includes a second array for modulating a mapping relationship between light pulses carrying image data and convolution kernel weights, and performing convolution operations on the image data carried by the light pulses.

[0043] The second array includes m rows × n columns of bifurcated multiplexed microring resonators, such as Figure 4 The schematic diagram of the bifurcation multiplexing microring resonator of the present invention is shown. Unlike the all-through microring resonator, the bifurcation multiplexing microring resonator 301 includes one input terminal and two output terminals, which are a through terminal and a download terminal, respectively.

[0044] The transfer function of the through-hole is Where α is the attenuation coefficient of the incident light, and r1 and r2 are the transmission coefficients.

[0045] The transfer function on the download side is Where α is the attenuation coefficient of the incident light, and r1 and r2 are the transmission coefficients.

[0046] The light pulses after passing through the input data mapper 200 are input to the convolution kernel weight mapper 300. By modulating the applied voltage to the m-row × n-column bifurcated multiplexed micro-ring resonator (second array) of the convolution kernel weight mapper 300, the m-dimensional convolution kernel weights are mapped onto the output light intensity. This achieves a mapping relationship between the light pulses carrying image data and the convolution kernel weights, and performs convolution operations on the image data carried by the light pulses. Figure 5The diagram shows the second array of the present invention modulating the light pulses carrying image data with the weights of the convolution kernel.

[0047] After being modulated by the m-row × n-column bifurcated multiplexed microring resonator (second array) of the convolution kernel weight mapper 300, the output light intensity at the through end is x·Tp, and the output light intensity at the download end is x·Td. Here, x represents both the input data of the convolution kernel and the input light intensity, that is, x is the light pulse carrying image data input to the convolution kernel weight mapper 300.

[0048] Since the transfer function of the bifurcation multiplexed microring direct end ranges from [0, 1] and the transfer function of the download end also ranges from [0, 1], the light intensity values ​​of the two output ports are always positive. To allow the photon convolution weights to have negative values, this invention connects the m-row × n-column bifurcation multiplexed microring resonator (second array) to the balanced detector 400, thereby achieving a subtraction effect.

[0049] The second array of the present invention is connected to a balanced detector 400. Specifically, a balanced detector 400 is connected to each row of m rows in the second array to convert light intensity into current and simultaneously realize the subtraction operation between the download end and the through end.

[0050] The output of the balanced detector 400 is x·Td-x·Tp, i.e. x·(Td-Tp), where x is the light pulse carrying image data input to the convolution kernel weight mapper 300, which represents both the input data of the convolution kernel and the input light intensity, and (Td-Tp) is the weight value obtained by the convolution kernel after training, which ranges from [-1, 1].

[0051] In order to extend the weight values ​​of the convolutional kernels in the convolutional layers of the convolutional neural network algorithm from [-1,1] to the entire real number range, the present invention connects the balanced detector 400 to a transimpedance amplifier 500. The transimpedance amplifier 500 is used to connect a 1×1 convolutional kernel to the convolutional layer of the convolutional neural network to form a 1×1 convolutional network layer.

[0052] This invention connects a balanced detector 400 to a transimpedance amplifier 500 to enable the insertion of a 1×1 convolutional kernel into the convolutional layer of a convolutional neural network algorithm, forming a 1×1 convolutional network layer, and then trains the convolutional neural network.

[0053] Specifically, the output of the convolutional layer of the convolutional neural network algorithm is x·(Td-Tp). In the 1x1 convolutional network layer, its output becomes x·(Td-Tp)·k, that is, x·((Td-Tp)·k), where k is the weight value of the 1x1 convolutional kernel. The weight value of the 1x1 convolutional kernel plays an amplification role and is automatically learned by the neural network without the need for manual calculation.

[0054] This invention discloses a photonic convolutional neural network chip that enables the extension of convolutional kernel weights to the entire real number domain, increasing the possible value space of neural network weights and effectively improving the accuracy of photonic convolutional neural networks.

[0055] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A photonic convolutional neural network chip, characterized in that, The chip includes a light pulse emitter, an input data mapper, and a convolutional kernel weight mapper. The light pulse emitter is used to generate m rows of light pulses, wherein each row of light pulses includes n light pulses of different wavelengths, and all light pulses have the same light intensity; The input data mapper includes a first array for modulating input image data and light pulses into a mapping relationship, so that the light pulses carry the image data. The convolution kernel weight mapper includes a second array, used to modulate the light pulses carrying image data to the convolution kernel weights to form a mapping relationship, and to perform convolution operations on the image data carried by the light pulses; The second array is connected to a balanced detector, which is connected to a transimpedance amplifier; The optical pulse emitter includes a frequency comb and a beam splitter; The frequency comb is used to generate m×n light pulses of different wavelengths, and the beam splitter is used to generate m rows of light pulses from the m×n light pulses of different wavelengths, wherein each row of light pulses includes n light pulses of different wavelengths, and all light pulses have the same light intensity. The transimpedance amplifier is used to connect a 1×1 convolutional kernel to the convolutional layer of a convolutional neural network; The first array comprises m rows × n columns of all-through micro-ring resonators; The second array includes m rows × n columns of bifurcated multiplexed microring resonators.

Images