Method for implementing variable gain electro-optic nonlinear activation function of optical neural network

Abstract

The application discloses an implementation method of a variable gain electro-optic nonlinear activation function of an optical neural network, and the optical neural network model comprises N full connection neural network layers and a Dropout layer, each full connection neural network layer comprises a 64*64 MZI array and a variable gain electro-optic nonlinear activation function, and the variable gain electro-optic nonlinear activation function comprises a directional coupler, an electro-optic modulation part and an MZI; the variable gain electro-optic nonlinear activation function can change the gain control word of a variable gain amplifier and adjust the bias voltage by using a lookup table, thereby realizing phase modulation on an optical signal, has two degrees of freedom of the gain control word and the bias voltage, and when an optical neural network is artificially built, the nonlinear activation function can be flexibly selected according to different machine learning tasks; and the threshold of the nonlinear activation function can also be adjusted, and the low-power input optical signal is also nonlinearly modulated.

Description

Technical Field

[0001] This invention belongs to the field of optical neural network technology, specifically relating to a method for implementing a variable gain electro-optic nonlinear activation function in an optical neural network. This method is applicable to nonlinear activation functions that use lookup tables to change the amplification factor of a variable gain amplifier and adjust the bias voltage to achieve phase modulation of optical signals. Background Technology

[0002] Optical Neural Networks (ONNs) are computational methods that use optical components and light signals to simulate artificial neural networks. The computation process mainly consists of two parts: optical matrix multiplication and nonlinear activation functions. The nonlinear activation function is a key factor in the powerful learning ability of ONNs, enabling them to learn complex nonlinear relationships between inputs and outputs, thereby improving classification accuracy. However, implementing nonlinear activation functions in large-scale integrated ONNs remains a significant challenge. One of the main difficulties in implementing on-chip optical nonlinear functions is the relatively weak nonlinearity in photonic platforms. Therefore, achieving strong nonlinear activation functions requires very long waveguide interaction lengths and high optical signal power, which increases the area occupied and power consumption of the ONN. Furthermore, during device fabrication, because the components are fixed, the nonlinear response of ONNs is usually constant, limiting the flexibility of the nonlinear activation function and making it difficult for ONNs to generate different responses to adapt to different machine learning tasks. Moreover, after the light signal is attenuated through multiple layers of the neural network, the power may drop below the activation function threshold. Therefore, constructing a reconfigurable optical nonlinear activation function is essential.

[0003] Currently, optical systems for realizing nonlinear activation functions are mainly divided into two categories: optical-electrical-optical (OEO) systems and all-optical systems. OEO systems primarily include two modulation methods: electro-absorption modulator (EAM) modulation and electro-optical modulator (EOM) modulation. Electro-absorption modulators achieve nonlinear modulation by adjusting the signal amplitude, while electro-optical modulators utilize phase shifts in Mach-Zehnder interferometers (MZIs) or micro-ring resonators (MRRs) to achieve nonlinear modulation. Since electro-optical modulators are field-driven rather than carrier-driven, their response speed is faster than that of electro-absorption modulators. Furthermore, electro-optical modulators have lower insertion loss and do not cause additional absorption of the optical signal, resulting in lower overall loss compared to electro-absorption modulators. However, electro-optical modulators have a more complex structure and lower integration density than electro-absorption modulators. In electro-absorption modulator (EAB) systems, nonlinear modulation of light is controlled by varying the bias voltage applied to the EAB, thus adjusting the absorbed light intensity; a higher bias voltage results in less absorbed light. In electro-optic modulator systems, the reconfigurable optical nonlinear activation function method splits the output optical signal of the optical matrix multiplication unit into two parts with different power values ​​via a directional coupler. The smaller part is converted into a current by a photodetector, amplified into a voltage by a transimpedance amplifier (TIA), and finally, a bias voltage is applied to form the modulation voltage. This modulation voltage acts on the modulator, modulating the larger portion of the output optical signal. However, this method has limitations: when the components are fixed, the activation function threshold is also fixed and cannot be adjusted; furthermore, a single-degree-of-freedom function transformation cannot meet the diverse needs of machine learning tasks. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned problems in the prior art by providing a method for implementing a variable gain electro-optic nonlinear activation function for an optical neural network.

[0005] The above-mentioned objectives of the present invention are achieved by the following technical means:

[0006] The implementation method of the variable gain electro-optic nonlinear activation function of optical neural networks includes the following steps:

[0007] Step 1: Obtain the dataset, preprocess the dataset to obtain samples for input into the optical neural network model, and then divide the samples into training set and test set;

[0008] Step 2: Build an optical neural network model. The optical neural network model includes N fully connected neural network layers and one Dropout layer. Each fully connected neural network layer includes a 64×64 MZI array and a variable gain electro-optic nonlinear activation function. The variable gain electro-optic nonlinear activation function includes a directional coupler, an electro-optic modulation part, and an MZI.

[0009] Step 3: Input the training set from Step 1 into the optical neural network model built in Step 2 to train the optical neural network model, so that the loss function of the optical neural network model is minimized.

[0010] Step 4: Test the trained optical neural network model using the test set from Step 1 to obtain the classification accuracy of the trained optical neural network model on the test set.

[0011] As described above, step 1 of the preprocessing of the dataset specifically includes the following steps:

[0012] First, perform a Fourier transform on each image in the dataset to obtain the Fourier transform matrix corresponding to each image. Then, move the zero-frequency component in the Fourier transform matrix to the center of the matrix. Finally, extract 64 data values ​​from the 8×8 region at the center of the matrix as samples input into the optical neural network model.

[0013] As described above, the input-output relationship of N fully connected neural network layers is as follows:

[0014] When i is 1, the input of the i-th fully connected neural network layer is the sample; when i is greater than or equal to 2 and less than or equal to N, the input of the i-th fully connected neural network layer is the output of the (i-1)-th fully connected neural network layer.

[0015] The input x of the i-th fully connected neural network layer i-1 The MZI array input to the i-th fully connected neural network layer is used to obtain the output m of the MZI array of the i-th fully connected neural network layer based on the following formula. i :

[0016] m i =w i ×x i-1 +b i

[0017] In the formula, w i Let b be the MZI weight matrix of the MZI array of the i-th fully connected neural network layer. i is the bias of the MZI array for the i-th fully connected neural network layer.

[0018] As described above, the output m of the MZI array of the i-th fully connected neural network layer i When input to a directional coupler with a coupling coefficient of α for a variable-gain electro-optic nonlinear activation function, the directional coupler will output m. i The optical power is α|m i | 2 The optical signal is separated and input into the electro-optic modulation part of the variable gain electro-optic nonlinear activation function, which will output m i The optical power is (1-α)|m i | 2 The optical signal is input into the MZI in the variable gain electro-optic nonlinear activation function for phase modulation;

[0019] The electro-optic modulation section includes a photodetector, a transimpedance amplifier, a variable gain amplifier, and a lookup table;

[0020] The electro-optic modulation section obtains the modulation voltage V based on the following formula. G :

[0021] V G =αKDGR|m i | 2 +V b

[0022] In the formula, K is the gain control coefficient of the variable gain amplifier, D is the gain control word of the variable gain amplifier, G is the gain of the transimpedance amplifier, R is the responsivity of the photodetector, and V... b This is the bias voltage for the electro-optic modulation section;

[0023] The gain control word D of the variable gain amplifier is given by the data at the corresponding position in the lookup table.

[0024] As described above, the MZI of the variable gain electro-optic nonlinear activation function includes an upper arm waveguide, a lower arm waveguide, and a thermal phase shifter;

[0025] The modulation voltage V output by the electro-optic modulation section G A thermal phase shifter, applied to an MZI with a variable gain electro-optic nonlinear activation function, generates a phase shift ΔΦ based on the following formula:

[0026]

[0027] In the formula, V π It is the voltage required for the thermal phase shifter to generate a phase shift of π amplitude;

[0028] A thermal phase shifter is loaded in the upper arm waveguide, and the optical power is (1-α)|m i | 2The optical signal is input to the upper arm waveguide, and the input to the lower arm waveguide is 0;

[0029] The MZI of the variable gain electro-optic nonlinear activation function is based on the following formula for optical power (1-α)|m i | 2 The optical signal is phase-modulated to obtain the output f(m) of the lower arm waveguide. i ):

[0030]

[0031] In the formula, j is the imaginary unit;

[0032] g Φ The phase gain parameter is calculated based on the following formula:

[0033]

[0034] Φ b The bias voltage gain parameter is calculated based on the following formula:

[0035]

[0036] Threshold P of nonlinear activation function th Calculated based on the following formula:

[0037]

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) The variable gain electro-optic nonlinear activation function of the present invention can use a lookup table to change the gain control word of the variable gain amplifier and adjust the bias voltage, thereby realizing phase modulation of the optical signal. Compared with the existing electro-optic nonlinear activation function, the variable gain electro-optic nonlinear activation function of the present invention has two degrees of freedom: gain control word and bias voltage. When artificially building an optical neural network, the nonlinear activation function can be flexibly selected according to different machine learning tasks.

[0040] (2) This invention can adjust the threshold of the nonlinear activation function by changing the gain control word of the variable gain amplifier. The threshold is reflected in the modulation voltage V. G The change in allows the nonlinear activation function to perform nonlinear modulation on low-power input optical signals. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of preprocessing the MNIST handwritten digit image dataset according to Embodiment 1 of the present invention;

[0042] Figure 2This is a schematic diagram of the framework of the optical neural network model of Embodiment 1 of the present invention;

[0043] Figure 3 This is a schematic diagram of the variable gain electro-optic nonlinear activation function of Embodiment 1 of the present invention;

[0044] Figure 4 This is Embodiment 1 of the present invention, in the variable gain electro-optic nonlinear activation function Φ b With α, K, G, and R fixed, change the nonlinear activation function image of the gain control word D;

[0045] Figure 5 This is Embodiment 1 of the present invention, in the variable gain electro-optic nonlinear activation function Φ b The relationship between the gain control word D and the threshold of the nonlinear activation function under the condition that α, K, G, and R are fixed;

[0046] Figure 6 This is a loss graph of the classification results on the test set in Embodiment 1 of the present invention;

[0047] Figure 7 This is an accuracy graph of the classification results on the test set in Embodiment 1 of the present invention;

[0048] Figure 8 This is a heatmap of the classification results on the test set in Embodiment 1 of the present invention. Detailed Implementation

[0049] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to embodiments. The embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0050] Example 1:

[0051] The implementation method of the variable gain electro-optic nonlinear activation function of optical neural networks includes the following steps:

[0052] Step 1: Obtain the dataset, preprocess the dataset to obtain samples for input into the optical neural network model, and then divide the samples into training and testing sets. This includes the following steps:

[0053] In this embodiment, the dataset is the MNIST handwritten digit image dataset, which is a public dataset used for image classification tasks in machine learning, including 70,000 handwritten digit images with a resolution of 28×28 pixels.

[0054] Step 1.1: Obtain the MNIST handwritten digit image dataset and preprocess the images in the MNIST handwritten digit image dataset: First, perform a Fourier transform on each image in the MNIST handwritten digit image dataset to obtain the Fourier transform matrix corresponding to each image. Then, move the zero-frequency component in the Fourier transform matrix to the center of the matrix. Finally, extract 64 data values ​​from the 8×8 region at the center of the matrix as samples input into the optical neural network model.

[0055] Step 1.2: Generate training and test sets based on the samples. In this embodiment, the training set contains 60,000 samples and the test set contains 10,000 samples.

[0056] Step 2: Construct the optical neural network model. The optical neural network model consists of N fully connected neural network layers and one Dropout layer. Each fully connected neural network layer includes a 64×64 MZI array and a variable gain electro-optic nonlinear activation function. The variable gain electro-optic nonlinear activation function includes a directional coupler, an electro-optic modulation part, and the MZI. The Dropout layer filters the output value of the last fully connected neural network layer to ten values ​​to match the number of categories in the MNIST handwritten digit image dataset.

[0057] The optical neural network model constructed in this embodiment includes three fully connected neural network layers and one Dropout layer.

[0058] The input-output relationship of N fully connected neural network layers is as follows:

[0059] Let i be the index of the fully connected neural network layer. When i is 1, the input of the i-th fully connected neural network layer is the sample. When i is greater than or equal to 2 and less than or equal to N, the input of the i-th fully connected neural network layer is the output of the (i-1)-th fully connected neural network layer.

[0060] The i-th fully connected neural network layer takes input x as input. i-1 The specific calculation process is as follows:

[0061] First, the input x of the i-th fully connected neural network layer i-1 The MZI array input to the i-th fully connected neural network layer is used to obtain the output m of the MZI array of the i-th fully connected neural network layer based on the following formula. i :

[0062] m i =w i ×x i-1 +b i (1)

[0063] In the formula, w iLet b be the MZI weight matrix of the MZI array of the i-th fully connected neural network layer. i is the bias of the MZI array for the i-th fully connected neural network layer.

[0064] Then, the output m of the MZI array of the i-th fully connected neural network layer i When input to a directional coupler with a coupling coefficient of α for a variable-gain electro-optic nonlinear activation function, the directional coupler will output m. i The optical power is α|m i | 2 The optical signal is separated and input into the electro-optic modulation part of the variable gain electro-optic nonlinear activation function, which will output m i The optical power is (1-α)|m i | 2 The optical signal is input into the MZI in the variable gain electro-optic nonlinear activation function for phase modulation, such as... Figure 3 As shown.

[0065] The specific calculation process of the electro-optic modulation part is as follows:

[0066] The electro-optic modulation section includes a photodetector, a transimpedance amplifier (TIA), a variable gain amplifier (VGA), and a lookup table (LUT).

[0067] The photodetector in the electro-optic modulation section receives an input optical power of α|m i | 2 The optical signal is converted into current I using the following formula. pd :

[0068] I pd =αR|m i | 2 (2)

[0069] In the formula, R is the photodetector responsivity of the photodetector.

[0070] Current I pd The voltage V is obtained after passing through a transimpedance amplifier (TIA) with a gain of G. in :

[0071] V in =αGR|m i | 2 (3)

[0072] Voltage V in The voltage V is then obtained after passing through a variable gain amplifier (VGA). out :

[0073] V out =αKDGR|mi | 2 (4)

[0074] In the formula, K is the gain control coefficient of the variable gain amplifier, and D is the gain control word of the variable gain amplifier. The gain control word D of the variable gain amplifier is given by the data at the corresponding position in the lookup table (LUT).

[0075] Voltage V out The bias voltage V of the superimposed electro-optic modulation section b The modulation voltage V is obtained from the final output of the electro-optic modulation section. G :

[0076] V G =αKDGR|m i | 2 +V b (5)

[0077] The MZI with variable gain electro-optic nonlinear activation function includes an upper arm waveguide, a lower arm waveguide, and a thermal phase shifter. The thermal phase shifter is loaded on the upper arm waveguide, and the input to the upper arm waveguide is the optical power output of the directional coupler, which is (1-α)|m i | 2 The optical signal; the input to the lower arm waveguide is 0.

[0078] Modulation voltage V G The thermal phase shifter, operating on the MZI in the variable gain electro-optic nonlinear activation function, generates a phase shift ΔΦ based on the following formula:

[0079]

[0080] In the formula, V π This represents the voltage required for the thermal phase shifter to generate a phase shift of π amplitude.

[0081] The optical power is (1-α)|m i | 2 The specific process of phase modulation of optical signals in the MZI of the variable gain electro-optic nonlinear activation function includes the following steps:

[0082]

[0083] In the formula, E1 is the output of the upper arm waveguide of MZI in the variable gain electro-optic nonlinear activation function, j is the imaginary unit, and f(m i ) represents the output of the lower arm waveguide of the MZI in the variable gain electro-optic nonlinear activation function.

[0084] According to formula (7):

[0085]

[0086] Substituting formula (6) into formula (8), we obtain the formula for the output of the lower arm waveguide of MZI in the variable gain electro-optic nonlinear activation function (i.e., the final output of MZI in the variable gain electro-optic nonlinear activation function):

[0087]

[0088] In the formula, g Φ For phase gain parameters, Φ b These are the bias voltage gain parameters.

[0089] Threshold P of nonlinear activation function th Based on the following formula:

[0090]

[0091] In the formula, Δ Φ Indicates that under a given Φ b Under these conditions, the phase shift required to produce a 50% power transfer change in the output optical signal relative to the input optical signal when there is zero input.

[0092] Step 3: Input the preprocessed training set from Step 1 into the optical neural network model built in Step 2 to train the optical neural network model, minimizing the loss function of the optical neural network model to obtain the best classification accuracy.

[0093] In this embodiment, the hyperparameters of the optical neural network model are set as follows: the number of iterations is set to 50, the batch size is set to 500, and the initial learning rate is set to 0.001. The parameters of the variable gain electro-optic nonlinear activation function are set as follows: α = 0.01, K = 0.01, D = 40, V b =10(V), V π =0.1(V), i.e., Φ b =π, the loss function is the MSE loss function, and the optimizer is the Adam optimizer.

[0094] Step 4: Test the trained optical neural network model using the preprocessed test set from Step 1 to obtain the classification accuracy of the trained optical neural network model on the test set.

[0095] Figure 4 In the variable gain electro-optic nonlinear activation function Φ b With α, K, G, and R fixed, changing the nonlinear activation function image of the gain control word D reveals that the variable gain electro-optic nonlinear activation function has a high degree of nonlinearity.

[0096] Figure 5 In the variable gain electro-optic nonlinear activation function Φ b The relationship between the gain control word D and the threshold of the nonlinear activation function is shown under the condition that α, K, G, and R are fixed. It can be seen that by changing the gain control word D, the threshold of the nonlinear activation function can be controlled.

[0097] Figure 6 The loss graph shows the classification results on the test set. Compared with the optical neural network with a linear activation function, the optical neural network with a variable gain electro-optic nonlinear activation function has a significantly smaller loss.

[0098] Figure 7 This is a graph showing the accuracy of classification results on the test set. Figure 8 This is a heatmap of the classification results on the test set. Combined with... Figure 7 and Figure 8 It can be seen that the final verification accuracy of the optical neural network with the variable gain electro-optic nonlinear activation function is 95.37%, which is significantly higher than that of the linear activation function.

[0099] Experimental results show that the present invention can achieve the gain control word D and bias voltage V of the nonlinear activation function. b The ability to control two degrees of freedom allows for adjustment of the threshold of the nonlinear activation function. When artificially constructing optical neural networks, the nonlinear activation function can be flexibly selected according to different machine learning tasks.

[0100] It should be noted that the embodiments described in this invention are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains can make various modifications or additions to the described embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for implementing a variable-gain electro-optic nonlinear activation function in an optical neural network, characterized in that, Includes the following steps: Step 1: Obtain the dataset, preprocess the dataset to obtain samples for input into the optical neural network model, and then divide the samples into training set and test set; Step 2: Build an optical neural network model. The optical neural network model includes N fully connected neural network layers and one Dropout layer. Each fully connected neural network layer includes a 64×64 MZI array and a variable gain electro-optic nonlinear activation function. The variable gain electro-optic nonlinear activation function includes a directional coupler, an electro-optic modulation part, and an MZI. Step 3: Input the training set from Step 1 into the optical neural network model built in Step 2 to train the optical neural network model, so that the loss function of the optical neural network model is minimized. Step 4: Test the trained optical neural network model using the test set from Step 1 to obtain the classification accuracy of the trained optical neural network model on the test set; The specific calculation process of the electro-optic modulation part is as follows: The electro-optic modulation section includes a photodetector, a transimpedance amplifier, a variable gain amplifier, and a lookup table; The photodetector in the electro-optic modulation section receives the input optical power as... The optical signal is converted into current using the following formula. : In the formula, The photodetector responsivity; Current After gain The transimpedance amplifier obtains voltage : Voltage The voltage is then obtained through a variable gain amplifier. : In the formula, This refers to the gain control coefficient of the variable gain amplifier. This refers to the gain control word of a variable gain amplifier. The data is given by the corresponding position in the lookup table; Voltage bias voltage of superimposed electro-optic modulation section The modulation voltage of the final output of the electro-optic modulation section is obtained. : 。 2. The method for implementing the variable gain electro-optic nonlinear activation function of the optical neural network according to claim 1, characterized in that, The preprocessing of the dataset in step 1 is specifically... The process includes the following: First, perform a Fourier transform on each image in the dataset to obtain the Fourier transform matrix corresponding to each image. Then, move the zero-frequency component in the Fourier transform matrix to the center of the matrix. Finally, extract 64 data values ​​from the 8×8 region at the center of the matrix as samples input into the optical neural network model.

3. The method for implementing the variable gain electro-optic nonlinear activation function of the optical neural network according to claim 2, characterized in that, The input-output relationship of the N fully connected neural network layers is as follows: when When it is 1, the first The input to each fully connected neural network layer is a sample; when When the number of digits is greater than or equal to 2 and less than or equal to N, the first digit is... The input to the nth fully connected neural network layer is the... The output of a fully connected neural network layer; No. Input to a fully connected neural network layer Enter the number The MZI array with the nth fully connected neural network layer, the nth The MZI array of the nth fully connected neural network layer is obtained based on the following formula. The output of an MZI array with one fully connected neural network layer : In the formula, For the first The MZI weight matrix of an MZI array of fully connected neural network layers. For the first The bias of an MZI array of fully connected neural network layers.

4. The method for implementing the variable gain electro-optic nonlinear activation function of the optical neural network according to claim 3, characterized in that, The first The output of an MZI array with one fully connected neural network layer The coupling coefficient input to the variable gain electro-optic nonlinear activation function is: In a directional coupler, a directional coupler with a variable gain electro-optic nonlinear activation function will output... Medium optical power is The optical signal is separated and input into the electro-optic modulation part of the variable gain electro-optic nonlinear activation function, and the output is... Medium optical power is The optical signal is input into the MZI in the variable gain electro-optic nonlinear activation function for phase modulation.

5. The method for implementing the variable gain electro-optic nonlinear activation function of the optical neural network according to claim 4, characterized in that, The MZI of the variable gain electro-optic nonlinear activation function includes an upper arm waveguide, a lower arm waveguide, and a thermal phase shifter; The modulation voltage output by the electro-optic modulation section A thermal phase shifter, applied to an MZI with a variable gain electro-optic nonlinear activation function, generates a phase shift based on the following formula. : In the formula, It is generated by the heat-shifting phase converter The voltage required for the phase shift of the amplitude; A thermal phase shifter is loaded in the upper arm waveguide, with an optical power of The optical signal is input to the upper arm waveguide, and the input to the lower arm waveguide is 0; The MZI of the variable gain electro-optic nonlinear activation function is based on the following formula for optical power. The optical signal is phase-modulated to obtain the output of the lower arm waveguide. : In the formula, The imaginary unit; The phase gain parameter is calculated based on the following formula: The bias voltage gain parameter is calculated based on the following formula: Threshold of nonlinear activation function Calculated based on the following formula: 。

Images