A photon residual deep network chip and its construction method

Abstract

The present invention discloses a photon residual deep network chip and a construction method thereof, belonging to the field of integrated photonic devices. By connecting on-chip photonic devices, photon residual blocks are cascaded to construct a photon residual deep network chip. The photon residual deep network corresponding to the function of the photon residual deep network chip is trained to obtain the parameters of each neuron in the hidden layer of each residual block of the photon residual deep network. The parameters are loaded into the modulator in the hidden layer of each photon residual block of the photon residual deep network chip, thereby realizing modulation of the input signal by the photon residual deep network chip. By mapping the input layer parameters of each residual block to the output layer with identity, the gradient information can be stably transmitted through the identity mapping during the training process, which can solve the gradient vanishing and explosion problems of the residual neural network.

Description

Technical Field

[0001] The present invention belongs to the field of integrated photonic devices, and more specifically, relates to a photonic residual deep network chip and a construction method thereof. Background Art

[0002] In the fields of information technology and data processing, the search for more efficient and faster computing methods has become a pressing task as the demand for data processing continues to grow. Photonic deep networks are an emerging technology that has attracted considerable attention for their ultra-high speed and low power consumption. However, increasing the scale and computational accuracy of photonic deep networks remains a major challenge in this field.

[0003] Existing photonic deep network chips are mostly based on traditional neural network structures, such as fully connected neural networks and convolutional neural networks. While these structures are feasible, they are difficult to scale up (in terms of the number of layers and neurons), leading to bottlenecks in computational accuracy. This is because traditional neural network structures, when the number of layers and neurons increases, will experience gradient vanishing and exploding problems during training, making it impossible for the network to obtain accurate parameters and thus encountering computational accuracy bottlenecks. Summary of the Invention

[0004] In response to the above defects or improvement needs of the existing technology, the present invention provides a photonic residual deep network chip and a construction method thereof, which can solve the gradient vanishing and explosion problems of residual neural networks.

[0005] To achieve the above-mentioned object, according to a first aspect of the present invention, there is provided a photon residual deep network chip, comprising: a plurality of sequentially connected photon residual blocks;

[0006] Wherein, each of the photon residual blocks includes:

[0007] The input layer includes a vertical coupler, an optical waveguide, N first modulators, and a first power splitter connected in sequence. The optical signal is input into the optical waveguide via the vertical coupler. The N first modulators load the N input signals onto the N wavelengths of the optical signal in a one-to-one correspondence to obtain a target signal. The target signal is split into two by the first power splitter, namely the first and second target signals.

[0008] The identity mapping layer includes a wavelength division multiplexer and N first photodetectors; the first target signal is divided into N signals by the wavelength division multiplexer and detected one by one by the N first photodetectors;

[0009] The hidden layer includes a second power splitter, N groups of second modulators, and N second photodetectors; the second target signal is divided into N equal parts by the second power splitter, modulated one-to-one by the N groups of second modulators, and detected one-to-one by the N second photodetectors;

[0010] an output layer, comprising an adder for adding the signals detected by the N first photodetectors and the signals detected by the N second photodetectors in a one-to-one correspondence to obtain an output signal;

[0011] Among them, each group of second modulators includes N second modulators, and its modulation parameters are determined according to the neuron weights of the hidden layer of each residual block in the trained residual deep network; the residual deep network includes multiple residual blocks connected in sequence, and each residual block includes an input layer, an identity mapping layer, a hidden layer and an output layer. The function of each layer corresponds one-to-one to the function of the input layer, identity mapping layer, hidden layer and output layer of the photon residual block. The hidden layer of each residual block includes N groups of neurons, each group of neurons includes N neurons, and the weight of each neuron is unknown. The other layers include N neurons and the weight of each neuron is a fixed value, N≥1.

[0012] According to a second aspect of the present invention, a method for constructing a photon residual deep network chip is provided, comprising:

[0013] S1, connecting multiple photon residual blocks in sequence; wherein each of the photon residual blocks includes:

[0014] The input layer includes a vertical coupler, an optical waveguide, N first modulators, and a first power splitter connected in sequence. The optical signal is input into the optical waveguide via the vertical coupler. The N first modulators load the N input signals one-to-one at the N wavelengths of the optical signal to obtain a target signal. The target signal is split into two by the power splitter, namely the first and second target signals.

[0015] The identity mapping layer includes a wavelength division multiplexer and N first photodetectors; the first target signal is divided into N signals by the wavelength division multiplexer and detected one by one by the N first photodetectors;

[0016] The hidden layer includes a second power splitter, N groups of second modulators, and N second photodetectors; the second target signal is divided into N equal parts by the second power splitter, modulated one-to-one by the N groups of second modulators, and detected one-to-one by the N second photodetectors; each group of second modulators includes N second modulators;

[0017] an output layer, comprising an adder for adding the signals detected by the N first photodetectors and the signals detected by the N second photodetectors in a one-to-one correspondence to obtain an output signal;

[0018] S2, training the residual deep network, and determining the modulation parameters of each second modulator in each group of second modulators according to the neuron weights of the hidden layer of each residual block in the trained residual deep network;

[0019] Among them, the residual deep network includes multiple residual blocks connected in sequence, each residual block includes an input layer, an identity mapping layer, a hidden layer and an output layer, and the function of each layer corresponds one-to-one to the function of the input layer, identity mapping layer, hidden layer and output layer of the photon residual block. The hidden layer of each residual block includes N groups of neurons, each group of neurons includes N neurons, and the weight of each neuron is unknown. The other layers include N neurons and the weight of each neuron is a fixed value, N≥1.

[0020] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0021] By connecting on-chip photonic devices, cascading photon residual blocks to construct a photon residual deep network chip, and then training a residual deep network corresponding to the functions of the photon residual deep network chip to obtain the parameters of each neuron in the hidden layer of each residual block of the photon residual deep network, the parameters are loaded into the modulators in the hidden layer of each photon residual block of the photon residual deep network chip, enabling the photon residual deep network chip to modulate the input signal. By mapping the input layer parameters of each residual block to the output layer with identity, the gradient information can be stably transmitted during the training process through the identity mapping, which can solve the gradient vanishing and explosion problems of residual neural networks.

[0022] Specifically, in the Residual Neural Network (ResNet), the main idea behind each residual block is to introduce a "shortcut connection," also called "identity mapping." For a residual block, the input x is directly added to the output, forming y = F(x) + x, where F(x) is a series of operations within the residual block, including convolution and activation functions. In traditional networks, the transitions between layers involve entirely new mappings, which can cause gradients to become unstable as the number of layers increases. However, in the residual block provided by the present invention, the input x is directly added to the output y. This means that even if F(x) is very small or very large, the input x will still directly affect the output via the shortcut connection, thereby ensuring stable gradient transmission. During backpropagation, the gradient not only passes through the parameters of F(x), but is also directly transmitted via the identity mapping. This means that even if the gradient of F(x) vanishes or explodes, the gradient of the identity mapping portion remains stable. Through this mechanism, the gradient in the residual network can be transmitted more stably during backpropagation. This ensures that even if the network is very deep, the gradient will not easily disappear or explode. Therefore, ResNet can train very deep networks (such as 100 layers or even deeper), and these deep networks can significantly improve the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of the photon residual deep network chip structure provided by an embodiment of the present invention.

[0024] Figure 2 A schematic diagram of the optical residual network structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0026] With the development of neural network structure, the gradient vanishing and explosion problems can be solved by utilizing its unique connection method, which makes it possible to significantly increase the scale and computational accuracy of photonic deep network chips.

[0027] Based on this, the embodiment of the present invention provides a photon residual deep network chip, such as Figure 1 Shown, including:

[0028] Multiple photon residual blocks connected sequentially;

[0029] Wherein, each of the photon residual blocks includes:

[0030] The input layer includes a vertical coupler, an optical waveguide, N first modulators, and a first power splitter connected in sequence. The optical signal is input into the optical waveguide via the vertical coupler. The N first modulators load the N input signals onto the N wavelengths of the optical signal in a one-to-one correspondence to obtain a target signal. The target signal is split into two by the first power splitter, namely the first and second target signals.

[0031] The identity mapping layer includes a wavelength division multiplexer and N first photodetectors, wherein the N photodetectors are all connected to the wavelength division multiplexer; the first target signal is divided into N signals by the wavelength division multiplexer and detected one by one by the N first photodetectors;

[0032] The hidden layer includes a second power splitter, N groups of second modulators, and N second photodetectors; the second target signal is divided into N equal parts by the second power splitter, modulated one-to-one by the N groups of second modulators, and detected one-to-one by the N second photodetectors;

[0033] The output layer includes an adder for adding the signals detected by the N first photodetectors and the signals detected by the N second photodetectors in a one-to-one correspondence to obtain an output signal as the input signal of the next-level photon residual block. It can be understood that the output signal of the last-level photon residual block is the output signal of the photon residual deep network chip;

[0034] Among them, each group of second modulators includes N second modulators, and its modulation parameters are determined according to the neuron weights of the hidden layer of each residual block in the trained residual deep network; the residual deep network includes multiple residual blocks connected in sequence, and each residual block includes an input layer, an identity mapping layer, a hidden layer and an output layer. The function of each layer corresponds one-to-one to the function of the input layer, identity mapping layer, hidden layer and output layer of the photon residual block. The hidden layer of each residual block includes N groups of neurons, each group of neurons includes N neurons, and the weight of each neuron is unknown. The other layers include N neurons and the weight of each neuron is a fixed value, N≥1.

[0035] Specifically, multiple photon residual blocks are arranged in sequence to form a photon residual deep network chip.

[0036] Each optical residual block includes:

[0037] The input layer includes a vertical coupler, an optical waveguide, N first modulators, and a first power splitter connected in sequence. Light is input into the optical waveguide through the vertical coupler and loaded into the N wavelengths [λ1, λ2…λn] by the microring modulator. The first power splitter then splits the N input signals into two, the first and second target signals.

[0038] The identity mapping layer includes a wavelength division multiplexer (WDM) and N first PD detectors, each of which is connected to the WDM. A first target signal passes through the WDM, splitting the N input signals into N signal paths for transmission. Each of the N signals is detected by the first PDs, and the signals detected by the N first PDs are the input signals [x1, x2…xn].

[0039] The hidden layer includes a second power splitter and N groups of second modulators, each group of second modulators including N second modulators connected in sequence, and each group of second modulators is connected to the second power splitter. The second target signal (i.e., the N input signals) is divided into N equal parts by the second power splitter, and each of the N groups of second modulators performs signal processing on the second target signal to obtain intermediate signals [F(x1), F(x2)…F(xn)].

[0040] The output layer includes an adder, which is used to add the input signal and the intermediate signal through circuit connection to obtain the output signal [F(x1)+x1, F(x2)+x2…F(x3)+F(xn)].

[0041] The modulation parameters of each second modulator in the N groups of second modulators in the hidden layer are obtained as follows:

[0042] Design a residual deep network such as Figure 2 As shown, there are multiple residual blocks connected in sequence, each of which includes:

[0043] Input layer: has N neurons and is used to load training samples (input data) in the training set.

[0044] Identity mapping layer: It has N neurons, which performs identity mapping on the data of the input layer and passes it directly to the output layer, retaining the original input information.

[0045] Hidden layer: has N groups of neurons, each group of neurons includes N neurons, that is, the hidden layer has N×N neurons, which are used to perform nonlinear transformation and feature extraction on the input data to capture the complex patterns of the input data.

[0046] Output layer: It has N neurons and is responsible for adding the direct mapping output of the identity mapping layer and the transformed output of the hidden layer to generate the final output data of the residual block.

[0047] It can be understood that the functions of the input layer, identity mapping layer, hidden layer, and output layer of each residual block are the same as those of the input layer, identity mapping layer, hidden layer, and output layer of each optical residual block. Accordingly, each residual block corresponds one-to-one to each photon residual block, and the N groups of second modulators in the hidden layer of each residual block correspond one-to-one to the N groups of neurons in the hidden layer of each photon residual block. Each second modulator in the N groups of second modulators corresponds one-to-one to each neuron in the N groups of neurons and has the same function. In the residual deep network, the neuron weight information of the hidden layer is unknown, and the neuron weight information of other layers is fixed. Among them, the neuron weight information of the input layer and the identity mapping layer is equal to the normalized value of the input signal, and the neuron weight information of the output layer is the weight information of the neurons in the last layer of the hidden layer plus the neuron weight information of the identity mapping layer. By training the residual deep network, the neuron weight information of its hidden layer can be obtained, thereby determining the modulation parameters of each group of second modulators in the hidden layer of the optical residual block (that is, the modulation parameters of each second modulator in each group of second modulators).

[0048] Preferably, the modulator is a micro-ring modulator or an MZI modulator, and the modulation parameter is light transmittance.

[0049] Preferably, the training data set of the residual deep network includes sample data and label data of the target task that needs to be implemented by the photonic residual deep network chip.

[0050] Preferably, the training data set of the residual deep network includes normalized sample data and label data. That is, before training the residual deep network, the method further includes: normalizing the training data set.

[0051] Preferably, the neuron weights of the input layer, identity mapping layer, hidden layer and output layer of each residual network are all in the interval [0,1].

[0052] Specifically, since the light transmittance of the second modulator is usually in the range of [0,1], in order to more conveniently determine the modulation parameters of each group of second modulators in the hidden layer of the optical residual block according to the neuron weight information of the hidden layer of the residual deep network, all parameters in the residual deep network are restricted to the interval of [0,1].

[0053] Accordingly, the sample data and corresponding labels in the training data set are transformed into one-dimensional vectors. The data set includes sample data [x1, x2…xn] and label data [y1, y2…yn], and the elements in the vector are normalized to the [0,1] interval to obtain the normalized input signal [x1`, x2…xn`] and target signal [y1`, y2`…yn`], and the residual deep network is trained to obtain the weight information of each neuron in the hidden layer.

[0054] Any existing loss function can be used during training. Preferably, mean square error is used for network training to obtain the weight information of each neuron in the hidden layer.

[0055] The weight information of each neuron in the hidden layer of each residual block of the photon residual deep network is loaded one-to-one into each second modulator in the hidden layer of each photon residual block of the photon residual deep network chip, so that the light transmittance of each second modulator is equal to the weight information of each neuron, thereby realizing the modulation of the input signal by the hidden layer.

[0056] It is understandable that if the training data set is not normalized, and the neuron weights of the input layer, identity mapping layer, hidden layer, and output layer of each residual network are not limited to the interval [0,1], then the neuron weights of the hidden layer of each residual network can be normalized after training is completed. Accordingly, the normalized weight information of each neuron in the hidden layer of each residual block of the photon residual deep network is loaded one-to-one into each second modulator in the hidden layer of each photon residual block of the photon residual deep network chip, even if the light transmittance of each second modulator is one-to-one equal to the normalized weight information of each neuron.

[0057] The photon residual deep network chip provided by the present invention is further illustrated below with a specific example.

[0058] In this example, the photon residual deep network chip includes 3 photon residual blocks, N=2.

[0059] The on-chip photon residual block consists of three sequentially arranged photon residual blocks, forming a photon residual deep network chip;

[0060] Each photon residual block includes:

[0061] Input layer: This layer includes a vertical coupler, an optical waveguide, two first modulators, and a first power splitter connected in sequence. Light enters the optical waveguide through the vertical coupler and is loaded into the waveguide by the first modulator, where it is loaded into two input signals [x1, x2] at two wavelengths [λ1, λ2]. The first power splitter then splits these signals into two, yielding the first and second signals.

[0062] Identity mapping layer: includes a wavelength division multiplexer and two first photodetectors; the wavelength division multiplexer divides the first signal into two signal transmission paths, and the two PD detectors respectively detect the input signals [x1, x2];

[0063] Hidden layer: includes a second power splitter, two groups of second modulators and two second photodetectors. Each group of second modulators includes two second modulators connected in sequence. When light passes through the power splitter, the second signal is divided into two groups [x1, x2]. The first group [x1, x2] is modulated on λ1 and λ2 by two second modulators connected in sequence in the first group of second modulators to obtain [F1(x1), F1(x2)]. The second group [x1, x2] is modulated on λ1 and λ2 respectively. 2] is modulated on λ1 and λ2 by two sequentially connected second modulators in the second group of second modulators, respectively, to obtain [F2(x1), F2(x2)], which are detected one by one by two second photodetectors to obtain intermediate signals [F1(x1), F1(x2)] and [F2(x1), F2(x2)], and finally [F(x1), F(x2)] = [F1(x1), F1(x2)] + [F2(x1), F2(x2)] is obtained;

[0064] The output layer includes an adder, which is used to add the input signal and the intermediate signal through circuit connection to obtain the output signal [F(x1)+x1, F(x2)+x2].

[0065] Then, the corresponding photonic residual network model is trained on a computer, and the parameters in each residual block are saved; the parameters in the residual block are loaded into the photonic residual deep network chip to complete the construction of the photonic residual deep network chip.

[0066] Among them, the photon residual network model has 3 residual blocks. Each residual block includes an input layer, an identity mapping layer, a hidden layer, and an output layer. The input layer has 2 neurons, the hidden layer has 4 neurons, and the output layer has 2 neurons. All parameters in the network are restricted to the interval [0, 1].

[0067] Before training the corresponding photon residual network model, it further includes: generating a training set for binary classification. The data points are located in a two-dimensional space, and each data point consists of two coordinate values (x1, x2). The classification rules are as follows: If x1 >= x2, the label of this data point is (1, 0); otherwise (i.e., x1 < x2), the label of this data point is (0, 1).

[0068] Transform the data and labels in the computer dataset into one-dimensional vectors, and normalize the elements in the vector to the interval [0, 1], corresponding to the input signal [x1, x2] and the target signal [y1, y2];

[0069] Use the mean square error for network training, save the weight information in the hidden layer after training, and then load the weight information in the hidden layer after training into the microring modulators in the hidden layer of the photon residual deep network chip. The optical power is modulated through the modulators to achieve weight loading.

[0070] An embodiment of the present invention provides a method for constructing a photon residual deep network chip, which is characterized by including:

[0071] S1, connecting multiple photon residual blocks in sequence; among them, each of the photon residual blocks includes:

[0072] An input layer, including a vertically coupled coupler, an optical waveguide, N first modulators, and a first power splitter connected in sequence; the optical signal is input into the optical waveguide through the vertically coupled coupler, and the N first modulators respectively load N input signals in the N wavelengths of the optical signal to obtain a target signal. The target signal is split into two by the power splitter, namely the first and second target signals;

[0073] [[ID=2I]]An identity mapping layer, including a wavelength division multiplexer and N first photodetectors; the first target signal is divided into N paths by the wavelength division multiplexer and is respectively detected by the N first photodetectors;

[0074] A hidden layer, including a second power splitter, N groups of second modulators, and N second photodetectors; the second target signal is equally divided into N parts by the second power splitter, and after being modulated by the N groups of second modulators respectively, it is respectively detected by the N second photodetectors; each group of second modulators includes N second modulators;

[0075] an output layer, comprising an adder, configured to add the signals detected by the N first photodetectors and the signals detected by the N second photodetectors in a one-to-one correspondence to obtain an output signal;

[0076] S2, training the residual deep network, and determining the modulation parameters of each second modulator in each group of second modulators according to the neuron weights of the hidden layer of each residual block in the trained residual deep network;

[0077] Among them, the residual deep network includes multiple residual blocks connected in sequence, each residual block includes an input layer, an identity mapping layer, a hidden layer and an output layer, and the function of each layer corresponds one-to-one to the function of the input layer, identity mapping layer, hidden layer and output layer of the photon residual block. The hidden layer of each residual block includes N groups of neurons, each group of neurons includes N neurons, and the weight of each neuron is unknown. The other layers include N neurons and the weight of each neuron is a fixed value, N≥1.

[0078] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A photon residual deep network chip, characterized in that: include: Multiple photon residual blocks connected sequentially; Wherein, each of the photon residual blocks includes: The input layer includes a vertical coupler, an optical waveguide, N first modulators, and a first power splitter connected in sequence. The optical signal is input into the optical waveguide via the vertical coupler. The N first modulators load the N input signals onto the N wavelengths of the optical signal in a one-to-one correspondence to obtain a target signal. The target signal is split into two by the first power splitter, namely the first and second target signals. The identity mapping layer includes a wavelength division multiplexer and N first photodetectors; the first target signal is divided into N signals by the wavelength division multiplexer and detected one by one by the N first photodetectors; The hidden layer includes a second power splitter, N groups of second modulators, and N second photodetectors; the second target signal is divided into N equal parts by the second power splitter, modulated one-to-one by the N groups of second modulators, and detected one-to-one by the N second photodetectors; an output layer, comprising an adder for adding the signals detected by the N first photodetectors and the signals detected by the N second photodetectors in a one-to-one correspondence to obtain an output signal; Among them, each group of second modulators includes N second modulators, and its modulation parameters are determined according to the neuron weights of the hidden layer of each residual block in the trained residual deep network; the residual deep network includes multiple residual blocks connected in sequence, and each residual block includes an input layer, an identity mapping layer, a hidden layer and an output layer. The function of each layer corresponds one-to-one to the function of the input layer, identity mapping layer, hidden layer and output layer of the photon residual block. The hidden layer of each residual block includes N groups of neurons, each group of neurons includes N neurons, and the weight of each neuron is unknown. The other layers include N neurons and the weight of each neuron is a fixed value, N≥1.

2. The chip according to claim 1, wherein: The training data set of the residual deep network includes sample data and label data of the target task.

3. The chip according to claim 2, wherein: The training data set of the residual deep network includes normalized sample data and label data.

4. The chip according to claim 3, wherein: The neuron weights of the input layer, identity mapping layer, hidden layer and output layer of each residual deep network are all in the range of [0,1].

5. The chip according to claim 1, wherein: During the training process of the residual deep network, the mean square error is used as the loss function.

6. The chip according to claim 1, wherein: The modulator is a micro-ring modulator or an MZI modulator, and the modulation parameter is light transmittance.

7. A method for constructing a photon residual deep network chip, characterized in that: include: S1, connecting multiple photon residual blocks in sequence; wherein each of the photon residual blocks includes: The input layer includes a vertical coupler, an optical waveguide, N first modulators, and a first power splitter connected in sequence. The optical signal is input into the optical waveguide via the vertical coupler. The N first modulators load the N input signals one-to-one at the N wavelengths of the optical signal to obtain a target signal. The target signal is split into two by the power splitter, namely the first and second target signals. The identity mapping layer includes a wavelength division multiplexer and N first photodetectors; the first target signal is divided into N signals by the wavelength division multiplexer and detected one by one by the N first photodetectors; The hidden layer includes a second power splitter, N groups of second modulators, and N second photodetectors; the second target signal is divided into N equal parts by the second power splitter, modulated one-to-one by the N groups of second modulators, and detected one-to-one by the N second photodetectors; each group of second modulators includes N second modulators; an output layer, comprising an adder for adding the signals detected by the N first photodetectors and the signals detected by the N second photodetectors in a one-to-one correspondence to obtain an output signal; S2, training the residual deep network, and determining the modulation parameters of each second modulator in each group of second modulators according to the neuron weights of the hidden layer of each residual block in the trained residual deep network; Among them, the residual deep network includes multiple residual blocks connected in sequence, each residual block includes an input layer, an identity mapping layer, a hidden layer and an output layer, and the function of each layer corresponds one-to-one to the function of the input layer, identity mapping layer, hidden layer and output layer of the photon residual block. The hidden layer of each residual block includes N groups of neurons, each group of neurons includes N neurons, and the weight of each neuron is unknown. The other layers include N neurons and the weight of each neuron is a fixed value, N≥1.

Images