Recurrent neural network device

Abstract

A recurrent neural network device (10) comprises: a hidden unit (12) that sequentially computes first to n-th computation target signals respectively including first to n-th time-series signals (where n is an integer of 3 or higher) respectively input at first to n-th discrete timings, and sequentially outputs first to n-th computation result signals indicating computation results; and a loop unit (Lp) that incorporates the first to (n-1)-th computation result signals output from the hidden unit (12) respectively into the second to n-th computation target signals, which are input to the hidden unit. The loop unit (Lp) holds, by means of RC delay, an m-th computation result signal output from the hidden unit from among the first to (n-2)-th computation result signals, and incorporates the m-th computation result signal into an (m+1)-th computation target signal together with an (m+1)-th computation result signal (where m is an integer satisfying 1≤m≤n-2).

Description

Recurrent Neural Network Device

[0001] This invention relates to a recurrent neural network device.

[0002] A recurrent neural network device that operates as a recurrent neural network capable of processing time-series signals is known (Non-Patent Literature 1). The recurrent neural network device comprises a hidden unit that performs calculations on a time-series signal and outputs a calculation result signal indicating the calculation result, and a loop unit that inputs the calculation result signal together with the next time-series signal to the hidden unit.

[0003] C. Feng et al., "Compact Design of On-Chip Elman Optical Recurrent Neural Network," Proc. CLEO, no. JTh2B (2020)

[0004] In the recurrent neural network device described in Non-Patent Document 1, the calculation result signal is affected by signal loss occurring in the loop section, which can lead to a decrease in the calculation accuracy of the time-series signal.

[0005] The present invention aims to suppress the decrease in calculation accuracy of time-series signals.

[0006] To solve the above problems, the recurrent neural network device according to the present invention comprises a hidden unit that sequentially performs calculations on first to nth target signals, each containing first to nth time-series signals (where n is an integer of 3 or more) that are input at first to nth discrete timings, and sequentially outputs first to nth calculation result signals indicating the calculation results; and a loop unit that includes the first to n-1st calculation result signals output from the hidden unit into the second to nth target signals input to the hidden unit, respectively, wherein the loop unit holds the mth calculation result signal output from the hidden unit among the first to n-2nd calculation result signals by RC delay and includes it into the m+1st target signal together with the m+1st calculation result signal (where m is an integer satisfying 1 ≤ m ≤ n-2).

[0007] The above configuration suppresses the decrease in the calculation accuracy of time-series signals.

[0008] Figure 1 is a diagram showing the configuration of an RNN device according to an embodiment of the present invention. Figure 2 is a circuit diagram of the photoelectric-to-optical conversion unit. Figure 3 is a diagram showing a specific example of the configuration of the hidden unit. Figure 4 is a diagram showing a specific example of the configuration of the hidden unit. Figure 5 is a diagram showing a specific example of the configuration of the hidden unit. Figure 6 is a diagram showing a specific example of the configuration of the photoelectric-to-optical conversion unit. Figure 7 is a diagram showing a specific example of the configuration of the photoelectric-to-optical conversion unit. Figure 8 is a diagram showing a specific example of the configuration of the photoelectric-to-optical conversion unit. Figure 9 is a plan view of the expansion substrate according to a modified example. Figure 10 is a diagram showing the configuration of an RNN device according to a modified example. Figure 11 is a diagram showing the configuration of an RNN device according to a modified example. Figure 12 is a diagram showing the configuration of an RNN device according to a modified example. Figure 13 is a diagram showing the configuration of an RNN device according to a modified example. Figure 14 is a diagram showing the configuration of an RNN device according to a modified example. Figure 15 is a graph showing the simulation results. Figure 16 is a graph showing the simulation results. Figure 17 is a graph showing the simulation results. Figure 18 is a graph showing the simulation results.

[0009] (Embodiment) This embodiment will be described below. In each figure, only some of the multiple identical elements may be denoted by reference numerals.

[0010] (Configuration of the Recurrent Neural Network Device) The recurrent neural network (RNN) device 10 according to this embodiment, shown in Figure 1, forms a recurrent neural network using hardware in which optical and electrical circuits are formed. The RNN device 10 is modeled on a computer before it is actually constructed. Each circuit configuration of the modeled RNN device 10 (for example, the shift amount of the phase shifter described later) is optimized by machine learning on a computer. By actually constructing each optimized circuit configuration, the machine learning-enabled RNN device 10 is constructed. The same applies to other RNN devices described later.

[0011] The RNN device 10 receives time-series signals x1, x2, ... x nThe target signal, which is an optical signal containing the time series, is input at discrete timings (e.g., discrete time intervals) t = 1, 2, ..., n. The time series signal is a signal that represents each data item arranged in chronological order, and the input order also has meaning. Specific examples of time series data include optical signals representing each word of a sentence and optical signals representing each character of a string. The discrete timing t arrives every time interval Δt has elapsed. The RNN device 10 receives the time series signals x1, x2, ..., x n The calculation is performed, and the result signals z1, z2, ..., z are optical signals representing the calculation result. n The output is as follows. Note that each time-series signal and / or each calculation result signal may be a signal representing a vector having multiple spatial dimensions. In this case, when the RNN device 10 processes (especially calculates) the signal, it may separate the signal into each component signal that constitutes the vector, calculate each separated component signal individually, or recombine any component signal before or after the calculation. Alternatively, the RNN device 10 may convert the signals to be processed, such as each time-series signal and / or each calculation result signal, into signals with multiple vector components before processing.

[0012] The RNN device 10 includes an optical summing unit 11, a hidden unit 12, an optical branching unit 13, an output unit 14, an OEO (Optical-Electrical-Optical) conversion unit (hereinafter also simply referred to as a conversion unit) 15, and a loop delay 16. These will be described below, but elements 11 to 16 are configured to process each signal representing a vector component as described above, and may be configured to perform the following processing (including calculations) in parallel for each signal. For example, the circuit in Figure 2, described later, may actually be provided for each vector component, and each circuit may process each vector component separately. The description of the processing below includes the possibility of performing the following processing for each vector component. Even in such a case, the processing of the signal as a whole can be considered to be performed as described below.

[0013] The optical addition unit 11 generates an operation target signal including both signals by adding an operation result signal, which will be described later and is supplied via the loop delay 16, to the input time-series signal. The optical addition unit 11 inputs the generated operation target signal to the hidden unit 12. The optical addition unit 11 is an optical circuit including at least one of, for example, an optical coupler such as a directional coupler or a multimode interference (MMI) waveguide, and an MZI (Mach Zehnder Interferometer) switch. If both the time-series signal and the operation result signal are of the same wavelength, the optical addition unit 11 is configured to perform complex addition of the two signals. If the time-series signal and the operation result signal are of different wavelengths, the optical addition unit 11 is configured to perform real addition of the two signals.

[0014] The hidden unit 12 linearly transforms the operation target signal by operating on the operation target signal from the optical addition unit 11. The hidden unit 12 optically operates on the product of the operation target signal and a fixed weight matrix and outputs an operation result signal indicating the operation result. The hidden unit 12 is an optical circuit combining optical circuit elements such as optical wiring, optical couplers, phase shifters, and MZI switches. The weight matrix is not limited to being fixed and may be set to vary for each time-series signal or for any arbitrary time period.

[0015] The optical branching unit 13 distributes (branches) the operation result signal from the hidden unit 12 to the output unit 14 and the conversion unit 15 at an arbitrary ratio. Similar to the optical addition unit 11, the optical branching unit 13 is an optical circuit including at least one of, for example, an optical coupler such as a directional coupler or a multimode interference waveguide, and an MZI switch.

[0016] The output unit 14 is an optical circuit that combines optical circuit elements such as optical wiring, optical couplers, phase shifters, and MZI switches. By increasing or decreasing the spatial dimension of the operation result signal and giving a fixed intensity distribution / phase change (weight) to the operation result signal, freedom is added to the output. The output unit 14 may be configured to include a photodiode or the like. In this case, the output unit 14 converts the operation result signal into an electrical signal, converts the converted electrical signal into an electrical signal by photoelectrically converting the operation result signal from the optical branching unit 13, and may output it to the outside of the RNN device 10 as the output of the recurrent neural network.

[0017] The conversion unit 15 is composed of a combination of various components such as a photoelectric converter, an electro-optical modulator, an electrical element, and an optical coupler. The conversion unit 15 photoelectrically converts the operation result signal from an optical signal into an electrical signal, electro-optically converts it back into an optical signal, and outputs the operation result signal after the electro-optic conversion.

[0018] The conversion unit 15 is configured to leave the operation result signal at the previous timing in the conversion unit 15 due to the RC delay described later in an electrical circuit that processes electrical signals, and add it to the operation result signal input at the next timing after Δt. The conversion unit 15 outputs the signal after the addition as the output signal h j and outputs it. Δt is the time interval of the discrete timing. The conversion unit 15 is configured such that the output signal h j at the discrete timing j becomes h j+1 = β * z j + α * h j (details will be described later). Note that "*" means multiplication (the same applies to "*" hereinafter). z j indicates the operation result signal input to the conversion unit 15 at the discrete timing j. α and β will be described later. Since the response of the conversion unit 15 to the input operation result signal is non-linear, it functions as an activation unit of the RNN.

[0019] The loop delay 16 is composed of an optical fiber delay line, a Bragg grating, and the like. The loop delay 16 is the output signal h of the conversion unit 15 jThe signal is delayed by Δt. Δt is a discrete timing interval as described above. The output signal h after passing through loop delay 16. j Due to a delay of Δt, the signal is input to the optical summing unit 11 at the next discrete timing, i.e., when the next time-series signal is input. Output signal h j This includes the past calculation result signal as described above. The loop delay 16 may consist of a signal path from the hidden unit 12 to the optical adder unit 11, or it may consist of a combination of the signal path and an optical fiber delay line.

[0020] The optical summing unit 11, the optical branching unit 13, the conversion unit 15, and the loop delay 16 constitute a loop Lp that feeds back the output of the hidden unit 12 to the hidden unit 12.

[0021] (RNN device operation) The time-series signal x1 input at the initial discrete timing t=1 is input to the hidden unit 12 as the target signal x1 without signal addition by the optical adder unit 11. However, for example, if the output signal h0 in the initial state of the loop Lp is not a zero vector, the optical adder unit 11 adds the time-series signal x1 and the output signal h0 and supplies the added signal to the hidden unit 12 as the target signal. The hidden unit 12 performs an operation to multiply the target signal, i.e., the time-series signal x1, by matrix W and outputs the calculation result signal z1 = W * x1 obtained from this operation. This calculation result signal z1 is distributed to the output unit 14 and the conversion unit 15 by the optical branching unit 13. The output unit 14 processes the calculation result signal z1 as appropriate and outputs it externally.

[0022] The calculation result signal z1 distributed to the conversion unit 15 is amplified by a gain (amplification factor) β and, as an output signal h1 = β * z1, is delayed by Δt by the loop delay 16 and input to the optical summing unit 11. The output signal h1 = β * z1 is input to the optical summing unit 11 at a discrete timing t = 2. Furthermore, the calculation result signal z1 is held in the conversion unit 15 (held in a capacitor as described later), and the held calculation result signal z1 decays over time, and at the time Δt has elapsed, in other words at the discrete timing t = 2, it decays to a decayed signal α * z1, which is the decayed calculation result signal z1. α is the RC delay coefficient, where α = 1 means no decay (discharge), and the smaller α is, the greater the decay.

[0023] At discrete timing t=2, the optical summing unit 11 adds the output signal h1 to the input time-series signal x2, and inputs the added signal h1+x2=β*z1+x2 to the hidden unit 12 as the signal to be calculated. In this way, the summing unit 11 includes the output signal and the time-series signal as the signal to be calculated. The hidden unit 12 performs an operation to multiply the signal to be calculated β*z1+x2 by matrix W, and outputs the calculated result signal z2=W(β*z1+x2) obtained from this operation. This calculated result signal z2 is distributed to the output unit 14 and the conversion unit 15 by the optical branching unit 13. The output unit 14 processes the calculated result signal z2 as appropriate and outputs it externally.

[0024] The conversion unit 15 adds the attenuated signal α*z1 to the calculation result signal z2 at discrete timing t=2 to obtain the added signal z2+α*z1. In the conversion unit 15, this added signal z2+α*z1 is amplified by gain β and output as the output signal h2 = β*z2+α*β*z1. The output signal h2 = β*z2+α*β*z1 is delayed by Δt by the loop delay 16 and input to the optical summation unit 11 at discrete timing t=3. Furthermore, the added signal z2+α*z1 is held in the conversion unit 15 (held in a capacitor as described later), and the held added signal z2+α*z1 is attenuated over time, and at Δt elapsed, in other words at the discrete timing t=3, the attenuated signal α*z2+α 2 *Attenuates to z1

[0025] At discrete timing t=3, the optical summing unit 11 adds the output signal h2 to the time-series signal x3, and inputs the added signal h2+x3=β*z2+α*β*z1+x3 to the hidden unit 12 as the signal to be calculated. The hidden unit 12 performs an operation to multiply the signal to be calculated β*z2+α*β*z1+x3 by matrix W, and outputs the calculated result signal z3=W(β*z2+α*β*z1+x3). This calculated result signal z3 is distributed to the output unit 14 and the conversion unit 15 by the optical branching unit 13. The output unit 14 processes the calculated result signal z3 as appropriate and outputs it externally.

[0026] The conversion unit 15, at discrete timing t=3, applies the attenuated signal α*z2+α to the calculation result signal z3. 2 * Add z1 to summation signal z3 + α * z2 + α 2 * z1 is obtained. In the conversion unit 15, this summation signal z3 + α * z2 + α 2 *z1 is amplified by gain β, and the output signal h3 = β*z3 + α*β*z2 + α 2 It is output as *β*z1. The output signal h3 is delayed by Δt by the loop delay 16 and input to the optical summing unit 11 at discrete timing t=4. Furthermore, the summing signal z3 + α * z2 + α 2 *z1 is held in the conversion unit 15 (held in a capacitor as described later), and the held summation signal z3 + α * z2 + α 2 *z1 decays over time, and at the time Δt has elapsed, in other words, at the discrete timing t=4, the decayed signal α*z3 + α 2 *z2+α 3 *Attenuates to z1. Attenuated signal α * z3 + α 2 *z2+α 3 * z1 is added to z4, which is input to the conversion unit 15 at discrete timing t=4.

[0027] At discrete timing t=4, the optical summing unit 11 adds the output signal h3 to the time-series signal x4, and the added signal h3 + x4 = β*z3 + α*β*z2 + α 2The signal *β*z1 + x4 is input to the hidden unit 12 as the signal to be calculated. The hidden unit 12 calculates the signal β*z3 + α*β*z2 + α 2 The operation is performed to multiply *β*z1 + x4 by the matrix W, and the resulting signal z4 = W(β*z3 + α*β*z2 + α) is obtained from this operation. 2 Output *β*z1 + x4).

[0028] Based on the above, the outputs h1 to h4 of the conversion unit 15 at discrete timings t=1 to 4 are given by equations (1) to (4) below, and generalizing these gives equation (5) below.

[0029] From equations (1) to (5) above, it can be seen that the calculation result signal is included in the output signal in a state that is attenuated according to the RC delay coefficient α each time a subsequent discrete timing occurs.

[0030] As a result of the conversion unit 15 described above, the calculation result signals z1 to z4 at discrete timings t = 1 to 4 are given by equations (6) to (9) below, and when generalized, they are given by equation (10) below. Here, j ≥ 2.

[0031] From equation (10) above, the calculated signal z at discrete timing t = j is j The state shows that the conversion unit 15 incorporates the state of the calculation result signal at discrete timings t = 1, 2, 3, ..., j-1. In contrast, the calculation result signal z when the conversion unit 15 is not used is shown. j is, z j = W * β ０ *z j-1 +Wx j It is expressed as (where β ０ (where is the circulating loss). As can be seen from the above equation, if the conversion unit 15 is not used, the calculation result signal z at discrete timing t = j j This is the signal z from the previous calculation result. j-1 and input signal x j It is affected only by [the specified frequency]. In the RNN device 10 using the conversion unit 15, time-series signals with many timings are used due to the holding of the calculation result signal and Δt.

[0032] According to this embodiment, since an output dependent on a time-series signal is obtained, it can be applied to tasks such as classifying specific input data based on the intensity distribution of the output port or time-series forecasting by using conventional machine learning schemes. This embodiment is particularly suitable for applications requiring high-speed signal processing due to its computation using optical signals.

[0033] (Structure and operation of the conversion unit) The conversion unit 15 is composed of, for example, the photoelectric circuit shown in Figure 2. The conversion unit 15 comprises a photoelectric converter 15A made of a photodiode or the like, and a drive power supply 15B to which a positive electrode is connected to the cathode of the photoelectric converter 15A and which applies a drive voltage to the photoelectric converter 15A. The photoelectric converter 15A receives the calculation result signals z1, z2, ..., z from the optical branching unit 13. n These signals are input and converted into current signals Ipd each time.

[0034] The conversion unit 15 further includes a load resistor 15C, a capacitor 15D, and an electro-optical converter 15E, all connected in series with the photoelectric converter 15A and in parallel with each other. The electro-optical converter 15E is, for example, composed of an electro-optical modulator (EOM). The load resistor 15C and the capacitor 15D constitute a parallel RC circuit 15Z. In addition to or instead of the load resistor 15C, a resistor may be placed between the photoelectric converter 15A, the capacitor 15D, and the electro-optical converter 15E. This resistor and the capacitor 15D may constitute an RC circuit. The resistor may also be a wiring resistor of the conversion unit 15.

[0035] The conversion unit 15 further includes a reference light source 15F. The electro-optic converter 15E (in particular, an electro-optic modulator) is configured to modulate the reference light (carrier wave) emitted by the reference light source 15F. The electro-optic converter 15E may include, for example, a Mach-Zehnder modulator (MZM) or a ring resonator.

[0036] At discrete timing t=1, the photoelectric converter 15A converts the calculation result signal z1 (optical signal) input from the optical branching unit 13 into a current signal Ipd1. This conversion generates a voltage across the load resistor 15C corresponding to the current signal Ipd1 and the resistance value of the load resistor 15C, i.e., a voltage representing the calculation result signal z1. This voltage is applied to the electrophotoconverter 15E, which is in parallel with the load resistor 15C. The electrophotoconverter 15E is driven by the applied voltage and modulates the reference light (carrier wave) from the reference light source 15F into an optical signal representing the calculation result signal z1, and outputs the modulated optical signal as an output signal h1. The output signal h1 is a signal obtained by amplifying the calculation result signal z1 with a gain β. In other words, the electrophotoconverter 15E outputs an optical signal representing the output signal h1 = β * z1. β is adjusted to an arbitrary value depending on the light intensity of the reference light, the photoreception level of the photoelectric converter 15A, the driving voltage, etc.

[0037] The voltage applied to the photoconverter 15E charges the capacitor 15D connected in parallel with the photoconverter 15E. The charging voltage of the charged capacitor 15D represents the calculation result signal z1. After the input of the calculation result signal z1, there is no signal input until the input of the next calculation result signal z2, so the charging voltage is discharged. After Δt has elapsed, that is, at the arrival of the next discrete timing t=2, the charging voltage has decreased to a voltage representing α*z1. Therefore, a voltage representing α*z1 remains across the load resistor 15C at the discrete timing t=2. α is α = e －Δｔ／ＣＲ Defined by the formula, where C is the capacitance of capacitor 15D and R is the resistance of load resistor 15C.

[0038] At discrete timing t=2, the photoelectric converter 15A converts the calculation result signal z2 (optical signal) input from the optical branching unit 13 into a current signal Ipd2. This conversion adds a new voltage representing the calculation result signal z2 across the load resistor 15C. At this timing, as described above, a voltage representing α*z1 remains across the load resistor 15C, so the voltage across the load resistor 15C becomes a voltage representing the signal z2 + α*z1, which is the calculation result signal z2 plus the attenuated calculation result signal z1. This voltage is applied to the photoelectric converter 15E. The photoelectric converter 15E is driven by the applied voltage and modulates the reference light (carrier wave) from the reference light source 15F into an optical signal representing the signal z2 + α*z1, and outputs the modulated optical signal β(z2 + α*z1) as the output signal h2.

[0039] As described above, the photoelectric converter 15A outputs the output signal h of formula (5) above. j The output is as follows. Note that equation (5) above can be transformed into equation (11) below. As can be seen from equation (11) below, the output signal is a signal obtained by adding the output of the calculation result signal at that discrete timing (output amplified by gain β) to the output signal from the previous discrete timing which has been attenuated by α, the number of times the discrete timing of that calculation result signal has occurred up to the present. The output signal from the previous discrete timing contains information of successive calculation result signals. Each successive calculation result signal is attenuated by being multiplied by α, which is the number of discrete timings that have occurred up to the present for that calculation result signal.

[0040] (Structure of the Hidden Unit) The structure of the hidden unit 12 is illustrated in Figures 3 to 5. The structure in Figure 3 is called the Clements type, and is a mesh-like arrangement of unit cells UC, each with one phase shifter 12A installed inside and outside the arm of the MZI 12B. When the number of input ports of the hidden unit 12 is N, N stages of unit cells UC are connected, and by further arranging the phase shifter 12A, which is an implementation form of a unitary diagonal matrix, in each signal path, it becomes possible to implement any unitary matrix. When introduced into an RNN, it is not necessarily required to use the same number of unit cells as the number of input ports. In other words, it is possible to consider connections of less than N stages. Conversely, it is also possible to consider redundant connections of more than N stages.

[0041] The structure shown in Figure 4 is called a butterfly type, and it is a circuit configuration that performs the Fourier transform of spatially parallel input signals by making extensive use of waveguide crossings 12G, which cross optical waveguides connecting unit cells UC. In particular, it has the effect of efficiently mixing signals with fewer stages of unit cells UC than the Clements type, and is one of the important configurations.

[0042] The structure shown in Figure 5 is called an MMI type, and it is a circuit configuration in which optical circuit elements 12K such as a flat slab waveguide element or optical coupler are connected to a unit cell UC, and the spatial Fourier transform is performed by diffraction of light. Similar to the butterfly type, it has the effect of efficiently mixing signals with fewer stages. An additional feature is that it is likely to have a smaller footprint than the butterfly type, making it an even more important configuration.

[0043] (Structure of the conversion unit) Figures 6 to 8 illustrate the conversion structures for photoelectric conversion and electro-optical conversion that the conversion unit 15 can employ. Figure 6 shows the simplest conversion structure. In the structure of Figure 6, the photoelectric converter 15A converts the optical signal A in1 (The above calculation result signals z1, z2, ..., z n ) is received, and this optical signal A in1 The signal is converted into an electrical signal (voltage, etc., representing the calculation result signal) ES. The electrical signal ES is input to the photoconverter 15E via an RC circuit 15Z including a load resistor 15C and a capacitor 15D (Figure 2) as described above. The photoconverter 15E modulates the reference light from the reference light source 15F based on the electrical signal ES, and the modulated optical signal A out (The above output signals h1, h2, ..., h n ) is output as. The optical signal A is determined by the characteristics of the photoelectric converter 15A and the electrophotoconverter 15E. in1 and A in2 The input-output relationship (especially β mentioned above) is determined.

[0044] The structure in Figure 7 utilizes single-arm modulation of a Mach-Zehnder modulator (MZM) in the photoconverter 15E. The photoconverter 15E includes a phase modulator 15EA electrically connected to the photoelectric converter 15A. The phase modulation amount of the phase modulator 15EA is the optical signal A in1(The above calculation result signals z1, z2, ..., z n The optical signal A fluctuates in proportion to the input intensity of ). in2 Distribution ratio (optical signal A) out1 Intensity / Optical signal A out2 The intensity of the optical signal A fluctuates. out1 and optical signal A out2 It is arbitrary whether to use one of the two optical output terminals from which light is emitted, or to use both.

[0045] The structure in Figure 8 utilizes a microring modulator (MRM) in the photoconverter 15E. The photoconverter 15E includes a phase modulator 15EA that is electrically connected to the photoelectric converter 15A. The phase modulation amount of the phase modulator 15EA is the optical signal A in1 (The above calculation result signals z1, z2, ..., z n The resonant wavelength of the photoelectric converter 15E, which acts as a microring modulator (MRM), changes in proportion to the input intensity of the optical signal A. in2 As the degree of optical coupling changes, the optical signal A in2 Distribution ratio (optical signal A) out1 Intensity / Optical signal A out2 The intensity of the optical signal A fluctuates. out1 and optical signal A out2 It is arbitrary whether to use one of the two optical output terminals from which light is emitted, or to use both.

[0046] (Modified RNN Apparatus) Modified RNN apparatus will be described with reference to Figures 9 to 13. In the following description, elements corresponding to the elements of the above embodiment will be denoted by the same reference numerals, and redundant explanations will be omitted. Furthermore, the contents of the above description can also be applied to the following configuration.

[0047] The RNN device 110 shown in Figure 9 includes an optical summing unit 11, a hidden unit 12, an optical branching unit 13, a conversion unit 15, and a loop delay 16, as well as a light source 121, an electro-optical (EO) conversion unit 122, an input layer (also called an input unit) 123, a control unit 124, an output layer (also called an output unit) 125, and an electro-optical (OE) converter 126. The electro-optical conversion unit 122 includes a plurality of electro-optic converters 122A to 122D.

[0048] The photoconverters 122A to 122D are sequentially supplied with time-series signals to be processed from an external device 90, such as an electrical memory. Here, the time-series signal represents a vector with spatial dimensions of 4 and includes four electrical signals, each representing one of the four values ​​that make up this vector component. The four electrical signals are input one to each of the photoconverters 122A to 122D. Each of the photoconverters 122A to 122D consists of, for example, an encoder that converts the input electrical signal into an optical signal. Each of the photoconverters 122A to 122D includes, for example, a digital-to-analog converter that converts a digital electrical signal from the external device 90 into an analog electrical signal, and an electro-optic modulator that converts the analog electrical signal into an optical signal. The electro-optic modulator converts the analog electrical signal into an optical signal by modulating the light from the light source 121 with the analog electrical signal input from the digital-to-analog converter. The modulation method can be phase modulation, amplitude modulation, or both. The number of photoconverters should be the same as the number of dimensions of the vector represented by the time-series signal. As a result, each electro-optical converter can convert the electrical signals of each value constituting the vector represented by the time-series signal into optical signals of each value, one by one, in parallel. The electro-optical conversion unit 122 generates a time-series signal consisting of each optical signal by performing the conversion to each optical signal using each of the electro-optical converters 122A to 122D. In this way, the electro-optical conversion unit 122 converts the time-series signal from an electrical signal to an optical signal. The time-series signal as an optical signal is input to the input layer 123. At this time, each optical signal constituting the time-series signal may be combined and input to the input layer 123, or it may be input in parallel to the four ports provided on the input layer 123. Optical time-series signals supplied in optical communication or optical sensing may be directly input to the input layer 123.

[0049] The input layer 123 processes the input time-series signal by increasing or decreasing its spatial dimensions and applying fixed weights (e.g., intensity distribution and / or phase shift). While the input layer 123 is not strictly necessary, its presence enhances the functionality and flexibility of the RNN device 110. The input layer 123 is configured as an optical circuit including optical wiring and one or more optical circuit elements such as optical couplers, phase shifters, and MZI switches. The input layer 123 may be configured to apply weights that are not fixed but differ for each time-series signal or fluctuate at an arbitrary time period. The time-series signal processed by the input layer 123 is input to the optical summing unit 11.

[0050] The control unit 124 is not mandatory, but is positioned between the conversion unit 15 and the loop delay 16. The control unit 124 is configured to include, for example, a variable attenuator or a phase shifter to improve the degree of freedom of calculation. The control unit 124 processes the output signal output by the conversion unit 15.

[0051] The output layer 125 receives the calculation result signals distributed by the optical branching unit 13. The output layer 125 processes the input calculation result signals by increasing or decreasing the spatial dimension and further assigning fixed weights (e.g., intensity distribution and / or phase change). The output layer 125 is not strictly necessary, but its presence ensures greater functionality and flexibility of the RNN device 110. The output layer 125 is configured as an optical circuit including optical wiring and one or more optical circuit elements such as optical couplers, phase shifters, and MZI switches. The output layer 125 may be configured to assign weights that differ for each time-series signal or that fluctuate at an arbitrary time period. The calculation result signals processed by the output layer 125 are input to the photoelectric converter 126.

[0052] The photoelectric converter 126 converts the calculation result signal from the output layer 125 from an optical signal to an electrical signal, and determines the processing result by threshold processing and / or determination processing, including analog-to-digital conversion. The photoelectric converter 126 is composed of a combination of a photodetector and electronic circuits. The photoelectric converter 126 receives a time-series signal and, after a specific time has elapsed, receives the output of the output layer 125, that is, a specific calculation result signal (for example, the last calculation result signal Z). n ) may be observed.

[0053] In the RNN device 110, the loop section Lp is composed of an optical summing unit 11, a hidden unit 12, an optical branching unit 13, a conversion unit 15, a loop delay 16, and a control unit 124.

[0054] The RNN device 210 shown in Figure 10 has a configuration that is a modified version of the RNN device 110. Specifically, in the RNN device 210, the optical summing unit 11 of the RNN device 110 is omitted, and a conversion unit 15 is placed in the position of the optical summing unit 11.

[0055] The RNN device 310 shown in Figure 11 has a configuration that is a modified version of the RNN device 110. In other words, the positions of the hidden unit 12 and the conversion unit 15 are reversed between the RNN device 310 and the RNN device 110.

[0056] The RNN device 410 shown in Figure 12 has a configuration that is a modified version of the RNN device 110. Specifically, in the RNN device 410, the conversion unit 15 is positioned between the hidden unit 12 and the optical addition unit 11.

[0057] The RNN device 510 shown in Figure 13 has a configuration that is a modified version of the RNN device 110. Specifically, in the RNN device 510, the optical summing unit 11 and the optical branching unit 13 are merged to form an addition / distribution unit 511. In addition, the conversion unit 15, the hidden unit 12, the control unit 124, and the loop delay 16 are connected in series. The calculation result signal distributed by the addition / distribution unit 511 is input to the conversion unit 15, and the signal output by the loop delay 16 is input to the addition / distribution unit 511 and added to the next time-series signal. The configuration of the RNN device 510 is the same as a configuration in which so-called ring resonators are connected in spatial parallel.

[0058] The configuration of the RNN device is arbitrary, and it is desirable to adopt the optimal configuration depending on the type of time-series signal being processed. In other words, the optimal configuration of the RNN device differs depending on the type of time-series signal.

[0059] (Details and simulation results of the RNN device) The RNN device 610 shown in Figure 14 is a configuration of the RNN device 110 described above with elements 121, 122, 125, and 126 omitted. The loop delay 16 is not shown in the figure, but its function is realized by the length of the optical signal line of the RNN device 610. The hidden unit 12 of the RNN device 610 consists of a 4x4 Clements optical circuit (see also Figure 3). Each of the input layer 123, optical summing unit 11, hidden unit 12, and optical branching unit 13 includes multiple Mach-Zehnder interferometers 613, each containing a directional coupler 611 and a phase shifter 612. The control unit 124 is equipped with multiple phase shifters 612. Multiple phase shifters 612 are also arranged at the last stage of each of the input layer 123 and hidden unit 12. Here, the input layer 123 converts the time-series signal into four vector-component optical signals, and outputs the four converted optical signals from four optical paths. The hidden unit 12 vectorizes the time-series signal (four optical signals) using a weight matrix. Note that, as in the RNN device 310, the positions of the hidden unit 12 and the conversion unit 15 may be reversed (not shown).

[0060] This section describes the circuit simulation of the RNN device 610 model. Here, the 28 × 28 = 784 pixel element vector of training image data (MNIST) of handwritten digit images G (e.g., digits 0-3) is converted from real numbers to complex numbers using a Fast Fourier Transform (FFT) on a computer. After this conversion, dimensionality reduction is performed by selecting four Fourier components with high output intensity relative to the training image data to obtain a set of time-series signals x. Multiple sets of these sets x are input into a model of the RNN device 510 constructed on a computer to simulate machine learning and image recognition after machine learning. Each time-series signal constituting set x is sequentially input to the input layer 123. In the input layer 123, the time-series signal is divided into four vector component optical signals. The operation of the hidden unit 12, the transformation unit 15, etc., is the same as described above (however, the four optical signals are processed individually). The calculation result signal based on the final time-series signal is branched to the output side by the optical branching unit 13, received by a photodetector (not shown), and the label corresponding to the port with the highest light intensity is used as the identification label for the handwritten digit image. In the machine learning before testing, the phase shift amounts of each phase shifter in the input layer 123, optical adder unit 11, hidden unit 12, and control unit 124 were used as learning parameters, and the power branching ratio of the optical branching unit 13 was fixed (output side: conversion unit 15 side = 0.04:0.96). In the image identification simulation, the identification rate is evaluated.

[0061] Figure 15 shows the trajectory of training accuracy with respect to the RC delay coefficient α of the conversion unit 15. Here, the amplification factor β of the conversion unit 15 is set to 1, meaning the net signal is not amplified. Although the trajectory differs for each RC delay coefficient α, it can be seen that the training accuracy at the final epoch is almost the same. Figure 16 shows the dependence of the discrimination rate on the RC delay coefficient α. Similar to the training accuracy, it can be seen that the discrimination rate achieves approximately 80% regardless of the value of the RC delay coefficient α. From these results, it can be seen that the RNN device can operate even when there is an RC delay in the conversion unit 15, and that the RC delay does not degrade the learning accuracy.

[0062] Figure 17 shows the trajectory of training accuracy with respect to the RC delay coefficient α when the amplification factor β = 0.8. Figure 18 shows the trajectory of training accuracy with respect to the RC delay coefficient α when the amplification factor β = 0.6. In both Figures 17 and 18, it can be seen that when α is small, the training accuracy is reduced compared to the case without loss due to the loss in the conversion unit 15 (loop loss of the calculation result signal). On the other hand, it can be seen that the training accuracy is improved when α is large. These results indicate that the integration effect of the calculation result signal due to the RC delay of the conversion unit 15 can compensate for the RNN learning accuracy reduced by the loss in the conversion unit 15.

[0063] (Configuration of the RNN device) As described above, the RNN device can have the following configuration.

[0064] (Configuration 1) A recurrent neural network device 10 comprising: a hidden unit 12 that sequentially performs calculations on first to nth target signals, each containing first to nth time-series signals (where n is an integer of 3 or more) that are input at first to nth discrete timings, and sequentially outputs first to nth calculation result signals indicating the calculation results; and a loop unit Lp that includes the first to n-1st calculation result signals output from the hidden unit into the second to nth target signals input to the hidden unit, wherein the loop unit Lp holds the mth calculation result signal output from the hidden unit among the first to n-2nd calculation result signals by RC delay and includes it into the m+1st target signal together with the m+1st calculation result signal (where m is an integer satisfying 1 ≤ m ≤ n-2).

[0065] With the above configuration, at least two past calculation result signals can be included in the signal to be calculated, suppressing a decrease in calculation accuracy (for example, no decrease in calculation accuracy, or a reduction in the degree of decrease in calculation accuracy). Furthermore, by setting m to each integer satisfying 1 ≤ m ≤ n-2, all past calculation result signals can be included in the signal to be calculated, further suppressing a decrease in calculation accuracy. These effects are achieved by RC delay. Conventionally, RC delay has been considered a factor that hinders calculation and should be eliminated as much as possible. However, the inventors of this application have found through the above simulation that RC delay does not adversely affect calculation accuracy, but rather can have a beneficial effect, leading to the above configuration. RC delay may be achieved by the internal resistance or parasitic capacitance of the elements constituting the loop section. Also, the hidden unit 12 and the loop section Lp may all be composed of electrical circuits.

[0066] (Configuration 2) The first to nth target signals and the first to nth result signals are optical signals, the hidden unit 12 is an optical circuit that sequentially performs calculations on the first to nth target signals, which are optical signals, and outputs the first to nth result signals as optical signals, the loop Lp includes a photoelectric-to-optical conversion unit 15, the photoelectric-to-optical conversion unit 15 includes a photoelectric converter 15A that converts each of the first to n-1st result signals from optical signals to electrical signals, an RC circuit 15Z that holds the m result signal from the first to n-1st result signals converted to electrical signals by the photoelectric converter by RC delay, and adds the held m result signal to the m+1 result signal to generate an added signal, and an electro-to-optical converter 15E connected in parallel to the RC circuit (for example, at least a capacitor 15D is connected in parallel) that converts the first result signal from an electrical signal to an optical signal and converts the added signal from an electrical signal to an optical signal, The recurrent neural network device according to configuration 1, wherein the loop Lp includes the first calculation result signal converted into an optical signal by the photoelectric converter as the second calculation target signal, and includes the summation signal converted into an optical signal by the photoelectric converter as the m+2 calculation target signal.

[0067] With the above configuration, even if the hidden unit 12 is an optical circuit, an RC delay can be introduced, thereby suppressing a decrease in calculation accuracy.

[0068] (Configuration 3) The recurrent neural network device according to Configuration 2, wherein the RC circuit 15Z includes a capacitor 15D, the capacitor 15D is charged with a voltage corresponding to the m calculation result signal to hold the m calculation result signal, the capacitor 15D is discharged until the m+1 calculation result signal is input, the m calculation result signal held by the capacitor 15D is attenuated, and the RC circuit 15Z adds the attenuated m calculation result signal to the m+1 calculation result signal to generate the added signal.

[0069] With the above configuration, the mth calculation result signal is attenuated and added, thus preventing the added signal voltage from becoming excessive. Furthermore, it is possible to distinguish between the mth calculation result signal and the m+1th calculation result signal within the added signal.

[0070] (Configuration 4) The recurrent neural network device according to Configuration 2 or 3, wherein the hidden layer and the loop section are configured such that the j-th operation result signal zj, which indicates the result of the operation on the j-th operation target signal (where j is an integer satisfying 2 ≤ j ≤ n) by the hidden layer, satisfies the following formula (A). However, W is the matrix used for calculations in the hidden layer, β is the amplification factor of the j-th calculation result signal output by the photoelectric photoconversion unit with respect to the j-th calculation result signal input to the photoelectric photoconversion unit, α is the RC delay constant of the RC circuit, and xj is the j-th time-series signal included in the j-th calculation target signal.

[0071] With the above configuration, all past calculation result signals are reflected in the calculation result signal, thus further suppressing the decrease in calculation accuracy.

[0072] (Configuration 5) A recurrent neural network device according to any one of Configurations 1 to 4, further comprising p (where p is an integer of 2 or more) electro-optical converters (e.g., 122A to 122D), wherein each of the first to nth time-series signals represents a vector with dimensions of p and includes p optical signals, each representing p values ​​that represent the vector, and the p electro-optical converters each convert p electronic data representing the p values ​​into the p optical signals.

[0073] With the above configuration, time-series signals with multiple dimensions can also be appropriately converted using electromagnetism.

[0074] (Scope of the Invention) The present invention is not limited to the embodiments and modifications described above. For example, the present invention includes various modifications to the embodiments and modifications described above that can be understood by those skilled in the art within the scope of the technical concept of the present invention. The configurations listed in the embodiments and modifications described above can be combined as appropriate to the extent that they do not contradict each other. It is also possible to delete any of the above configurations.

[0075] 10...Recurrent neural network device, 11...Optical summing unit, 12...Hidden unit, 12A...Phase shifter, 12G...Waveguide crossing, 12K...Optical circuit element, 13...Optical branching unit, 14...Output unit, 15...Photoelectric-to-photoelectric conversion unit, 15A...Photoelectric converter, 15B...Drive power supply, 15C...Load resistor, 15D...Capacitor, 15E...Electron-to-photoelectric converter, 15EA...Phase modulator, 15F...Reference light source, 15Z...RC circuit, 16...Loop delay, 90...External device, 110...Recurrent neural network device, 121...Light source, 122...Electron-to-photoelectric conversion section, 122A-122D...Electric Optical converter, 123...input layer, 124...control unit, 125...output layer, 126...photoelectric converter, 210...recurrent neural network device, 310...recurrent neural network device, 410...recurrent neural network device, 510...recurrent neural network device, 511...addition / distribution unit, 610...recurrent neural network device, 611...directional coupler, 612...phase shifter, 613...Mach-Zehnder interferometer, ES...electrical signal, G...handwritten digit image, Lp...loop section, UC...unit cell, x...set of time-series signals.

Claims

1. A recurrent neural network device comprising: a hidden unit that sequentially performs calculations on first to nth target signals, each containing first to nth time-series signals (where n is an integer of 3 or greater) that are input at first to nth discrete timings, and sequentially outputs first to nth calculation result signals indicating the calculation results; and a loop unit that includes the first to n-1st calculation result signals output from the hidden unit into the second to nth target signals input to the hidden unit, wherein the loop unit holds the mth calculation result signal output from the hidden unit among the first to n-2nd calculation result signals by RC delay and includes it together with the m+1st calculation result signal into the m+1st target signal (where m is an integer satisfying 1 ≤ m ≤ n-2).

2. The first to nth target signals and the first to nth result signals are optical signals, the hidden unit is an optical circuit that sequentially performs calculations on the first to nth target signals, which are optical signals, and outputs the first to nth result signals as optical signals, the loop section includes a photoelectric-to-optical conversion unit, the photoelectric-to-optical conversion unit includes a photoelectric converter that converts each of the first to (n-1)th result signals from optical signals to electrical signals, an RC circuit that holds the m result signal from the first to (n-1)th result signals converted to electrical signals by the photoelectric converter by RC delay, and adds the held m result signal to the m+1 result signal to generate an added signal, and an electro-to-optical converter connected in parallel to the RC circuit that converts the first result signal from an electrical signal to an optical signal and converts the added signal from an electrical signal to an optical signal, The recurrent neural network device according to claim 1, wherein the loop portion includes the first calculation result signal converted into an optical signal by the photoelectric converter as the second calculation target signal, and includes the summation signal converted into an optical signal by the photoelectric converter as the m+2 calculation target signal.

3. The recurrent neural network device according to claim 2, wherein the RC circuit includes a capacitor, the capacitor is charged with a voltage corresponding to the m calculation result signal to hold the m calculation result signal, the capacitor discharges before the m+1 calculation result signal is input to attenuate the m calculation result signal held by the capacitor, and the RC circuit adds the attenuated m calculation result signal to the m+1 calculation result signal to generate the sum signal.

4. The hidden layer and the loop section are configured such that a j-th operation result signal z indicating an operation result of an operation on a j-th operation target signal (where j is an integer satisfying 2 ≤ j ≤ n) by the hidden layer j satisfies the following formula (A). The recurrent neural network device according to claim 2. Here, W is a matrix used for an operation in the hidden layer, β is an amplification factor of the j-th operation result signal output by the optoelectro-optical conversion unit with respect to the j-th operation result signal input to the optoelectro-optical conversion unit, α is an RC delay constant of the RC circuit, and x j is a j-th time series signal included in the j-th operation target signal.

5. The recurrent neural network device according to claim 1, further comprising p (where p is an integer of 2 or more) electro-optical converters, each of the first to nth time-series signals representing a vector of dimension p and including p optical signals each representing p values ​​representing the vector, and each of the p electro-optical converters converts p electronic data representing the p values ​​into the p optical signals, respectively.

Images