Anti-code pattern effect training method of time division multiplexing optical neural network

Abstract

The invention discloses an anti-code pattern effect training method of a time division multiplexing optical neural network, and belongs to the technical field of information. According to the method, the analysis of a high-speed optical signal time sequence input sequence in the nonlinear activation process or the complete modeling of a physical model of a numerical value is carried out, and the model is coupled into a neural network training algorithm based on back propagation. Specifically, when loss function back propagation training is performed to derive a nonlinear activation function, a code pattern effect physical model is completely coupled into the nonlinear function. When a high-speed time division multiplexing optical neural network architecture is trained, network training can still keep good performance under the condition that a code pattern effect objectively exists. Compared with a general back propagation training method which is directly used, the optical neural network trained by the method is good in convergence performance, and the recognition accuracy is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method for training a time-division multiplexed optical neural network to resist pattern effect. Background Art

[0002] Since the advent of integrated circuit technology in the mid-twentieth century, electronic computing systems have been guided by Moore's Law, continuously driving computing power growth through transistor miniaturization and increased chip integration. Traditional electronic processors, based on the von Neumann architecture, have enabled the large-scale development of general-purpose computing, supporting breakthroughs in fields such as artificial intelligence and big data processing. However, as process nodes approach physical limits, quantum tunneling and leakage currents caused by parasitic resistance have surged, leading to a stagnation in transistor switching energy efficiency and clock frequency increases, posing a fundamental challenge to the continuation of Moore's Law. Furthermore, the "memory wall" problem caused by the separation of memory and computing architectures has resulted in data transfer accounting for over 60% of energy consumption, severely limiting system energy efficiency. Furthermore, the RC delay and bandwidth limitations of copper interconnects exacerbate communication bottlenecks in highly parallel computing scenarios.

[0003] In this context, optical computing, with its ultra-high speed, high parallelism, and low heat dissipation, has become a key technological path to breaking the energy efficiency boundaries of electronic computing. Optical signals propagate at the speed of light, and wavelength multiplexing can achieve data throughput in the terabytes per second (Tb / s). Its multi-dimensional multiplexing properties naturally complement the parallel multiplication and accumulation computations required by neural network algorithms. Optical computing devices directly perform matrix operations in the analog domain, bypassing analog-to-digital conversion overhead. The static zero-power nature of their passive optical components significantly reduces computational energy consumption. Among various multiplexing architectures, time-division multiplexing (TDM), which encodes large amounts of data information streams on a time scale, has attracted significant research attention due to its advantages, including independent dependence on physical carriers, high reconfigurability, and low energy consumption. However, when timing signals are too fast, inter-symbol crosstalk (I / SCD) in computing systems inevitably leads to a decrease in system accuracy. The demand for higher computing speeds leads to even more severe I / SCD, necessitating the need for effective countermeasures. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention provides a method for training time-division multiplexed optical neural networks to resist pattern effects. This method fully models the crosstalk experienced by high-speed signals as they enter a nonlinearly activated device in a time-sequential manner. This model is then coupled to the nonlinear function during backpropagation training, where the derivative of the nonlinear function is derived. Compared to directly applying conventional backpropagation training algorithms, the time-division multiplexed optical neural networks trained with this method exhibit superior convergence performance and significantly improved recognition accuracy.

[0005] The present invention discloses a method for training a time-division multiplexed optical neural network against pattern effects, which comprises the following steps:

[0006] A linear computation layer with an n×m weight matrix w accepts an m-element input vector from the previous layer and performs matrix-vector multiplication to produce an intermediate vector x with n elements. o =(x1x2…x n ); intermediate vector x o The time sequence encoding is performed element by element to obtain a complete time vector T. After the linear calculation layer, a time-division multiplexed nonlinear activation device is deployed as a nonlinear activation layer, which sequentially completes the nonlinear activation function of the elements in each time window and outputs a new result vector y = (y1y2…y n ) and its corresponding new time vector T y ;

[0007] The training method is a back propagation training method based on the gradient descent of the loss function. When performing back propagation training, the output result vector y=(y1y2…y n ) and the n-dimensional label vector e constitute the mean square error loss function L, and the back propagation method calculates the partial differential And combined with the learning rate η to iteratively update the weight matrix w′;

[0008]

[0009] Among them, the partial differential Expand using the chain rule:

[0010]

[0011] Define nonlinear derivatives When there is a pattern effect, the derivative of the j-th vector after the pattern effect correction is a vector containing j elements:

[0012] The corresponding β expression is a lower triangular matrix:

[0013]

[0014] During backpropagation training, when the loss function gradient descends through a nonlinear activation device with a coding effect, the partial differential is calculated according to the β expression The learning rate η is combined with the iterative updating of the weight matrix w′ to achieve the anti-code effect training of time-division multiplexed optical neural network.

[0015] In the expression of β, each matrix element except the diagonal element is a correction term after considering the crosstalk term. With these correction terms, the gradient can descend in the correct direction, so that the training of the time-division multiplexing neural network can achieve convergence in the scenario with crosstalk.

[0016] In a preferred embodiment of the present invention, at least one layer of the time-division multiplexing optical neural network adopts a time-division multiplexing architecture; the training method is used to train all layers of the time-division multiplexing optical neural network that adopt a time-division multiplexing architecture, wherein the layer that adopts a time-division multiplexing architecture can be an input layer, a hidden layer or an output layer, or multiple layers can all adopt a time-division multiplexing architecture.

[0017] In a preferred embodiment of the present invention, the intermediate vector x o Element-by-element temporal encoding to obtain the complete time vector T includes the following steps:

[0018] The intermediate vectors are arranged in time order from the beginning to the end, forming n-1 sequentially arranged time windows and t corresponding to the nth element. sp The light signal in each time window carries the numerical information of an element in the form of light intensity, and the duration of the light signal is t sp Each time window corresponds to the number of each element in the intermediate vector in its time sequence to achieve distinction; for different calculation speeds, each time window will have a different time interval Δt, and a higher speed calculation corresponds to a shorter t sp and Δt; all time windows and time intervals are arranged in sequence to form a complete time vector T.

[0019] In a preferred embodiment of the present invention, the nonlinear activation device is an active device.

[0020] In a preferred embodiment of the present invention, the nonlinear activation device receives an input signal, the input signal nonlinearly compresses the gain level of the nonlinear activation device, and the nonlinear activation device gains the input signal to perform a nonlinear mathematical operation on the signal.

[0021] In a preferred embodiment of the present invention, the pattern effect is the timing crosstalk introduced by the nonlinear activation device, which causes the optical signal intensities in different time windows to interfere with the later input signal caused by the earlier input signal.

[0022] In a preferred embodiment of the present invention, for nonlinear activation calculations with pattern effects, the output of the jth neuron becomes a composite function of the jth input and all previous neuron inputs. The output of the n neurons in the nonlinear activation layer after operation is expressed in the mathematical form of a vector as follows:

[0023]

[0024] Among them, f j (x1,x2,…,x j ) represents the nonlinear function of the jth neuron with coding effect.

[0025] This method fully couples the physical model of pattern effects into the nonlinear function during backpropagation training of the loss function, even when the nonlinear activation function is derived. When training a high-speed time-division multiplexed optical neural network architecture, network training can maintain good performance despite the presence of pattern effects. Compared to directly applying general backpropagation training methods, optical neural networks trained with this method exhibit superior convergence performance and significantly improved recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of the optical neural network architecture used in Example 1 of the present invention.

[0027] Figure 2 This is a schematic diagram of the time-sequential input of optical information flow into a nonlinear activation device.

[0028] Figure 3 This is the accuracy convergence situation when the optical neural network is trained using the present invention in Example 1 of the present invention.

[0029] Figure 4 This is a comparison of the recognition accuracy of the optical neural network trained using the present invention in Example 1 of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be described in detail below with reference to the accompanying drawings.

[0031] The present invention first describes the optical signal processing process in a time-division multiplexing optical neural network (TDM-ONN). Taking a hidden layer as an example, the optical signal processing process includes the following steps:

[0032] Step 1: The linear computation layer of the optical neural network with weight matrix w receives the input vector x from the previous layer input , realize matrix-vector multiplication w×x i , and generate an intermediate vector x o .

[0033] Step 2: This intermediate vector will be arranged in time order from the beginning to the end, with an intermediate vector x having n elements. o =(x1x2…x n) as an example, n sequentially arranged time windows will be formed. Specifically: 1. The light signal in each time window carries the numerical information of an element in the form of light intensity, and the duration of the light signal is t sp ; 2. Each time window corresponds to the number of each element in the intermediate vector in its time sequence to achieve distinction; 3. For different calculation speeds, each time window will have a different time interval Δt, and a higher speed calculation corresponds to a shorter t sp and Δt; 4. All time windows and time intervals are arranged in sequence to form a complete time vector T.

[0034] Step 3: The time vector encoded in step 2 is input into a single nonlinear activation device. The single nonlinear activation device plays a role in different Linear The nonlinear activation function of the intermediate vectors generated between the calculation layers sequentially completes the nonlinear activation function of the elements on each time window; and outputs a new result vector y and its corresponding new time vector T y , the light intensity on each time window in the new time vector carries the information after nonlinear activation.

[0035] The nonlinear activation device is an active device that receives an input signal, which nonlinearly compresses the gain level of the nonlinear activation device. The nonlinear activation device gains the input signal to perform a nonlinear mathematical operation on the signal.

[0036] The nonlinear activation device can be deployed and used between any two linear computing layers.

[0037] In the above steps, nonlinear activation devices will introduce timing crosstalk, causing the optical signal intensity in different time windows to produce interference between earlier input signals and later input signals, namely, the code effect. The faster the calculation speed, the more significant the code effect.

[0038] In one possible form of temporal crosstalk, photons from the optical signal in the previous time window interact with and consume carriers in the nonlinear activation device (active device). Because the time intervals between multiple time windows are too short, and the time intervals between optical signals entering the active device are too short, the optical signal in the previous time window consumes carriers in the device. When the optical signal in the next time window enters the active device, the carriers in the active device cannot be fully replenished in time, resulting in the next input optical signal being unable to obtain full nonlinear activation gain.

[0039] Specifically, for an intermediate vector x o =(x1x2…x n), will be represented by a time vector T xo =(t1t2…t n ) carries its element information, taking t1 as an example, which contains a fixed time length t sp and Δt,t sp The light signal intensity on is x1, Δt does not carry information, and the light intensity on it is 0. The new result vector y generated after nonlinear activation and its time vector T y The corresponding rules of are the same as those of x.

[0040] Considering that the nonlinear activation device has a nonlinear activation function f(x), when there is no coding effect, the output of the ordinary nonlinear operation is:

[0041]

[0042] In the above formula, each vector element x i will have the same non-linear activation function.

[0043] When performing nonlinear activation calculations with timing crosstalk (code effect), due to the forward and backward code effect of the timing, the nonlinear function of the jth neuron is changed from y in the case of no code effect to j =f(x j ) becomes y j =f j (x1,x2,…,x j ), that is, due to the existence of the pattern effect, the output of the jth neuron becomes a composite function of the jth input and all previous neuron inputs. The output of the n neurons after the nonlinear activation layer is expressed in the mathematical form of a vector as follows:

[0044]

[0045] For the j-th output in this function, all input elements except the corresponding j-th input element are crosstalk terms.

[0046] Let's write the complete gradient descent matrix operation process:

[0047] The present invention couples the crosstalk term into the back-propagation algorithm. The principle of the back-propagation algorithm is to update the weight matrix values in the linear calculation layer of the optical neural network by descending the loss function along the gradient direction. Now, using the scenario of time-division multiplexing nonlinear activation devices deployed between the hidden layer and the output layer, the present invention illustrates how to make the gradient of the optical neural network descend along the optimal solution when there is a code effect.

[0048] Consider the first hidden layer with weight matrix w1, which accepts the input vector x from the previous layer input , the operation obtains the intermediate vector xo , and then deploy time-division multiplexing nonlinear activation devices as nonlinear activation layers. When performing back-propagation training, the output result vector y=(y1y2…y n ) and the n-dimensional label vector e constitute the mean square error loss function L:

[0049]

[0050] The backpropagation algorithm needs to calculate the first hidden layer weight matrix w′1 after the loss function gradient descent, which requires calculating the partial differential And it is updated by the learning rate η, which can be expressed as:

[0051]

[0052] By using the chain rule, we can To expand:

[0053]

[0054] Notice Term, y versus x o The derivative operation involves the derivation of nonlinear functions. It is helpful to define the nonlinear derivative term If the nonlinear process has no pattern effect, then the nonlinear derivative of the jth vector is There is only one term, which is a standard vector; the expression of β is as follows:

[0055]

[0056] As mentioned above, when the vector elements of the previous time input produce a pattern effect in the nonlinear process, the nonlinear derivative of the jth vector should include the partial derivatives of the jth output with respect to the 1st to jth input items. Then, the derivative of the jth vector after the pattern effect correction changes from only one element to The form of becomes a vector term containing partial derivatives with respect to other inputs:

[0057]

[0058] The corresponding β expression will change from a one-dimensional vector to a two-dimensional lower triangular matrix:

[0059]

[0060] Each matrix element except the diagonal elements is a correction term after considering the crosstalk term. With these correction terms, the gradient can descend in the correct direction, so that the training of the time-division multiplexing neural network can achieve convergence in the scenario with crosstalk.

[0061] The present invention is further described below with reference to specific embodiments.

[0062] Example 1

[0063] Take a time-division multiplexing optical neural network architecture to which the method of the present invention is applicable as an example. Figure 1 As shown, the first hidden layer has a weight matrix of 784×128 dimensions, and the second output layer has a weight matrix of 128×10 dimensions. The first hidden layer uses a semiconductor optical amplifier (SOA) to perform nonlinear activation on the time vector T encoded in the time series, and the second output layer uses the Sigmoid function commonly used in neural networks for classification output. The function of the entire optical neural network in this embodiment is to classify and recognize 28×28 handwritten digit images from the National Institute of Standards and Technology (MNIST) of the United States.

[0064] The time vector T from the first hidden layer is input into SOA as shown below: Figure 2 When the 128 elements of the time vector are input into the SOA at high speed, each element will cause crosstalk to the subsequent input elements. This is because the gain consumption cannot be fully compensated when the next input signal enters.

[0065] The present invention now illustrates how to combine the entire crosstalk process with the gain dynamic equation of the SOA and analytically introduce the entire crosstalk process into the back-propagation equation (7) to achieve the effects of crosstalk resolution and neural network gradient descent.

[0066] The input-output characteristic of SOA is y = g(x, t) x, and the gain dynamic equation is:

[0067]

[0068] Where g is the gain of SOA, g0 is the small signal gain of SOA, τ C is the carrier lifetime of SOA, E sat =P sat ×τ c is the saturation energy of SOA, P sat is the saturation optical power, Pin(x,t) is the time and intensity information of the time window:

[0069]

[0070] Now let's take the first two input signals as an example to illustrate how the second input signal is affected by the pattern effect of the first input signal. After the first input optical signal enters the nonlinear activation device, after a time interval Δt, the second input optical signal enters the nonlinear activation device. At this time, the gain of the nonlinear activation device is determined by two processes, namely, the gain of the first input signal x o1Start at t sp x is input during the time o1 Consumption process, and exponential recovery process within Δt time, combined with equation (8) to solve the gain dynamic equation (9) in these two processes, let the moment when the first input optical signal enters the nonlinear activation device be t = 0. Since there is no signal entering before, the gain is not consumed. The boundary condition at this time is g(t = 0) = g0, then the first input signal x o1 The gain received is the sP Compression of the complete small signal gain g0 at all times:

[0071]

[0072] The corresponding output signal is:

[0073] y1=x o1 ·g(x o1 ,t sp ) (12)

[0074] Formula (11) is to solve the sp From time t sp The boundary condition of the gain dynamic equation at time +Δt is that there is no signal in the time interval Δt, so Equation (9) can be simplified to:

[0075]

[0076] Solve equation (13), and finally get sp The initial gain g(x o1 ,Δt,t sp ):

[0077]

[0078] This formula shows that the gain will be given by g(x o1 ,t sp ) in terms of time index Restore to g0, and at the same time, it also shows that the second input signal x o2 The gain received is the sp Compression of Equation (14) at the moment:

[0079]

[0080] It is easy to observe from equations (11) and (14):

[0081] 1) A shorter Δt will result in a smaller time recovery term Indicates that the gain recovery time is short.

[0082] 2) Larger x o1 This will make the gain compression effect at input more significant.

[0083] These two points mean stronger timing crosstalk effects.

[0084] Comparing formula (11) and formula (15), we can see that x o2 The gain g0 becomes g(x o1 ,Δt,t sp ), which is x o1 Passed to x o2 Specifically, Equation (15) shows that at x o2 When passing through SOA, the gain it receives will be x o2 For x o1 and t sp The gain g(x o1 ,Δt,t sp ) is compressed. After knowing the gain of each input signal, any For the subsequent timing input x o3 、x o4 ,…,x on , can refer to x o1 to x o2 The crosstalk calculation method of is continuously nested, and finally the gain change caused by the previous input for each input is obtained. At this point, all matrix elements of Equation (8) are solved.

[0085] Figure 3 The results show that the recognition accuracy of the optical neural network trained by the present invention and the optical neural network trained by the common back propagation algorithm changes with the number of training times during 80 back propagation trainings. It can be seen that the training of the present invention has the performance of maintaining network convergence.

[0086] Figure 4 shows that the carrier lifetime τ c =378.15ps, t sp =50ps, the recognition accuracy of the optical neural network trained using the present invention and the optical neural network trained using the common back propagation algorithm under different Δt conditions. It can be seen that in the presence of crosstalk, the present invention can still achieve the effect of anti-crosstalk, so that the network recognition accuracy can be maintained.

[0087] The above-described embodiment only expresses one embodiment of the present invention. If the input signal has no analytical solution, the algorithm can still be used when its numerical model is known. The description of this embodiment is relatively specific and detailed, but it should not be understood as limiting the scope of the patent of the present invention. For those skilled in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A time-division multiplexed optical neural network anti-code pattern effect training method, characterized in that: The training method comprises: The linear computation layer with weight matrix w accepts the input vector from the previous layer, which implements matrix-vector multiplication to produce an intermediate vector x with n elements. o =(x1x2…x n ); intermediate vector x o The time sequence encoding is performed element by element to obtain a complete time vector T. After the linear calculation layer, a time-division multiplexed nonlinear activation device is deployed as a nonlinear activation layer, which sequentially completes the nonlinear activation function of the elements in each time window and outputs a new result vector y = (y1y2…y n ) and its corresponding new time vector T y ; The training method is a back propagation training method based on the gradient descent of the loss function. When performing back propagation training, the output result vector y=(y1y2…y n ) and the n-dimensional label vector e constitute the mean square error loss function L, and the back propagation method calculates the partial differential And combined with the learning rate η to iteratively update the weight matrix w′; Among them, the partial differential Expand using the chain rule: Define nonlinear derivatives When there is a pattern effect, the derivative of the j-th vector after the pattern effect correction is a vector containing j elements: The corresponding β expression is a lower triangular matrix: During backpropagation training, when the loss function gradient descends through a nonlinear activation device with a coding effect, the partial differential is calculated according to the β expression The learning rate η is combined with the iterative updating of the weight matrix w′ to achieve the anti-code effect training of time-division multiplexed optical neural network.

2. The method for training a time-division multiplexed optical neural network against pattern effects according to claim 1, wherein: At least one layer of the time-division multiplexing optical neural network adopts a time-division multiplexing architecture; the training method is used to train all layers of the time-division multiplexing optical neural network that adopt a time-division multiplexing architecture.

3. The method for training a time-division multiplexed optical neural network against pattern effects according to claim 1, wherein: The intermediate vector x o Element-by-element temporal encoding to obtain the complete time vector T includes the following steps: The intermediate vectors are arranged in time order from the beginning to the end, forming n-1 sequentially arranged time windows and t corresponding to the nth element. sp The light signal in each time window carries the numerical information of an element in the form of light intensity, and the duration of the light signal is t sp Each time window corresponds to the number of each element in the intermediate vector in the order of its time to achieve distinction; for different calculation speeds, each time window will have different time intervals Δt and t sp , higher speed calculations correspond to shorter t sp and Δt; all time windows and time intervals are arranged in sequence to form a complete time vector T.

4. The method for training a time-division multiplexed optical neural network against pattern effects according to claim 1, wherein: The nonlinear activation device is an active device.

5. The method for training a time-division multiplexed optical neural network against pattern effects according to claim 1, wherein: The nonlinear activation device receives an input signal, and the input signal nonlinearly compresses the gain level of the nonlinear activation device. The nonlinear activation device gains the input signal to complete a nonlinear mathematical operation on the signal.

6. The method for training a time-division multiplexed optical neural network against pattern effects according to claim 1, wherein: The pattern effect is the timing crosstalk introduced by the nonlinear activation device, which causes the optical signal intensities in different time windows to interfere with the later input signal caused by the earlier input signal.

7. The method for training a time-division multiplexed optical neural network against pattern effects according to claim 1, wherein: For nonlinear activation calculations with pattern effects, the output of the jth neuron becomes a composite function of the jth input and all previous neuron inputs. The output of the n neurons in the nonlinear activation layer after operation is expressed in the mathematical form of a vector as follows: Among them, f j (x1,x2,…,x j ) represents the nonlinear function of the jth neuron with coding effect.

8. The method for training a time-division multiplexed optical neural network against pattern effects according to claim 1, wherein: The loss function uses the mean square error loss function L:

Images