Optical modulation neuron and all-optical diffractive neural network method for signal processing

Abstract

The application discloses an optical modulation neuron and a full-optical diffraction neural network method for signal processing, establishes an optical modulation neuron, constructs a full-optical diffraction neural network based on the optical modulation neuron and applies the full-optical diffraction neural network to signal processing; an input layer of the full-optical diffraction neural network comprises a laser source array and an encoding circuit; after application data is encoded and converted, light waves emitted by the laser source array are used as signals, that is, optical input signals are generated by the input layer of the full-optical diffraction neural network; diffraction propagation of the light waves between layers of the full-optical diffraction neural network is represented as linear transformation in a complex domain; after receiving the signals, the optical modulation neuron passes through a nonlinear activation unit, modulates the signals by using a linear modulation unit and then transmits the signals to a next layer; output is obtained through forward propagation, parameters to be trained of the network are optimized according to the output by using a back propagation algorithm, and a trained full-optical diffraction neural network is obtained. The technical scheme provided by the application has low power consumption and high performance.

Description

Technical Field

[0001] This invention belongs to the field of optical neural network technology, and relates to optical signal processing and deep learning technology, and particularly to an optical modulation neuron and all-optical diffraction neural network method for signal processing. Background Technology

[0002] Deep learning is one of the fastest-growing machine learning methods of the past decade. Its goal is to build a neural network that simulates the human brain's analytical and learning processes, mimicking the brain's working mechanisms to process data. In recent years, deep learning has made significant progress in machine learning in areas such as medical image analysis, speech recognition, language translation, and image classification.

[0003] The concept of deep learning originates from research on artificial neural networks. Deep learning learns the inherent patterns and hierarchical representations of sample data and builds neural networks that simulate the human brain's analytical learning process. Artificial neural networks consist of a large number of neurons. A neuron is a structural device that mimics the function of biological neurons, using input and output signals to simulate the input and output of biological neurons. Each input synapse of a neuron has corresponding weights. The input signal is multiplied and summed with the corresponding synaptic weights, and the result is input into a nonlinear activation function located at the neuron's output. After processing by the activation function, the neuron's output signal is generated. In artificial neural networks, each neuron corresponds to a network layer, and these neurons are connected in a hierarchical manner to form the neural network model. Signals are transmitted between the different network layers of the artificial neural network to simulate information transmission between neurons in a biological system.

[0004] Currently, electronic computing is the most important computational support for realizing deep learning models. However, electronic methods have inherent drawbacks: first, signals are prone to mutual interference, posing difficulties for neural networks requiring high-density connections; second, the energy demand is too high, resulting in high computational costs. For traditional deep neural networks, both the training and practical applications require enormous computational resources. Computer-based deep neural networks require substantial computational costs and resources.

[0005] Integrated circuit (IC) chips are the hardware carriers for training and testing mainstream neural network models today. Traditional neural networks can run on CPUs, GPUs, field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs). However, the above-mentioned electrical domain-based computing methods face challenges in several aspects, such as power consumption, bandwidth, and parallelism.

[0006] Optical computing technology boasts advantages such as two-dimensional parallel processing, high speed, large capacity, spatial transmission, and resistance to electromagnetic interference. Optical neural networks, which combine optical computing with neural networks, simulate neurons and synaptic connections using optical methods to construct neural network models. Their main characteristics are group parallelism, high interconnect density, high computing speed, and low power consumption. Currently, technologies for manufacturing neural network devices based on optical processes are emerging. Compared to traditional electrical domain technologies, light has inherent advantages in parallelism, bandwidth, and speed.

[0007] Although existing technologies have provided some physical implementations of optical neurons, existing optical neurons either lack nonlinear activation functions or require photoelectric conversion to achieve nonlinear functions, which greatly limits the speed and performance of optical neural networks. The literature (Lin X, Rivenson Y, Yardimci NT, et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 2018, 361(6406):1004-1008.) records that researchers at UCLA used 3D printing technology to create an all-optical diffractive neural network. The literature (LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324) records that MNIST (Mixed National Institute of Standards and Technology database) can perform handwritten MNIST digit recognition and imaging tasks. It is inexpensive and can build larger and more numerous network layers. However, due to the lack of nonlinear structure, its network performance is greatly limited. The Hong Kong University of Science and Technology proposed an all-optical neural network based on the electromagnetic induction transparency property to realize non-activation functions. Feldmann et al. from the Institute of Physics at the University of Münster, Germany, proposed an all-optical neural synapse system that uses wavelength division multiplexing (WDM) technology to realize a scalable loop architecture for photonic neural networks, enabling pattern recognition in the optical field. However, this network system is not yet suitable as an optical neural network for complex signal processing. Summary of the Invention

[0008] To overcome the shortcomings of the aforementioned optical neural network technologies, this invention designs a novel optical modulation neuron and proposes a new all-optical diffractive neural network based on this neuron. The aim is to leverage the advantages of optical computation in all-optical networks, while simultaneously enabling the network to learn and train, increasing network capacity, improving network performance, and providing reconfigurable characteristics. The optical modulation neuron and the all-optical diffractive neural network can be used for signal processing, improving the speed of signal processing.

[0009] In the all-optical diffraction neural network proposed in this invention, the input layer hardware includes a laser source array and an encoding circuit. Application scenario data (such as images) is encoded and converted, and then light waves are emitted from the laser source array as signals. The optical input signal generated by the all-optical diffraction neural network input layer propagates between layers through diffraction. This inter-layer diffraction propagation of light waves in the diffraction neural network is equivalent to a linear transformation in the complex domain. After receiving the signal, the neuron first passes it through a nonlinear unit, then modulates it using a linear modulation unit (spatial light modulator) before transmitting the signal to the next layer. When training a neural network, data (specifically, image data used for image classification and recognition) is fed into the input layer. The output is obtained through forward propagation, and the parameters to be trained on the network are optimized using a backpropagation algorithm based on the output. The modulation parameters of each neuron are optimized through the backpropagation algorithm.

[0010] The technical solution of this invention is:

[0011] A method for implementing an optically modulated neuron and an all-optical diffraction neural network includes the following steps: An optically modulated neuron is established, and an all-optical diffraction neural network is constructed based on this optically modulated neuron, which can be applied to signal processing.

[0012] 1) Establish optically modulated neurons;

[0013] This invention provides an optically modulated neuron and constructs an all-optical diffraction neural network based on this optically modulated neuron.

[0014] The optical modulation neuron includes a linear modulation unit and a nonlinear activation unit. The linear modulation unit adds linear modulation to the amplitude / phase of the received optical signal, which is equivalent to a multiplicative operator in the complex domain, i.e., the parameters that the network needs to train. This unit is implemented by a spatial light modulator. The nonlinear activation unit performs nonlinear modulation on the received optical signal. This module can be implemented based on optical nonlinear effects, such as semiconductor optical amplifier (SOA), saturated absorption, Kerr effect, self-phase modulation, etc.

[0015] The linear modulation unit in the optical modulation neuron is implemented by a spatial light modulator (SLM). A spatial light modulator is a programmable device that dynamically changes the characteristics of a light wave in time and space. It modulates the input light wave signal according to a certain functional relationship based on a control signal, and the changes in the input and output light signals can be one or more parameter values ​​among the following attributes: amplitude, phase, polarization state, wavelength, and coherence. Here, we only use a spatial light modulator to modulate the amplitude and phase information of the input light. In mathematical operations, the linear modulation of the SLM is equivalent to a multiplicative operator in the complex domain, and the mathematical expression is: S Lout =C(x, y, z)·S Lin S Lin The input optical signal for the SLM, S Lout Let C(x, y, z) be the output optical signal of the SLM, and C(x, y, z) be the modulation information added by the SLM control unit. In this system, (x, y, z) represents the three-dimensional coordinates of the SLM in the network. C(x, y, z) is a complex number containing both amplitude and phase modulation components, and C(x, y, z) = C·e jφ , where C and φ represent amplitude and phase modulation information, respectively, and are the parameters that the network needs to learn.

[0016] The nonlinear activation unit in the optically modulated neuron utilizes optical nonlinearity to perform a nonlinear transformation on the input light wave signal, i.e., S Nout =σ(S Nin ), where S Nin S represents the input optical signal of the nonlinear activation unit. Nout This represents the output optical signal of the nonlinear activation unit. The implementation of this nonlinear unit is based on optical nonlinear effects and can utilize semiconductor optical amplifiers (SOA), saturated absorption, self-phase modulation, optical Kerr effect, etc. Specific devices can be implemented using optical nonlinear devices such as GC-SOA, graphene, atomic gas cells, photonic crystals, tunable ring resonators, and Mach-Zehnder interferometers. The nonlinear function σ will also be determined by the specific device selected.

[0017] The selection of nonlinear devices in the optically modulated neuron must meet the following requirements: ① the function expression σ is known; ② the σ function curve satisfies nonlinear characteristics; ③ the σ function is differentiable. Based on these two points, the backpropagation algorithm can be used to update the network parameters during network training. In this invention, we select a third-order nonlinear optical material as the nonlinear device.

[0018] Third-order nonlinear optical materials exhibit the optical Kerr effect. When light waves pass through the medium, if the amplitude is strong enough, the electric field of the light wave itself will cause a change in the refractive index. This change is related to the local irradiance of the light wave, and the relationship between the refractive index and the irradiance is: n = n0 + n2I, where n0 is the linear refractive index, n2 is the second-order nonlinear refractive index, I is the irradiance of the light wave, and n is the refractive index of the medium. When light waves pass through the medium, a phase difference is generated. Where λ is the wavelength of the light wave, and l is the length of the medium. By modulating the time of the light wave, we can change its amplitude and phase structure. Here, we only use its phase structure, letting S... Nin and S Nout These are the input and output of the nonlinear activation unit, respectively, and its nonlinear transformation function is:

[0019]

[0020] 2) Construct an all-optical diffraction neural network based on optically modulated neurons;

[0021] A novel all-optical diffraction neural network was constructed based on the aforementioned optically modulated neurons, comprising an input layer, a hidden layer, an output layer, and a processing module. This network addresses the lack of nonlinearity in existing all-optical diffraction neural networks, thereby improving the performance and applications of all-optical diffraction neural networks.

[0022] The input layer is a signal transmission module, consisting of a laser source array and a control circuit. The control circuit encodes the input digital signal, and the encoded signals are emitted by multiple laser sources, each using light waves of the same frequency. Different encoding schemes can be used in this module; for example, the signal can be encoded as the polarization state of the light wave, as the phase or time delay, or as the amplitude of the light wave.

[0023] The hidden layer comprises multiple layers, each consisting of an array of optically modulated neurons. All neurons are uniformly controlled by a control circuit. The optical neurons in each layer are responsible for receiving the coherent superposition signal of the light waves output by the neurons in the previous layer, modulating it, and then outputting it to the next layer. The light wave signals between the hidden layers propagate in a free-space diffraction manner. The control circuit controls the modulation parameters of the optically modulated neurons in each layer.

[0024] The output layer is determined by the specific downlink task of the network. For binary classification tasks, the output layer can be equipped with two photodetectors to receive the output light wave signal from the last hidden layer. By comparing the received light intensity of the photodetectors, the category of the input signal is determined. For imaging tasks, the output layer can be set as a blank receiving screen to receive the imaging pattern. For other tasks, suitable output layer modules can also be designed. The processing module analyzes the output of the entire network, which can be used for performance testing, as well as as a gradient calculation module, and the optical neural network is trained through gradient backpropagation.

[0025] In this network, signals are transmitted between adjacent layers through the diffraction of light waves, which can be viewed as matrix multiplication in the complex field. W represents the input of layer l and the output of layer (l-1), respectively. l-1,l The connection weight between the two layers is calculated according to the Rayleigh-Sommerfeld diffraction formula:

[0026]

[0027] Where l represents the l-th network layer, and i represents the position of the i-th node in the l-th layer (x, y, y). i ,y i ,z i The i-th neuron at position () where λ represents the wavelength of the light source. This weight is determined solely by the relative position of the neurons and the wavelength of the light wave.

[0028] This step is similar to convolution, but the elements in the convolution kernel are not independent of each other. They are determined by the relative positions of the preceding and following neurons, and the weights are calculated according to the formula above.

[0029] 3) Train the diffractive neural network constructed in step 2) using the training set data (the training set is a dataset collected and labeled according to the specific task of signal processing, such as an image classification dataset) to obtain the trained diffractive neural network.

[0030] During training, a portion of the image data in the training set is extracted and fed into the network's input layer. Based on the error between the network's predicted output and the standard result, the network's parameters are adjusted using a backpropagation algorithm. The control circuitry in the hidden layer is connected to the processing module, which adjusts the parameters of each neuron based on the training error obtained from the processing module, enabling the network to achieve the desired output. After training, the network can be used to process new data signals.

[0031] Alternatively, the above training steps can also be completed on a computer. The parameters obtained from the training will be used by the control circuit to set the neurons and then to process new data signals.

[0032] For the aforementioned all-optical diffraction neural network based on optical modulation neurons, further, before network training, the number of hidden layers, the number of optical modulation neurons in a single hidden layer, the network layer spacing, and the loss function for the training task are determined, and the network is assembled according to these parameters. During network training, a portion of training data is extracted each time, encoded into optical signals by the encoding circuit of the input layer, and transmitted to the network for processing. Based on the loss function and the backpropagation algorithm, the control module adjusts the modulation information of the optical modulation neurons in the hidden layers to achieve the learning and training objective. Alternatively, the network model can be pre-trained by a computer, and the modulation information of the network layer neurons can be set according to the parameters obtained from the training.

[0033] In practice, a trained diffractive neural network is used to process the data signals collected in the actual scene; the output of the all-optical diffractive neural network is the corresponding signal processing result.

[0034] Furthermore, in this embodiment of the invention, the number of hidden layers will be determined by the complexity of the task and the performance requirements. Increasing the number of layers increases the depth of the network, which can be used to complete more complex tasks and improve performance.

[0035] Furthermore, in the aforementioned new diffractive neural network system, the neurons in the single-layer hidden layer are generally arranged in a square shape, and their number is determined by the complexity of the task and the performance requirements. Increasing the number of neurons in the single-layer hidden layer increases the breadth of the network, which can be used to complete more complex tasks, and the performance will also be improved.

[0036] Furthermore, for the aforementioned new diffractive neural network system, the input layer, hidden layer, and output layer are parallel to each other, with a short interval, generally between a few to tens of millimeters. The specific value depends on the specific situation and can also be used as network parameters for training.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] Existing technologies use 3D-printed neurons as optical neurons. These neurons utilize optical transmission elements obtained through a 3D printer. By controlling the thickness of the optical transmission elements during 3D printing, the transmission effect of the optical neuron on light signals within the network can be controlled. This invention replaces the existing 3D-printed optical transmission neurons with an optical modulation neuron system. In addition to phase modulation, amplitude modulation is added, and a nonlinear activation function is introduced. This invention has the following technical advantages:

[0039] (i) In the all-optical diffraction neural network described in this invention, the entire processing does not involve photoelectric conversion. Light is used to process data signals, ensuring the parallel and high-speed characteristics of optical computing. In addition, compared with electrical networks, this network also has the advantages of low power consumption and high efficiency.

[0040] (II) The optical modulation neuron system described in this invention uses a spatial light modulator (SLM) as a linear modulation unit. Compared with 3D-printed optical neurons, it can improve the diversity of light wave modulation and modulate the input light wave signal from multiple dimensions such as amplitude and phase. On the other hand, 3D-printed neurons will cause energy loss during light wave propagation, which is not conducive to increasing the number of network layers. SLM can make up for this shortcoming by applying an external voltage.

[0041] (III) The optically modulated neuron described in this invention incorporates a nonlinear activation unit and utilizes optical nonlinear effects and nonlinear optical elements to achieve a nonlinear activation function, giving the network nonlinear characteristics and transforming it from a simple linear network. This significantly increases the network capacity, enabling it to perform more complex tasks. On the same task, it exhibits superior performance compared to the original 3D-printed diffractive neural network.

[0042] (iv) The new diffractive neural network described in this invention uses a light-modulated neuron system and a control module, which are reprogrammable elements. The modulation parameters can be changed through the control module, so that the network has reconfigurable features. Compared with 3D printed neurons, which need to be reprinted each time, this saves resources and time.

[0043] (v) The novel diffractive neural network described in this invention is based on the diffraction connection between optical neurons, which has the advantages of being easy to build and easy to scale up, and is easier to build large-scale neural networks.

[0044] (vi) The performance of the new diffractive neural network described in this invention depends on the network depth and network width, that is, the number of hidden layers and the number of neurons in each hidden layer. When the depth is greater and the width is larger, the network performance described in this invention is also higher.

[0045] (vii) The novel diffractive neural network described in this invention can be applied to micro-optical network chips, integrating modules such as optical modulation neurons, input layers, and hidden layers onto a nanochip to realize a nanoscale diffractive neural network. Using the trained diffractive neural network, data signals in various specific scenarios can be processed, improving the parallelism and speed of signal processing. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the all-optical diffraction neural network system provided by the present invention.

[0047] Figure 2 A comparison diagram of the diffraction neural network layer provided by this invention and a standard neural network.

[0048] Figure 3 This is a schematic diagram of the structure of the optically modulated neuron provided in this invention. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the scope of the invention is not limited in any way.

[0050] In this embodiment, the task that the network needs to complete is the MNIST handwritten digit recognition task; as shown in the attached diagram. Figure 1 As shown, the novel diffractive neural network system based on optically modulated neurons proposed in this invention includes: an input layer, multiple hidden layers, an output layer, and a processing module. The input layer is a signal emission module, composed of a laser source array and an encoding circuit. The control circuit encodes the input digital signal, and the encoded signal is emitted by the laser source array. In this task, we use an amplitude modulation scheme. The data in the MNIST handwritten digit set is a two-dimensional image, which is converted into a two-dimensional matrix, where each element represents the pixel grayscale value at that location. The hidden layer consists of a control module and an array of optically modulated neurons. The optically modulated neurons receive the signal after coherent superposition of the output light waves of the previous layer's neurons, modulate it, and output a light wave to transmit the signal to the next layer. The optically modulated neurons include two parts: a linear modulation unit and a nonlinear activation unit. The linear modulation unit is a spatial light modulator, and the nonlinear activation unit is implemented using a third-order nonlinear optical material with an optical Kerr effect. The control module controls the specific modulation parameters of each phase modulator neuron. The output layer receives the output signal of the highest hidden layer and collects the received optical power information at each point.

[0051] After the encoding circuit encodes the input signal, the laser source array in the input layer emits an optical signal. The signal propagates between layers through the diffraction of the light wave. It is processed by optical modulation neurons in the hidden layer and propagates to the first layer. Finally, the received optical power information at each point is obtained in the output layer. The processing module performs a simple analysis to obtain the image classification and recognition result (i.e., the recognition result of digital images).

[0052] In this embodiment, the laser frequency is 0.4 THz;

[0053] In this embodiment, the input layer laser source array is 28*28, and the image resolution in the MNIST dataset is also 28*28. Each laser source corresponds to the pixel grayscale value of a point in the image.

[0054] In this embodiment, there are five hidden layers with a 3cm interval between layers. Each hidden layer contains 28*28 optically modulated neurons with a 4mm interval between neurons.

[0055] In this embodiment, the processing module divides the output layer into 10 sub-regions, each sub-region corresponding to a category, calculates the sum of the received optical power of each sub-region, and classifies the image into the category corresponding to the sub-region with the maximum received optical power.

[0056] In this embodiment, the structural diagram of the light-modulated neuron is as follows: Figure 3 As shown, the linear modulation unit (SLM) of the optical modulation neuron performs linear modulation on the amplitude and phase of the light wave, with parameters C(x, y, z) = C·e jφ The output light waves of each neuron are perpendicular to the network layer plane and point towards the next layer. The nonlinear activation unit utilizes a third-order nonlinear optical material with optical Kerr effect.

[0057] In this embodiment, the network is first trained computationally, and the optical modulation neurons are configured based on the parameters obtained from the training.

[0058] Specifically, the steps include the following:

[0059] Step 1: Network training process, operations performed:

[0060] Step 1a: Determine the task the network needs to perform: MNIST handwritten digit recognition; collect data: MNIST handwritten digit image set, and divide the data into training set, validation set and test set, with a quantity of 55,000, 5,000 and 10,000 respectively.

[0061] Step 1b: Determine the number of hidden layers, the interlayer spacing, the number of neurons in a single hidden layer, the size and spacing of neurons, and determine the network training parameters. Taking phase modulation as an example, the parameter is the additional phase of the light wave by the spatial light modulator.

[0062] Step 1c: Determine the loss function for this task. In the processing module, substitute the total received power of each sub-region into the softmax function to convert the power into the probability of each sub-class. The loss function is the cross-entropy function of the predicted probability and the data label. The data label is a one-hot vector.

[0063] Step 1d: Extract a fixed number (100) of data from the training set each time and populate it into the network. Update the network parameters using the loss function and backpropagation algorithm described above: the phase modulation parameters of the spatial light modulator in the light modulation neuron; validate the network performance on the validation set data. Repeat this step until the network performance on the validation set data begins to decline.

[0064] Step 2: Network application process, perform the following operations:

[0065] Step 2a: Based on the parameters in Step 1, construct the hidden layer and assemble the network according to the predetermined interlayer spacing, ensuring that each layer is parallel and the center line is perpendicular to the plane, ensuring that the laser source beam is perpendicular to the input layer plane, and the diffraction propagation of the hidden layer is as follows. Figure 2 The right side of the image is shown ( Figure 2 A schematic diagram of the propagation of a standard neural network is also provided. Figure 2 (Left side of the middle section) In a standard neural network, the weight matrix for interlayer propagation is the parameter that needs to be trained, while in a light diffraction network, the interlayer propagation is through light diffraction, and the received signal of the next layer is the superposition signal of the light from the previous layer at that point.

[0066] Step 2b: For images that need to be classified or identified, their pixel information is encoded as the amplitude of the light signals emitted by each source in the laser source array;

[0067] Step 2c: Turn on the laser source, propagate the light wave signal into the network, and collect the received optical power information at each point on the output layer;

[0068] Step 2d: Based on the optical power information on the output layer, the processing module calculates the total power of each sub-region, finds the sub-region with the highest power, and the category to which it belongs is the image digital category;

[0069] Step 3: Network reconstruction process. If the network needs to perform other functions or its performance is poor, its parameters need to be adjusted. We only change the constant additional phase of each optical modulation neuron. Execution:

[0070] Step 3a: Repeat steps 1b-1d in step 1 to redetermine the network's training parameters;

[0071] Step 3b: Reset the modulation parameters of each optical modulation neuron through the control circuit of the hidden layer;

[0072] Step 3c: Execute steps 2b-2c in step 2 on the new task.

[0073] In this invention, the interlayer diffraction in step 2c satisfies the Rayleigh-Sommerfeld diffraction formula:

[0074]

[0075] Where l represents the l-th network layer, and i represents the position of the i-th node in the l-th layer (x, y, y). i ,y i ,z i The i-th neuron at position () where λ represents the wavelength of the light source.

[0076] In this embodiment, the final image classification accuracy on the MNIST validation set is 95.8%.

[0077] The purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the claims.

Claims

1. A light-modulated neuron device for signal processing, characterized in that, The optical modulation neuron includes a linear modulation unit and a nonlinear activation unit; The linear modulation unit adds linear modulation to the amplitude / phase of the received optical signal, that is, adds a multiplicative operator in the complex field; The linear modulation unit is implemented by a spatial light modulator; the spatial light modulator is used to modulate the amplitude and phase information of the input light; the amplitude and phase information of the light are the parameters that the network needs to learn. The nonlinear activation unit performs nonlinear modulation based on the received optical signal; The nonlinear activation unit is implemented based on optical nonlinear effects; it utilizes time modulation of light waves to change the amplitude and phase structure of the light waves. The relationship between the refractive index of light and the irradiance of light as it passes through a medium is expressed as: Where n is the refractive index of the optical medium, n0 is the linear refractive index, n2 is the second-order nonlinear refractive index, and I is the irradiance of the optical wave. A phase difference is generated when light waves pass through a medium. : ;in, It is the wavelength of light. It is the length of the medium; The nonlinear activation unit utilizes optical nonlinear effects to perform nonlinear transformation on the input optical wave signal; Third-order nonlinear optical materials exhibiting the optical Kerr effect are used as nonlinear devices; time modulation of light waves is employed to alter their amplitude and phase structure; the nonlinear transformation function of the light wave phase structure is described. Represented as: in, and These are the input and output optical signals of the nonlinear activation unit, respectively.

2. A method for using an all-optical diffraction neural network based on optically modulated neurons, characterized in that, A light-modulated neuron is established, and an all-optical diffraction neural network is constructed based on this neuron and applied to signal processing. The input layer of the all-optical diffraction neural network includes a laser source array and an encoding circuit. The light wave emitted by the laser source array after encoding and conversion of the application data is used as the signal, i.e., the optical input signal is generated by the input layer of the all-optical diffraction neural network. The interlayer diffraction propagation of the light wave in the all-optical diffraction neural network is represented as a linear transformation in the complex domain. After receiving the signal, the light-modulated neuron passes it through a nonlinear activation unit, modulates it using a linear modulation unit, and then transmits the signal to the next layer. The output is then obtained through forward propagation, and the parameters to be trained in the network are optimized using a backpropagation algorithm based on the output, thereby obtaining a trained all-optical diffraction neural network. The process includes the following steps: 1) Establish optically modulated neurons, including linear modulation units and nonlinear activation units; The linear modulation unit adds linear modulation to the amplitude / phase of the received optical signal, that is, adds a multiplicative operator in the complex field; The linear modulation unit is implemented by a spatial light modulator; the spatial light modulator is used to modulate the amplitude and phase information of the input light; the amplitude and phase information of the light are the parameters that the network needs to learn. The nonlinear activation unit performs nonlinear modulation based on the received optical signal; The nonlinear activation unit is implemented based on optical nonlinear effects; it utilizes time modulation of light waves to change the amplitude and phase structure of the light waves. The relationship between the refractive index of light and the irradiance of light when light passes through a medium is expressed as: Where n is the refractive index of the optical medium, n0 is the linear refractive index, n2 is the second-order nonlinear refractive index, and I is the irradiance of the optical wave; a phase difference is generated when the optical wave passes through the medium. : ;in, It is the wavelength of light. It is the length of the medium; The nonlinear activation unit utilizes optical nonlinear effects to perform nonlinear transformation on the input optical wave signal; Specifically, a third-order nonlinear optical material exhibiting the optical Kerr effect is selected as the nonlinear device; the amplitude and phase structure of the light wave are altered by time modulation of the light wave; the nonlinear transformation function of the light wave phase structure... Represented as: in, and These are the input and output optical signals of the nonlinear activation unit, respectively. 2) Construct an all-optical diffraction neural network based on optical modulation neurons; the adjacent layers of the all-optical diffraction neural network transmit signals through the diffraction of light waves, that is, perform matrix multiplication in the complex field; The all-optical diffraction neural network includes an input layer, hidden layers, an output layer, and a processing module. The input layer is a signal emission module, comprising a laser source array and a control circuit. The control circuit encodes the input digital signal, and the encoded signal is emitted by multiple laser sources using light waves of the same frequency. Multiple encoding methods can be used. The control circuit controls the modulation parameters of the optical modulation neurons in each layer. The hidden layer contains multiple layers, each composed of an array of optical modulation neurons. Optical neurons in each layer receive the coherent superposition signal of the output light waves from the neurons in the previous layer, modulate it, and then output it to the next layer. The light wave signals between hidden layers propagate via spatial diffraction. The output layer settings are determined by the network's downlink task. The processing module analyzes the network output, which can be used for performance testing or gradient calculation, and trains the optical neural network through gradient backpropagation. 3) Train the all-optical diffraction neural network constructed in step 2) using the training set data to obtain the trained all-optical diffraction neural network; During training, a portion of the training set data is extracted and fed into the network input layer. Based on the error between the network's predicted output and the standard result, the parameters of the network to be trained are adjusted through the backpropagation algorithm. The control circuit in the hidden layer is connected to the processing module, and the parameters of each neuron are adjusted based on the training error obtained by the processing module, thereby obtaining the network output. The constructed all-optical diffraction neural network uses an optical modulation neuron system and a control module, which are reprogrammable elements. By changing the modulation parameters through the control module, the network has reconfigurable characteristics. During network training, a portion of training data is extracted each time, encoded into optical signals by the encoding circuit of the input layer, and transmitted to the network for processing. Based on the loss function and the backpropagation algorithm, the control module adjusts the modulation information of the optical modulation neurons in the hidden layer, thereby completing the training of the all-optical diffraction neural network. After training, the all-optical diffraction neural network can be used to process new data signals; or it can be trained on a computer to obtain parameters and then set up neurons through a control circuit to process new data signals; the trained diffraction neural network is used to process the application data signals, and the output of the all-optical diffraction neural network is the signal processing result.

3. The optical modulation neuron and all-optical diffraction neural network method for signal processing as described in claim 2, characterized in that, The linear modulation unit in the optical modulation neuron is implemented by a spatial light modulator; the spatial light modulator is a programmable device that can dynamically change the characteristics of light waves in time and space; the multiplicative operator of the spatial light modulator in the complex domain of the SLM is represented as: in, This is the input optical signal for the SLM. This is the output optical signal of the SLM. Modulation information added to the control unit of the SLM; Represents the three-dimensional coordinates of the SLM in the network; It is a complex number, encompassing both amplitude and phase modulation; ,in and These represent amplitude and phase modulation information, respectively.

4. The all-optical diffraction neural network method based on optically modulated neurons as described in claim 2, characterized in that, The nonlinear activation unit is based on optical nonlinear effects; the optical nonlinear effects specifically include one or more of semiconductor optical amplification, saturated absorption, Kerr effect and self-phase modulation.

5. The all-optical diffraction neural network method based on optically modulated neurons as described in claim 2, characterized in that, The requirements for the optical nonlinear devices used in the nonlinear unit include: ① Function expression It can be seen that, ② The function curve satisfies nonlinear characteristics; ③ The function is differentiable; the specific optical nonlinear devices used include one or more of the following: GC-SOA, graphene, atomic gas cell, photonic crystal, tunable ring resonator, and Mach-Zehnder interferometer.

6. The all-optical diffraction neural network method based on optically modulated neurons as described in claim 2, characterized in that, The all-optical diffraction neural network constructed based on optical modulation neurons uses the following encoding methods to encode the input digital signal: encoding the signal into the polarization state of the optical wave signal, encoding the signal in the phase or time delay, or encoding the signal in the amplitude of the optical wave. The output layer settings include: For binary classification tasks, the output layer is set up with two photodetectors to receive the output light wave signal from the last hidden layer. The category of the input signal is determined by comparing the intensity of the light received by the photodetectors. For imaging tasks, the output layer is set up as a blank receiving screen to receive the imaging pattern.

7. The all-optical diffraction neural network method based on optically modulated neurons as described in claim 2, characterized in that, In the all-optical diffraction neural network, signals are transmitted between adjacent layers through the diffraction of light waves, and matrix multiplication in the complex domain is performed; the connection weights between two layers... Calculated according to the Rayleigh-Sommerfeld diffraction formula, it is expressed as: in, Representing the Layer network layer, represent The upper layer is located The first One neuron, Indicates the wavelength of the light source. , The connection weights between the two layers are determined by the relative positions of the neurons and the wavelength of the light waves.

8. The all-optical diffraction neural network method based on optically modulated neurons as described in claim 2, characterized in that, Before training an all-optical diffraction neural network, determine the number of hidden layers, the number of optically modulated neurons in a single hidden layer, the network layer spacing, and the loss function for the training task.

9. The all-optical diffraction neural network method based on optically modulated neurons as described in claim 2, characterized in that, The training set data used to train the diffractive neural network constructed in step 2) is an image classification dataset. The trained all-optical diffractive neural network is then used for image classification processing.

Images