Error-adaptive optical diffractive neural network in-situ training method

Abstract

The application belongs to the technical field of optical computing, and discloses an error adaptive optical diffraction neural network in-situ training method, which comprises the following steps: based on the phase of each layer of phase modulation layer, calculating the optical field complex amplitude of the input sample complex amplitude after forward propagation on the rear surface of each layer of phase modulation layer U k and the optical field complex amplitude at the output plane U o ; based on U o calculating the error optical field P and performing three-dimensional central symmetry on the backward propagation light path without changing the phase distribution of each phase modulation layer, calculating the optical field complex amplitude after forward propagation of the complex amplitude of the symmetrical error optical field at the front surface of each phase modulation layer, and obtaining P the optical field complex amplitude backward propagated to the rear surface of each phase modulation layer U n+1‑k ’ ; based on U k and U n+1‑k ’ calculating the gradient of the loss function with respect to the loss function to update, performing the next round of training, and stopping until the preset training round or loss convergence is reached. The application can adapt to the error existing in the optical system itself.

Description

Technical Field

[0001] This invention belongs to the field of optical computing technology, and more specifically, relates to an in-situ training method for an error adaptive optical diffraction neural network. Background Technology

[0002] Optical computing offers unparalleled advantages over electronic computing in terms of energy consumption, processing speed, and processing power. Diffractive Optical Neural Networks (DONNs) have attracted widespread research due to their low cost, scalability, and low latency. DONNs simulate the computational process of neural networks using the physical properties of light, such as diffraction and interference, eliminating the need for complex circuits in traditional electronic chips and allowing matrix operations and inference tasks to be performed directly within the optical system. However, most current training methods for DONNs still require electronic computers to calculate their network weights. This limits the size and training speed of DONNs to the capabilities of electronic computers, and such networks lack self-learning capabilities and the ability to adapt to system errors.

[0003] To further reduce reliance on computers and improve network robustness, an in-situ training method for diffractive neural networks has been disclosed in the prior art. This method utilizes the principles of optical reciprocity and diffraction propagation to calculate the loss function and the gradients of each phase parameter by measuring the light field information during forward and backward propagation. However, its experimental system is complex. After building the forward propagation system, a corresponding backward propagation system needs to be precisely built. The forward and backward propagation systems must be strictly aligned to ensure the accuracy of gradient calculation. Therefore, its physical implementation remains highly challenging and susceptible to errors. As an improvement to the aforementioned in-situ training method, a simplified scheme that uses spatial symmetry to replace backward propagation with forward propagation has also been disclosed in the prior art. However, this scheme still requires strict alignment of the forward and backward propagation optical paths to function properly, which limits its further application.

[0004] Therefore, existing in-situ training methods for optical diffraction neural networks suffer from high alignment costs, significant susceptibility to errors, and difficulties in commercialization. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides an error-adaptive in-situ training method for optical diffraction neural networks. The purpose of this method is to enable the in-situ training of diffraction optical neural networks to adapt to the errors inherent in the system itself, rather than calibrating the system before training. This solves the problems of high alignment costs, significant susceptibility to errors, and difficulty in productization inherent in in-situ training methods for optical diffraction neural networks.

[0006] To achieve the above objectives, the present invention provides an in-situ training method for an error adaptive optics diffraction neural network, comprising:

[0007] S1, Phase based on each phase modulation layer Calculate the complex amplitude of the input sample. U i The complex amplitude of the optical field on the back surface of each phase modulation layer after the forward propagation of the optical diffraction neural network. U 1 、U 2 、 … 、U n and the complex amplitude of the output optical field at the output plane U o , n This represents the total number of phase modulation layers.

[0008] S2, Calculation U o Corresponding light intensity O With target light intensity T loss function between L about U o The gradient is used as the error optical field. P The error optical field P The reverse propagation optical path is made three-dimensionally centrosymmetric without changing the phase distribution on each phase modulation layer, so as to transform the reverse propagation optical path into a forward propagation optical path and obtain the error optical field. P Light field after two-dimensional centrosymmetry P ⊙ ;

[0009] S3. Calculate the light field. P ⊙ The complex amplitude of the optical field propagating forward to the front surface of each phase modulation layer U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ and will U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. P The complex amplitude of the optical field propagating backward to the back surface of each phase modulation layer U 1 ’ 、U 2 ’ 、…、Un ’ ; wherein, the error optical field P Backpropagation to the k The complex amplitude of the optical field on the back surface of the phase modulation layer is U n+1-k ’ ,1≤ k ≤ n ;

[0010] S4, based on U k and U n+1-k ’ Calculate the loss function L Regarding the aforementioned number k Layer phase modulation layer phase The gradient is calculated, and the update is performed based on that gradient. ;

[0011] S5, switch to S1 for the next round of training, until the preset number of training rounds is reached or the loss converges.

[0012] Further, in S3, the light field is calculated. P ⊙ The complex amplitude of the optical field propagating forward to the front surface of each phase modulation layer U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ and will U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. P The complex amplitude of the optical field propagating backward to the back surface of each phase modulation layer U 1 ’ 、U 2 ’ 、…、U n ’ ,include:

[0013] The light field P ⊙ Loaded in n On the phase modulation layer, the complex amplitude of the optical field is obtained at the output plane. U 1 ’⊙ ,Will U 1 ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. PBackpropagation to the n Complex amplitude of optical field on the back surface of the phase modulation layer U 1 ’ ;

[0014] Will U n-k ’⊙ Loaded in k On the phase modulation layer, at the first k The complex amplitude of the optical field is obtained at the front surface of the +1 phase modulation layer. U n+1-k ’⊙ Then in the first k The complex amplitude of the optical field obtained at the imaging surface of the +1 phase modulation layer is: U n+1-k ’⊙ With the k +1 layer phase modulation layer phase The product; will be in the first k The complex amplitude of the optical field obtained at the imaging surface of the +1 phase modulation layer is divided by ,get U n+1-k ’⊙ ,Will U n+1-k ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. P Backpropagation to the k Complex amplitude of optical field on the back surface of the phase modulation layer U n+1-k ’ ;1≤ k < n .

[0015] Furthermore, in S1, the space where the input sample is located is on the same plane as the first phase modulation layer; wherein, U 1 equals U i Phase with the first phase modulation layer The product of U k equal U k-1 After the first k -1st floor to the k The complex amplitude of the optical field after free-space diffraction between the phase modulation layers and the first layer k Layer phase modulation layer phase The product of 2 ≤ k ≤ n .

[0016] Furthermore, in S2, the error optical field P for:

[0017]

[0018] Among them, the loss function ;

[0019] In S4, the loss function L Regarding the aforementioned number k Layer phase modulation layer phase gradient for:

[0020] ;

[0021] in, real () represents the real part operation. j The imaginary unit, abs () performs the absolute value operation, and angle() performs the phase angle operation. U k The complex amplitude of the input sample U i Forward propagation to the 1st k Complex amplitude of the optical field on the back surface of the phase modulation layer U n+1-k ’ For error light field P Backpropagation to the k The complex amplitude of the optical field on the back surface of the phase modulation layer.

[0022] Furthermore, the optical diffraction neural network... k The forward propagation optical path of the layer includes the first layer set along the optical path. k Spatial light modulator of the first layer, k The 4f-system of the layer and the first k Layered CCD camera;

[0023] The first k The spatial light modulator of the layer is the first k A phase modulation layer is used to modulate the phase of the incident light; the incident light is used to provide a light source for training an optical diffraction neural network.

[0024] The first k The 4f-system of the layer is used to map the optical field of the interference light onto the first... k In front of the CDD camera of the layer, so that the optical diffraction neural network can generate a preset diffraction distance; wherein, the interference light is the light generated after the outgoing light after phase modulation by the spatial light modulator and the reference light generated by the incident light beam splitting are interfered;

[0025] The first k The CCD camera of the first layer is the kThe imaging surface of the phase modulation layer is used to collect the intensity distribution of the interference light in order to calculate the first... k The complex amplitude of the optical field on the back surface of the phase modulation layer.

[0026] Furthermore, before the incident light is incident on the first layer of spatial light modulator, a beam preprocessing module is also included to preprocess the incident light; wherein, the beam preprocessing module includes, along the optical path direction: a laser, a collimating lens, an optical attenuator, a beam expander, a half-wave plate, a polarizer, and a beam splitter.

[0027] The laser is used to generate the incident light;

[0028] The collimating lens is used to collimate the incident light;

[0029] The optical attenuator is used to reduce the intensity of the collimated incident light;

[0030] The beam expander is used to increase the size of the incident light spot after passing through the optical attenuator;

[0031] The half-wave plate and the polarizer are used together to adjust the polarization direction of the incident light to the working polarization direction of the spatial light modulator;

[0032] The beam splitter is used to split the incident light after the polarization direction has been adjusted. One beam is used as the reference light, and the other beam is incident on the first layer spatial light modulator for phase modulation.

[0033] Furthermore, a time-sharing generation method is adopted to calculate... U o , U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ ,include:

[0034] The time-division generation method is used to generate the complex amplitude of the optical field. U It is decomposed into two phase diagrams; the amplitudes of the two phase diagrams are equal and both are constants;

[0035] The two decomposed phase images are loaded onto the spatial light modulator twice, and the intensity distribution of the output interference light is measured at the CCD camera twice to obtain the corresponding complex amplitude of the optical field. The two complex amplitudes are then added together to obtain the complex amplitude of the optical field. Among them, the complex amplitude of the light field U For the complex amplitude of the light field U At time 1, the spatial light modulator is the first-layer spatial light modulator, the CCD camera is the output plane, and the complex amplitude of the light field... forU o ;

[0036] complex amplitude of light field U The light field P ⊙ At that time, the spatial light modulator is the first n The spatial light modulator, wherein the CCD camera is the output plane, and the complex amplitude of the light field... for U 1 ’⊙ ;

[0037] complex amplitude of light field U for U n-k ’⊙ At that time, the spatial light modulator is the first k The spatial light modulator, wherein the CCD camera is the first k The CCD camera with +1 layer, the complex amplitude of the light field for U n+1-k ’⊙ With the k The product of phases on the +1 layer spatial light modulator will Divided by the k +1 layer spatial light modulator phase, obtain U n+1-k ’⊙ ,1≤ k < n .

[0038] Furthermore, at the CCD camera, the intensity distribution of the interference light is measured using a four-step phase-shifting method;

[0039] Before each training session begins, the following steps are also included: first, measuring the phase of the incident light at the beam splitter; during each training session, the following steps are also included: U 1 、U 2 、 … 、U n 、U o 、U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The phase of the incident light at the beam splitter is subtracted from the phase of the incident light in the phase of the incident light, and the phase after the subtraction is taken as the phase of the incident light. U 1 、U 2 、 … 、U n、U o 、U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The corresponding phase.

[0040] The present invention also provides an in-situ training system for an error adaptive optical diffraction neural network, including a computer-readable storage medium and a processor;

[0041] The computer-readable storage medium is used to store executable instructions;

[0042] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the in-situ training method for the error adaptive optics diffraction neural network described above.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the in-situ training method for error adaptive optics diffraction neural networks as described in any of the preceding claims.

[0044] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0045] (1) The error adaptive optics diffraction neural network in-situ training method provided by the present invention is based on spatial rotation invariance. It achieves three-dimensional central symmetry of the back propagation optical path without changing the phase distribution on each phase modulation layer, that is, it achieves the error optical field. P The symmetry of each free-space diffraction segment and the intermediate optical field during backward propagation is maintained, but the phase distribution on each phase modulation layer remains unchanged. In this case, besides replacing backward propagation with forward propagation, the input optical field (error field) of the backward propagation is also affected. P The method is centrally symmetric, which ensures that the translation error (error caused by misalignment) caused by the optical axis not being located at the center of the input and output planes in the optical system cancels out the effects of the forward and backward propagation. This guarantees that the method of the present invention has the ability to adapt to system errors and solves the problems of high alignment cost, large influence of error, and difficulty in productization in the in-situ training method of optical diffraction neural network.

[0046] (2) Preferably, the space where the input sample is located is on the same plane as the phase modulation layer of the optical diffraction neural network, so that the light field on the back surface of the first phase modulation layer can be directly calculated by multiplying the complex amplitude of the input light field carrying the input sample information by the phase of the first phase modulation layer, thereby saving an imaging surface and making the system simpler.

[0047] (3) Furthermore, the components used in this invention are simple. Apart from passive components such as beam expanders and polarizers, only lasers, spatial light modulators and CCD cameras are used to realize in-situ training of optical diffraction neural networks, and the implementation method is simple.

[0048] (4) Preferably, the present invention uses the complex amplitude time-division generation method, which can generate arbitrary complex amplitudes simply and quickly. For each complex amplitude to be generated, only a single spatial light modulator is needed to meet the beam control requirements of in-situ training of optical diffraction neural network, and also avoids the strict alignment problem between the two spatial light modulators required to generate two phase maps.

[0049] (5) As a preferred method, the complex amplitude of the beam is measured using the four-step phase shift method. Each phase measurement during training will subtract the phase of the incident light itself, which can adapt to the inherent phase error in the optical system, such as the phase error of the laser itself, the phase added by the components in the optical path, etc.

[0050] In summary, in view of the problems of high alignment cost, large susceptibility to error and difficulty in productization in the in-situ training method of optical diffraction neural network in the existing technology, the present invention provides an optical diffraction neural network training method with simple components and adaptive system error. The training method of the present invention does not require alignment and has the ability to adapt to system error. Experiments have also proved that the optical diffraction neural network trained by the training method of the present invention can adapt to various errors existing in the optical path. Attached Figure Description

[0051] Figure 1 This is a flowchart of the in-situ training method for error adaptive optics diffraction neural network provided in the embodiments of the present invention.

[0052] Figure 2 This is a schematic diagram of the optical path structure of a single-layer optical diffraction neural network during in-situ training in an embodiment of the present invention.

[0053] Figure 3 This is a schematic diagram of the optical path structure of a multilayer optical diffraction neural network during in-situ training in an embodiment of the present invention.

[0054] Figure 4 This is a schematic diagram of the adaptive translation error in the in-situ training method of the error adaptive optics diffraction neural network provided in the embodiment of the present invention.

[0055] Figure 5 This is a schematic diagram of adaptive tilt error in the in-situ training method of error adaptive optics diffraction neural network provided in the embodiment of the present invention. In the figure, (a) is a schematic diagram when there is no tilt error, and (b) is a schematic diagram when there is tilt error.

[0056] Figure 6This is a schematic diagram of the complex amplitude time-division generation method provided in the embodiment of the present invention.

[0057] Figure 7 This is one of the application scenarios of the method in this embodiment of the invention: real-time face turning recognition. In the figure, (a) shows the different output results corresponding to face image inputs with different orientations, (b) shows the phase distribution of the optical diffraction neural network, and (c) shows the experimental image of real-time face turning recognition.

[0058] Figure 8 The figure shows the performance of the method provided in the embodiment of the present invention in real-time face turning. Figure (a) shows the recognition accuracy when the incident light is incident on the spatial light modulator at different tilt angles. Figure (b) shows the loss function decrease curve when the incident light is incident on the spatial light modulator at different tilt angles. Figure (c) shows the experimental schematic diagram when the incident light is incident at different tilt angles.

[0059] Figure 9 This describes the performance of the method in unsupervised learning application scenarios according to embodiments of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0061] Example 1

[0062] like Figure 1 As shown, this embodiment of the invention provides an in-situ training method for an error adaptive optics diffraction neural network, including:

[0063] S1, Phase based on each phase modulation layer Calculate the complex amplitude of the input sample. U i The complex amplitude of the optical field on the back surface of each phase modulation layer after the forward propagation of the optical diffraction neural network. U 1 、U 2 、 … 、U n and the complex amplitude of the output optical field at the output plane U o ;in, n This represents the total number of phase modulation layers. n ≥1, when nWhen = 1, it is a single-layer optical diffraction neural network, that is, the optical diffraction neural network contains only one phase modulation layer. In this embodiment of the invention, the rear surface of the phase modulation layer is the plane after the light field has been phase-modulated by the phase modulation layer in the forward propagation optical path.

[0064] S2, Calculate the complex amplitude of the output light field. U o Corresponding light intensity O With target light intensity T loss function between L about U o The gradient is used as the error optical field. P ; to adjust the error optical field P The reverse propagation optical path is made three-dimensionally centrosymmetric without changing the phase distribution on each phase modulation layer, so as to transform the reverse propagation optical path into the forward propagation optical path and obtain the error optical field. P Light field after two-dimensional centrosymmetry P ⊙ .

[0065] S3, Calculate the light field P ⊙ The complex amplitude of the optical field propagating through the forward propagation optical path to the front surface of each phase modulation layer U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ and will U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ By performing two-dimensional centrosymmetry, the error optical field is obtained. P The complex amplitude of the optical field propagating backward to the back surface of each phase modulation layer U 1 ’ 、U 2 ’ 、…、U n ’ Among them, the error light field P Backpropagation to the k The complex amplitude of the optical field on the back surface of the phase modulation layer is U n+1-k ’ ,1≤ k ≤ n .

[0066] S4. Complex amplitude based on input sample U iAfter passing through the forward propagation path of the optical diffraction neural network, at the first... k Complex amplitude of optical field on the back surface of the phase modulation layer U k and error light field P Backpropagation to the k Complex amplitude of optical field on the back surface of the phase modulation layer U n+1-k ’ Calculate the loss function L Regarding the first k Layer phase modulation layer phase The gradient of the first element is obtained, and the first element is updated based on that gradient. k Layer phase modulation layer phase .

[0067] S5. Proceed to S1 for the next round of training, until the preset number of training rounds or loss function is reached. L When convergence occurs, training ends, and the trained optical diffraction neural network is obtained.

[0068] Preferably, in S1, the space where the input sample is located is on the same plane as the first phase modulation layer; at this time... U 1 equals U i Phase with the first phase modulation layer The product of U k equal U k-1 After the first k -1st floor to the k The complex amplitude of the optical field after free-space diffraction between the phase modulation layers and the first layer k Layer phase modulation layer phase The product of 2 ≤ k ≤ n .

[0069] Specifically, in S1, the complex amplitude of the input sample U i for:

[0070] ;in, The input sample is a digital signal, specifically amplitude information. For the first k The phase of the phase modulation layer is initialized to the initial phase of the input sample.

[0071] As a specific implementation method, in S2, the error optical field P for:

[0072] Among them, the loss function .

[0073] In S3, the light field is calculated. P ⊙ The complex amplitude of the optical field propagating forward to the front surface of each phase modulation layer U 1 ’⊙ , U 2 ’⊙ , …、 U n ’⊙ and will U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ By performing two-dimensional centrosymmetry, the error optical field is obtained. P The complex amplitude of the optical field propagating backward to the back surface of each phase modulation layer U 1 ’ 、U 2 ’ 、…、U n ’ ,include:

[0074] light field P ⊙ Loaded in n On the phase modulation layer, the complex amplitude of the optical field is obtained at the output plane. U 1 ’⊙ ,Will U 1 ’⊙ By performing two-dimensional centrosymmetry, the error optical field is obtained. P Backpropagation to the n Complex amplitude of optical field on the back surface of the phase modulation layer U 1 ’ ;Will U 1 ’⊙ Loaded in n On the -1st phase modulation layer, at the... n The complex amplitude of the optical field is obtained at the imaging surface of the phase modulation layer. U 2 ’⊙ With the n The product of the phases of the phase modulation layers will... n The complex amplitude of the optical field obtained at the imaging surface of the phase modulation layer is divided by the first... n The phase of the phase modulation layer is obtained U 2 ’⊙ ,Will U 2 ’⊙ By performing two-dimensional centrosymmetry, the error optical field is obtained. P Backpropagation to the n The complex amplitude of the optical field on the back surface of the -1 layer phase modulation layerU 2 ’ ...and so on, to U n-1 ’⊙ The complex amplitude of the optical field is obtained at the imaging surface of the second phase modulation layer after being loaded onto the first phase modulation layer. U n ’⊙ The product of the phase of the second phase modulation layer and the phase of the second phase modulation layer, divided by the phase of the second phase modulation layer, yields the complex amplitude of the optical field at the imaging surface of the second phase modulation layer. U n ’⊙ ,Will U n ’⊙ By performing two-dimensional centrosymmetry, the error optical field is obtained. P The complex amplitude of the optical field propagating backward to the back surface of the first phase modulation layer U n ’ .

[0075] For multilayer optical diffraction neural networks, it is necessary to obtain the error optical field. P The complex amplitude of the optical field propagating backward to the surface of each phase modulation layer U 1 ’ 、U 2 ’ 、…、U n ’ In this embodiment of the invention, the backpropagation process of the error optical field is divided into several sub-processes: error optical field P Backpropagation from the output plane to the... n The complex amplitude of the optical field on the back surface of the phase modulation layer is U 1 ’ complex amplitude of light field U 1 ’ Multiply by the n After layer phase modulation of the layer phase (error optical field backpropagation to the first layer) n The intermediate light field on the back surface of the phase modulation layer propagates backward to the first... n -1 phase modulation layer obtained U 2 ’ ...and so on until the complex amplitude of the light field. U n-1 ’ Multiplying by the phase of the second phase modulation layer and then backpropagating to the first phase modulation layer yields... U n ’ Totaln Secondary back propagation. In this embodiment of the invention, for the intermediate optical field of each sub-process, three-dimensional centrosymmetry is first performed on the back propagation optical path, simultaneously realizing the transformation of the back propagation optical path into the forward propagation optical path and two-dimensional centrosymmetry of the back propagation optical field in this sub-process. The complex amplitude of the optical field... U k ’ Multiply by the n +1- k After the layer phase is modulated, the layer phase is backpropagated to the first layer. n - k Layer phase modulation layer obtained U k+1 ’ For example, 1≤k≤n-1, the optical path of this reverse propagation sub-process is subjected to three-dimensional centrosymmetry. After three-dimensional centrosymmetry, the reverse propagation optical path is transformed into the forward propagation optical path, and the reverse propagation optical path is transformed into the forward propagation optical path. U k ’ Multiply by the n +1- k The optical field after layer phase modulation is subjected to two-dimensional centrosymmetry to obtain... U k ’⊙ .calculate U k ’⊙ After the forward propagation optical path, at the th n - k At the imaging plane (the first) n - k The complex amplitude of the optical field of the first CCD is divided by the complex amplitude of the first CCD. n +1- k After layer phase modulation, the layer phase is obtained U k+1 ’⊙ and will U k+1 ’⊙ After performing two-dimensional central symmetry, we obtain U k ’ Complex amplitude multiplied by the first n +1- k After the layer phase is modulated, the layer phase is backpropagated to the first layer. n - k Complex amplitude of optical field on the back surface of the phase modulation layer U k+1 ’ By breaking down the complete backpropagation process into multiple backpropagation sub-processes and performing three-dimensional centrosymmetric operations on the backpropagation optical path in each sub-process, the method of this invention can be applied to multilayer optical diffraction neural networks.

[0076] As a specific implementation method, in S4, the loss function L Regarding the first k Layer phase modulation layer phase gradient for:

[0077]

[0078] in, For the first k Layer phase modulation layer phase, real () indicates the operation of taking the real part. j The imaginary unit, abs () represents the absolute value operation, and angle() represents the phase angle operation. U k The complex amplitude of the input sample U i Forward propagation to the 1st k Complex amplitude of the optical field on the back surface of the phase modulation layer U n+1-k ’ For error light field P Backpropagation to the k The complex amplitude of the optical field on the back surface of the phase modulation layer.

[0079] like Figure 2 As shown in this embodiment of the invention, for a single-layer optical diffraction neural network, its forward propagation optical path includes: a spatial light modulator, a 4f-system, and a CCD camera arranged along the optical path. Before the incident light is incident on the spatial light modulator, a beam preprocessing module is also included to preprocess the incident light. The beam preprocessing module includes, along the optical path direction, a laser, a collimating lens, an optical attenuator, a beam expander, a half-wave plate, a polarizer, a beam splitter, and a reflector. Figure 2 The spatial light modulator shown is a reflective spatial light modulator, but in other embodiments it can also be a transmissive spatial light modulator.

[0080] Lasers are used to provide a light source for training optical diffraction neural networks, generating incident light.

[0081] Collimating lenses are used to collimate incident light, reduce the divergence angle of the incident light, and make the transmission of incident light more stable.

[0082] An optical attenuator is used to reduce the intensity of incident light.

[0083] A beam expander is used to enlarge the size of the incident light spot so that it covers the modulation region on the spatial light modulator, while making the phase and amplitude of the incident light in the modulation region flatter, approximating parallel light incidence.

[0084] A half-wave plate is used to adjust the polarization direction of incident light to prevent the polarization direction of the incident light from being perpendicular to the plate, which would prevent light from passing through.

[0085] Polarizers are used to ensure that the polarization direction of the incident light is the same as the working polarization direction of the spatial light modulator.

[0086] Beam splitters are used to split incident light after it has passed through a polarizer into two beams. One beam is used as a reference beam, and the other beam is incident on a spatial light modulator for phase modulation. The spatial light modulator is the phase modulation layer mentioned above, and each pixel of the spatial light modulator is a neuron of an optical diffraction neural network.

[0087] The mirror is used to reflect the reference light and cause the reflected reference light to interfere with the outgoing light after phase modulation by the spatial light modulator, thus producing interference light.

[0088] The 4f-system is used to map the light field of the interference light onto the front of the CDD camera so that the optical diffraction neural network can generate a preset diffraction distance.

[0089] The CCD camera is used to collect the intensity distribution of the interference light. In practical applications, the above-mentioned in-situ training method can be mounted on a controller, which is connected to the spatial light modulators and CCD cameras of each layer. The controller updates the phase modulation amount of the spatial light modulator based on the beam information detected by the CCD camera, thereby modulating the phase of the incident light.

[0090] Correspondingly, the reverse propagation optical path is the optical path from the imaging surface (CCD camera) to the spatial light modulator (phase modulation layer).

[0091] For multilayer optical diffraction neural networks, its first k The forward propagation optical path of the first layer is similar to that of a single-layer optical diffraction neural network, including the first layer set along the optical path. k Spatial light modulator of the first layer, k The 4f-system of the layer and the first k A layered CCD camera. The beam preprocessing module is similar to the beam preprocessing module in a single-layer optical diffraction neural network, used to preprocess the incident light before it reaches the first-layer spatial light modulator, such as... Figure 3 As shown, Figure 3 The spatial light modulator is a transmissive spatial light modulator, therefore the optical path does not include a reflector.

[0092] Taking a single-layer optical diffraction neural network as an example, such as Figure 4 The diagram shown illustrates the principle of adaptive translation error in the in-situ training method for an error-adaptive optics diffraction neural network provided in an embodiment of the present invention. Figure 4As can be seen, directly reversing the optical path to make reverse propagation into forward propagation results in a "forward propagation" optical path that is not identical to the original optical path (the actual forward propagation optical path) if the center of the image received by the CCD camera does not coincide with the optical axis. This means there is a translation error (misalignment). To address this issue, this embodiment directly performs three-dimensional central symmetry on the reverse propagation optical path. Besides replacing reverse propagation with forward propagation, it also adjusts the input optical field (error optical field) of the reverse propagation. P The optical path is centrally symmetric. Therefore, even with translation errors, the optical path used to simulate "forward propagation" in this embodiment remains identical to the original optical path, enabling the network to adapt to translation errors.

[0093] Preferably, in this embodiment of the invention, a four-step phase-shifting method is used to measure the output light intensity at the imaging surface, and then the corresponding complex beam amplitude is calculated. Before network training begins, the phase of the incident light at the beam splitter is measured. During each phase measurement during training, the phase of the incident light at the beam splitter is subtracted, i.e. U 1 、U 2 、 … 、U n 、U o 、U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The phase of the incident light at the beam splitter is subtracted from the phase of the incident light in the equation, and the phase after the subtraction is taken as the phase of the incident light. U 1 、U 2 、 … 、U n 、U o 、U 1 ’⊙ , U 2 ’⊙ , …、 U n ’⊙ The corresponding phase is used to ensure that the phase of each measurement is not affected by the initial phase of the incident light. In this way, the inherent phase error in the optical path can be eliminated, such as the phase error of the laser itself, the phase added by the components in the optical path, etc.

[0094] like Figure 5As shown, when a tilt error occurs in the optical path, the phase of the incident light on the receiving surface of the CCD camera will change, and the magnitude of the change is related to the angle of the CCD camera offset. However, in this embodiment of the invention, when using the four-step phase-shift method to measure the intensity at the CCD camera, the actual measured phase is the phase difference between the optical field after the incident light passes through the system and the reference light. Note that the phase change introduced by the tilt error occurs simultaneously in both the incident light and the reference light; therefore, the phase difference will not change, meaning the measurement result will not be affected by the tilt error, thus enabling the training process to adapt to tilt errors.

[0095] Based on the above analysis, the method in the embodiments of the present invention can adapt to the errors existing in the in-situ training of optical diffraction neural networks, including inherent phase error, tilt error and translation error.

[0096] In this embodiment of the invention, the input sample is an image, and the ground truth label used during loss training is the target interference light intensity corresponding to the image. T .

[0097] As a preferred embodiment of the invention, the complex amplitude of the light field at each CCD camera imaging surface is calculated using a time-division generation method.

[0098] In S1, the complex amplitude of the output light field at the output plane is calculated. U o ,include:

[0099] The time-division generation method is used to generate the complex amplitude of the input sample. U i The complex amplitude of the optical field after the forward propagation of the optical diffraction neural network on the back surface of the first phase modulation layer U 1. Decomposed into two phase diagrams; wherein the amplitudes of the two phase diagrams are equal and both are constants;

[0100] The two decomposed phase maps are applied twice to the first-layer spatial light modulator. The intensity distribution of the output interference light is measured at the output plane twice to obtain the corresponding complex amplitude of the optical field. The two complex amplitudes are then added together to obtain the complex amplitude of the output optical field at the output plane. U o The complex amplitude includes both amplitude and phase distribution. Specifically, the output light intensity at the CCD camera... O k (Intensity distribution of interference light) and complex amplitude of the light field U k The relationship between them is: ,in, conj () indicates the conjugate operation.

[0101] Similarly, in S3, the light field P⊙ Loaded in n On the phase modulation layer, the complex amplitude of the optical field is obtained at the output plane. U 1 ’⊙ ,include:

[0102] Using a time-division generation method, the light field P ⊙ It is decomposed into two phase diagrams; the amplitudes of the two phase diagrams are equal and both are constants;

[0103] The two phase diagrams after decomposition are in the... n The spatial light modulator is loaded twice, and the intensity distribution of the interference light output is measured at the output plane twice to obtain the corresponding complex amplitude of the optical field. The two complex amplitudes are then added together to obtain the complex amplitude of the output optical field at the output plane. U 1 ’⊙ ;

[0104] Will U n-k ’⊙ Loaded in k On the phase modulation layer, at the first k The complex amplitude of the optical field is obtained at the imaging surface of the +1 phase modulation layer. U n+1-k ’⊙ With the k The product of the phases of the +1 phase modulation layer, will be the product of the phases of the +1 phase modulation layer. k The complex amplitude of the optical field obtained at the imaging surface of the +1 phase modulation layer is divided by the first... k The phase of the +1 phase modulation layer is obtained U n+1-k ’⊙ ,1≤ k < n ,include:

[0105] Using a time-sharing generation method, U n-k ’⊙ It is decomposed into two phase diagrams; the amplitudes of the two phase diagrams are equal and both are constants;

[0106] The two phase diagrams after decomposition are in the... k The spatial light modulator is loaded twice, in the first... k The intensity distribution of the output interference light was measured twice at the imaging surface of the +1 phase modulation layer to obtain the corresponding complex amplitude of the optical field. The two complex amplitudes were then added together and divided by the first phase modulation layer. k The phase of the +1 phase modulation layer is obtained to get the first... k Complex amplitude of the light field at the imaging surface of the +1 phase modulation layer U n+1-k ’⊙.

[0107] Specifically, such as Figure 6 As shown, the time-division generation method is used to decompose the complex amplitude into two phase diagrams, as follows:

[0108]

[0109] in, That is, the complex amplitude that we want to generate. For amplitude, For phase, j The imaginary unit; in S1, U The complex amplitude of the input sample U i The complex amplitude after multiplying with the phase of the phase modulation layer, i.e. U 1. In S3, U Light field P ⊙ 、U 1 ’⊙ , U 2 ’⊙ , …、U n-1 ’⊙ ; and These are the two phase diagrams after decomposition; , is a constant. The outputs of these two phase diagrams in the linear system are measured sequentially and then added together, which gives the output of the complex amplitude in the linear system before decomposition.

[0110] Based on the above embodiments, in order to verify the practicality of the present invention, in another embodiment, such as... Figure 7 As shown, one application of the present invention is provided: face left and right turning recognition. For example... Figure 7 As shown in (a), when the face in the test image is facing to the right, the light intensity in the right frame of the corresponding output image is greater, and when the face in the test image is facing to the left, the light intensity in the left frame of the corresponding output image is greater. Figure 7 (b) in the diagram represents the phase distribution of the optical diffraction neural network. Figure 7 (c) in the figure is an experimental photo of real-time face turning recognition. In this invention, face data is collected in real time by a camera and uploaded to an optical diffraction neural network for recognition. It can be seen that when the test subject’s face is facing different directions, the area where the light intensity converges is also different, thus realizing real-time face turning recognition.

[0111] Based on the above embodiments, in order to verify the error adaptive capability of the present invention, in another embodiment, such as... Figure 8 As shown, the training results of the present invention are provided when tilt error and translation error coexist. Figure 8(c) illustrates the method for introducing errors in this embodiment: by rotating the spatial light modulator, the incident angle of the incident light on the spatial light modulator is changed, thereby simultaneously introducing translation error and tilt error. Figure 8 Figures (a) and (b) show the recognition accuracy after network training and the loss function decrease curve during training, respectively, when the spatial light modulator is rotated ±15°. It can be seen that the method of this embodiment can adapt well to various errors existing in the optical path.

[0112] Based on the above embodiments, in order to verify the practicality of the present invention, in another embodiment, such as... Figure 9 As shown, a second application of the present invention is provided: unsupervised learning. In this embodiment, unsupervised learning uses 0 and 7 randomly selected from MNIST handwritten digits after removing their labels. It can be seen that in this embodiment, the network distinguishes between 0 and 7 during the unsupervised learning process, with 0 clustering to the left detector and 7 clustering to the right detector. As more and more images are learned, the clustering accuracy increases.

[0113] Example 2

[0114] This invention provides an in-situ training system for an error adaptive optical diffraction neural network, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the in-situ training method for the error adaptive optical diffraction neural network in Embodiment 1 above.

[0115] The relevant technical solutions are the same as above, and will not be repeated here.

[0116] Example 3

[0117] This invention provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the in-situ training method for the error adaptive optical diffraction neural network in Embodiment 1 above.

[0118] Specifically, the memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0119] The relevant technical solutions are the same as above, and will not be repeated here.

[0120] Example 4

[0121] This application provides a computer program product, including a computer program that, when run on a computer, causes the computer to perform the steps of the error adaptive optics diffraction neural network in-situ training method in Embodiment 1 above.

[0122] The relevant technical solutions are the same as above, and will not be repeated here.

[0123] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An in-situ training method for an error adaptive optical diffraction neural network, characterized in that, include: S1, Phase based on each phase modulation layer Calculate the complex amplitude of the input sample. U i The complex amplitude of the optical field on the back surface of each phase modulation layer after the forward propagation of the optical diffraction neural network. U 1 、U 2 、 … 、U n and the complex amplitude of the output optical field at the output plane U o , n This represents the total number of phase modulation layers. S2, Calculation U o Corresponding light intensity O With target light intensity T loss function between L about U o The gradient is used as the error optical field. P The error optical field P The reverse propagation optical path is made three-dimensionally centrosymmetric without changing the phase distribution on each phase modulation layer, so as to transform the reverse propagation optical path into a forward propagation optical path and obtain the error optical field. P Light field after two-dimensional centrosymmetry P ⊙ ; S3. Calculate the light field. P ⊙ The complex amplitude of the optical field propagating forward to the front surface of each phase modulation layer U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ and will U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. P The complex amplitude of the optical field propagating backward to the back surface of each phase modulation layer U 1 ’ 、U 2 ’ 、…、U n ’ ; wherein, the error optical field P Backpropagation to the k The complex amplitude of the optical field on the back surface of the phase modulation layer is U n+1-k ’ ,1≤ k ≤ n ; S4, based on U k and U n+1-k ’ Calculate the loss function L Regarding the aforementioned number k Layer phase modulation layer phase The gradient is calculated, and the update is performed based on that gradient. ; S5, switch to S1 for the next round of training, until the preset number of training rounds is reached or the loss converges.

2. The in-situ training method for error adaptive optics diffraction neural networks according to claim 1, characterized in that, In S3, the light field is calculated. P ⊙ The complex amplitude of the optical field propagating forward to the front surface of each phase modulation layer U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ and will U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. P The complex amplitude of the optical field propagating backward to the back surface of each phase modulation layer U 1 ’ 、U 2 ’ 、…、U n ’ ,include: The light field P ⊙ Loaded in n On the phase modulation layer, the complex amplitude of the optical field is obtained at the output plane. U 1 ’⊙ ,Will U 1 ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. P Backpropagation to the n Complex amplitude of optical field on the back surface of the phase modulation layer U 1 ’ ; Will U n-k ’⊙ Loaded in k On the phase modulation layer, at the first k The complex amplitude of the optical field is obtained at the front surface of the +1 phase modulation layer. U n+1-k ’⊙ Then in the first k The complex amplitude of the optical field obtained at the imaging surface of the +1 phase modulation layer is: U n+1-k ’⊙ With the k +1 layer phase modulation layer phase The product; will be in the first k The complex amplitude of the optical field obtained at the imaging surface of the +1 phase modulation layer is divided by ,get U n+1-k ’⊙ ,Will U n+1-k ’⊙ The error optical field is obtained by performing two-dimensional centrosymmetry. P Backpropagation to the k Complex amplitude of optical field on the back surface of the phase modulation layer U n+1-k ’ ;1≤ k < n .

3. The in-situ training method for error adaptive optics diffraction neural networks according to claim 1 or 2, characterized in that, In S1, the space where the input sample is located is on the same plane as the first phase modulation layer; wherein... U 1 equals U i Phase with the first phase modulation layer The product of U k equal U k-1 After the first k -1st floor to the k The complex amplitude of the optical field after free-space diffraction between the phase modulation layers and the first layer k Layer phase modulation layer phase The product of 2 ≤ k ≤ n .

4. The in-situ training method for error adaptive optics diffraction neural networks according to claim 3, characterized in that, In S2, the error optical field P for: Among them, the loss function ; In S4, the loss function L Regarding the aforementioned number k Layer phase modulation layer phase gradient for: ;in, real () represents the real part operation. j The imaginary unit, abs () performs the absolute value operation, and angle() performs the phase angle operation. U k The complex amplitude of the input sample U i Forward propagation to the 1st k Complex amplitude of the optical field on the back surface of the phase modulation layer U n+1-k ’ For error light field P Backpropagation to the k The complex amplitude of the optical field on the back surface of the phase modulation layer.

5. The in-situ training method for error adaptive optics diffraction neural networks according to claim 1 or 2, characterized in that, The optical diffraction neural network is the first k The forward propagation optical path of the layer includes the first layer set along the optical path. k Spatial light modulator of the first layer, k The 4f-system of the layer and the first k Layered CCD camera; The first k The spatial light modulator of the layer is the first k A phase modulation layer is used to modulate the phase of the incident light; the incident light is used to provide a light source for training an optical diffraction neural network. The first k The 4f-system of the layer is used to map the optical field of the interference light onto the first... k In front of the CDD camera of the layer, so that the optical diffraction neural network can generate a preset diffraction distance; wherein, the interference light is the light generated after the outgoing light after phase modulation by the spatial light modulator and the reference light generated by the incident light beam splitting are interfered; The first k The CCD camera of the first layer is the k The imaging surface of the phase modulation layer is used to collect the intensity distribution of the interference light in order to calculate the first... k The complex amplitude of the optical field on the back surface of the phase modulation layer.

6. The in-situ training method for error adaptive optics diffraction neural networks according to claim 5, characterized in that, Before the incident light is incident on the first layer of spatial light modulator, a beam preprocessing module is also included to preprocess the incident light; wherein, the beam preprocessing module includes, along the optical path direction: a laser, a collimating lens, an optical attenuator, a beam expander, a half-wave plate, a polarizer, and a beam splitter. The laser is used to generate the incident light; The collimating lens is used to collimate the incident light; The optical attenuator is used to reduce the intensity of the collimated incident light; The beam expander is used to increase the size of the incident light spot after passing through the optical attenuator; The half-wave plate and the polarizer are used together to adjust the polarization direction of the incident light to the working polarization direction of the spatial light modulator; The beam splitter is used to split the incident light after the polarization direction has been adjusted. One beam is used as the reference light, and the other beam is incident on the first layer spatial light modulator for phase modulation.

7. The in-situ training method for error adaptive optics diffraction neural networks according to claim 6, characterized in that, Using a time-sharing generation method, calculate U o , U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ ,include: The time-division generation method is used to generate the complex amplitude of the optical field. U It is decomposed into two phase diagrams; the amplitudes of the two phase diagrams are equal and both are constants; The two decomposed phase images are loaded onto the spatial light modulator twice, and the intensity distribution of the output interference light is measured at the CCD camera twice to obtain the corresponding complex amplitude of the optical field. The two complex amplitudes are then added together to obtain the complex amplitude of the optical field. Among them, the complex amplitude of the light field U For the complex amplitude of the light field U At time 1, the spatial light modulator is the first-layer spatial light modulator, the CCD camera is the output plane, and the complex amplitude of the light field... for U o ; complex amplitude of light field U The light field P ⊙ At that time, the spatial light modulator is the first n The spatial light modulator, wherein the CCD camera is the output plane, and the complex amplitude of the light field... for U 1 ’⊙ ; complex amplitude of light field U for U n-k ’⊙ At that time, the spatial light modulator is the first k The spatial light modulator, wherein the CCD camera is the first k The CCD camera with +1 layer, the complex amplitude of the light field for U n+1-k ’⊙ With the k The product of phases on the +1 layer spatial light modulator will Divided by the k The phase of the +1 layer spatial light modulator is obtained U n+1-k ’⊙ ,1≤ k < n .

8. The in-situ training method for error adaptive optics diffraction neural networks according to claim 7, characterized in that, At the CCD camera, the intensity distribution of the interference light is measured using a four-step phase-shifting method; Before each round of training begins, the phase of the incident light at the beam splitter is measured. Each round of training also includes... U 1 、U 2 、 … 、U n 、U o 、U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The phase of the incident light at the beam splitter is subtracted from the phase of the incident light in the phase of the incident light, and the phase after the subtraction is taken as the phase of the incident light. U 1 、U 2 、 … 、U n 、U o 、U 1 ’⊙ , U 2 ’⊙ , …、U n ’⊙ The corresponding phase.

9. An in-situ training system for an error adaptive optical diffraction neural network, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the in-situ training method for error adaptive optics diffraction neural networks according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the in-situ training method for error adaptive optical diffraction neural networks as described in any one of claims 1-8.

Images