Diffractive optical network system with parameterized noise exploration and training method thereof

Abstract

The invention discloses a diffractive optical network system with parameterized noise exploration and a training method thereof. The system comprises an input light field unit, a light splitting unit, a first modulation unit of a mean value light path, a second modulation unit of a variance light path, a dynamic scattering medium of the variance light path, a light field superposition unit, a photoelectric detection unit and a control unit. The dynamic scattering medium generates disturbance epsilon in a variance light path; the first modulation unit and the second modulation unit respectively correspond to trainable parameters mu and sigma, so that the system equivalently transmits a parameterized noise structure of (mu + sigma epsilon); and through interference superposition and photoelectric detection, output is obtained. According to the training method, the mu, the sigma and the sampling rhythm are jointly optimized by reinforcing a learning target; a variance light path can be closed after training is completed, and reasoning is performed only through a mean value light path. According to the method, a trainable parameterized noise network is constructed by adopting mean / variance double paths and dynamic scattering sampling, so that the diffractive optical network in the training process has a state-related exploration capability, and meanwhile, the trained diffractive optical network has higher robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical computing and machine learning, in particular to a diffractive optical neural network system with parameterized noise exploration and a training method thereof. BACKGROUND

[0002] Diffractive optical network (DON) modulates the incident light field carrying information based on the principle of light diffraction, thereby realizing inference with extremely low latency and high parallelism. DON has shown application potential in image generation, light field transmission and data encryption. Facing scenarios such as autonomous driving and robot control, DON is expected to compress and map high-dimensional optical information to low-dimensional control variables, realizing end-to-end all-optical inference of “perception—decision—control” integration.

[0003] When the task involves decision-making, reinforcement learning (RL) aims at long-term return and considers the balance between exploration and utilization, which is more helpful to alleviate the performance degradation caused by the difference in data distribution than imitation learning. Therefore, it is reasonable to place the training of DON in the RL framework in the above target scenarios. Further, the exploration strategy directly affects the sample efficiency and stability of training: common methods such as ε-greedy, entropy regularization or adding Gaussian noise to activations / weights, Dropout, etc. are mostly weakly related random perturbations to the state, which can easily lead to insufficient exploration efficiency or training shock.

[0004] The comparative document “Noisy Networks For Exploration” (M. Fortunato et al. presented at International Conference on Learning Representations, 15 February 2018.) proposes to directly introduce parameterized noise into network weights, representing the weights as w = μ + σ ⊙ ε (μ, σ are trainable parameters, and ε is a random variable), and the randomness is affected by the network and changes adaptively with the input state, forming a more effective state-dependent exploration. The published results show that replacing the exploration method in the reinforcement learning algorithm with NoisyNet usually achieves more stable and better performance. At present, there is still a lack of systematic solutions to effectively couple NoisyNet to DON to improve the system stability, exploration ability and convergence quality.

[0005] For optical systems, scattering media can be viewed as naturally parallelizable and physically implementable "sampling / noise" channels, providing the conditions for mapping "learnable parameterized noise" mechanisms to the physical training process of DONs. Therefore, how to effectively utilize scattering media to construct a noise-based diffraction optical network training method remains to be explored. Summary of the Invention

[0006] To address the challenges faced by diffractive optical networks in tasks involving decision-making or closed-loop evaluation, such as the lack of state-dependent exploration mechanisms during the training phase, the difficulty in controllably introducing and learning noise structures within the physical system, and the challenge in unifying training and deployment paths, this paper proposes a system and method for implementing trainable parameterized noise in the optical domain. This enables the network to possess state-dependent exploration capabilities during training and provides greater robustness and a convenient deployment path after training is completed.

[0007] To achieve the above objectives, this invention provides a diffraction optical neural network system with parameterized noise detection. The system includes: an input light field unit, a beam splitting unit, a first modulation unit, a second modulation unit, a physical sampling unit, a light field superposition unit, a photoelectric detection unit, and a control unit. Wherein:

[0008] a) The input light field unit is used to generate and inject the input light field under coherent illumination conditions;

[0009] b) The beam splitting unit is used to divide the input optical field into two independent mean optical paths and variance optical paths;

[0010] c) The first modulation unit is disposed in the mean optical path and performs programmable phase and / or amplitude modulation on the passing optical field according to the first parameter set μ;

[0011] d) The second modulation unit is located in the variance optical path and performs programmable phase and / or amplitude modulation on the passing optical field according to the second parameter set σ;

[0012] e) The physical sampling unit is set in the variance optical path and contains a dynamic scattering medium, which is used to randomly scatter the light field modulated by the second modulation unit to generate a sampling perturbation ε that satisfies the preset statistical characteristics.

[0013] f) The optical field superposition unit is used to interferometrically superimpose two optical fields to obtain a superimposed optical field;

[0014] g) The photoelectric detection unit is used to read out the square-law intensity of the superimposed light field and output an electrical signal;

[0015] h) the control unit is electrically connected with the first and second modulation units, and is configured to provide a control signal corresponding to the parameter set μ, σ, and / or set a sampling rhythm of the physical sampling unit and a statistical target; when the dynamic scattering medium operates in a spontaneous random manner, the control unit can not impose control on it.

[0016] In an optional embodiment, the light splitting unit and / or the light field superposition unit employs a light splitting prism and / or a beam splitting device, and together with the two light paths forms a Mach-Zehnder interference structure; the light splitting / superposition can also employ one or more of polarization multiplexing, time division multiplexing, frequency division multiplexing or spatial multiplexing.

[0017] In an optional embodiment, the first and / or second modulation unit is one or more of a liquid crystal spatial light modulator (SLM), a digital micromirror device (DMD), an electro-optic modulator (EOM), and the modulation mode is phase modulation, amplitude modulation or phase-amplitude joint modulation; the dynamic scattering medium can be a programmable scatterer, a rotating / vibrating ground glass, a micro-electro-mechanical mirror array, a multi-mode optical fiber, a liquid crystal scatterer or a combination thereof.

[0018] The application also provides a training method applied to the above system, comprising:

[0019] S0, initialization: set the parameter set μ, σ to be optimized and map them to the first modulation unit and the second modulation unit by the control unit, and set the sampling rhythm and / or the sampling distribution parameters;

[0020] S1, spontaneous update of sampling: before loading the input, spontaneously and / or according to the sampling rhythm, update the sampling state of the dynamic scattering medium of the variance light path to generate a new sampling disturbance ε;

[0021] S2, load input: provide the input light field unit with an input state related to a task and inject the system;

[0022] S3, optical path execution: automatically complete the light splitting of the input light field, the modulation of the mean value light path and the variance light path, the physical sampling of the variance light path and the interference superposition through the optical path of the system;

[0023] S4, readout: perform square-law intensity readout by the photoelectric detection unit to obtain an observation signal;

[0024] S5, optimization: based on the observation signal and the task target, construct an optimization criterion, update the parameter set μ, σ, and write the updated μ, σ into the first modulation unit and the second modulation unit, respectively;

[0025] S6, determination: determine according to a preset training completion condition; if the condition is not met, return to execute S1.

[0026] In one optional implementation, the optimization criterion is a reinforcement learning objective aimed at long-term reward, employing a policy gradient, value function, or a combination of both in an Actor-Critic framework; the training completion condition includes at least one of the following: the validation set performance reaches a preset threshold, the objective function improvement is lower than a preset threshold for several consecutive rounds, or the policy stability index reaches a preset threshold; after training is completed, a deployment switch is performed: the variance optical path is closed and / or the second modulation unit parameter σ is frozen or cleared, retaining only the mean optical path for inference output.

[0027] Compared with the prior art, the present invention has the following effects and features:

[0028] a) A trainable parameterized noise structure (equivalent to μ+σ⊙ε) is implemented in the optical domain through mean / variance dual optical paths and dynamic scattering sampling, enabling the network to have state-dependent exploration capabilities during training.

[0029] b) After training is complete, the variance optical path can be turned off and only the mean optical path can be retained for inference, making the deployment path simple in structure and easy to implement;

[0030] c) The system and method are more robust under physical fluctuations introduced by scattering sampling, making them suitable for application scenarios involving environmental disturbances.

[0031] Other features and details of the present invention can be found in the accompanying drawings and detailed description. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0033] Figure 1 A schematic diagram of a diffractive optical network system with parameterized noise detection according to an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the optical structure of a diffractive optical network system with parameterized noise detection according to an embodiment of the present invention;

[0035] Figure 3 This is a flowchart illustrating a training method for a diffractive optical network with parameterized noise exploration according to an embodiment of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] like Figure 1 As shown, the system 100 of this embodiment includes an input light field unit 110, a beam splitting unit 120, a first modulation unit 130 disposed in the mean light path, a second modulation unit 140 disposed in the variance light path, a physical sampling unit (dynamic scattering medium) 150 located in the variance light path, a light field superposition unit 160, a photoelectric detection unit 170, and a control unit 180. Under coherent illumination conditions, the input light field is split to form two light paths; the two light paths are respectively modulated by the first modulation unit (corresponding to parameter μ) and the second modulation unit (corresponding to parameter σ); the variance light path is further subjected to a sampling perturbation ε by the dynamic scattering medium; the two light fields interfere and superimpose at the light field superposition unit, and the output signal is obtained by square-law intensity readout by the photoelectric detection unit.

[0038] In some embodiments, beam splitting and superposition can be achieved by forming an interference structure using beam-splitting prisms or beam-splitting devices; alternatively, polarization multiplexing, time-division multiplexing, frequency-division multiplexing, or spatial multiplexing can be used to achieve separation and interference of the two paths. The first / second modulation unit can be one or more of a liquid crystal spatial light modulator, a digital micromirror device, or an electro-optic modulator; the dynamic scattering medium can be a programmable scatterer, a rotating / vibrating frosted glass, a microelectromechanical microscope array, a multimode fiber end face, or a liquid crystal scatterer, etc.

[0039] In some embodiments, the control unit is configured to: write the parameters μ and σ of the current iteration into the first modulation unit and the second modulation unit, respectively, during the training phase; optionally set the sampling rhythm and statistical target of the physical sampling unit; and when the dynamic scattering medium operates in a spontaneous random manner, the control unit does not exert control over it. During the deployment phase, the variance optical path may be shut down and / or σ may be frozen or cleared, and inference may be performed only through the mean optical path.

[0040] like Figure 2The schematic diagram of the optical structure shown is an optional implementation: the laser output from the laser is collimated and expanded by lens 1 and beam expander, and then incident at a specific angle onto the digital micromirror device under the action of mirror 1. The input data of the diffraction optical network is loaded onto the digital micromirror device to generate an input light field; the input light field is split into a mean light path and a variance light path by beam splitter 1; spatial light modulator 1 and spatial light modulator 2 apply phase modulation determined by μ and σ to the two light fields respectively; polarizers 1, 2, and 3 are placed before and after the spatial light modulators to ensure accurate modulation; at the end of the variance light path, the light field is further randomly scattered by rotating frosted glass to generate ε; beam splitter 2, beam splitter 3, and mirror 2 form a light field superposition unit, and the two light fields are superimposed by interference at beam splitter 3, and the output signal is obtained by square-law intensity readout by charge-coupled device. It should be noted that... Figure 2 The specific devices shown (beam splitters, polarizers, SLM / DMD, lenses, mirrors, etc.) are for illustrative purposes only and do not constitute a limitation on the selection and layout of system devices.

[0041] like Figure 3 As shown, in some embodiments, the training method of the present invention includes the following steps: S0: Initialization, setting initial parameters μ, σ, and sampling rhythm and / or sampling distribution parameters; S1: Before loading input, the dynamic scattering medium spontaneously and / or updates the sampling state according to the sampling rhythm to generate new sampling perturbation ε; S2: Providing task-related input states to the input optical field unit and injecting them into the system; S3: The system automatically completes beam splitting, two-path modulation (μ / σ), physical sampling (ε) of the variance optical path, and interference superposition; S4: The photoelectric detection unit performs square-law intensity readout to obtain the observation signal; S5: Based on the observation signal and the task objective, construct optimization criteria, update parameters μ and σ (optionally update the sampling rhythm and / or sampling distribution parameters simultaneously), and write the updated parameters into the first / second modulation unit respectively; S6: Determine according to the preset training completion conditions; if not met, return to execute S1; if met, end the training and enter the deployment stage.

[0042] In some embodiments, the optimization criterion can be a reinforcement learning objective aimed at long-term returns, employing a policy gradient, a value function, or a combination of both in an Actor-Critic paradigm; training completion conditions can include at least one of the following: validation set performance reaches a threshold, the objective function improvement is below a threshold for several consecutive rounds, or the policy stability index reaches a threshold. The above choices do not constitute a limitation on the algorithm.

[0043] For ease of understanding, another embodiment provides a specific application scenario. Taking an inverted pendulum control task as an example (continuous action space), the system takes the environmental state as input, generates a control quantity through the optical network of this invention, and applies it to the controlled object in a closed loop. This embodiment is only used to illustrate the application of this invention in sequential decision-making problems and does not constitute a limitation on the task type.

[0044] In this embodiment, the environmental state s can be adopted Formal representation, in which For swing angle, The angular velocity is used. The control unit can normalize the state and encode it into an input light field: for example, by setting multiple regions on a spatial light modulator in a spatial multiplexing manner, corresponding to the amplitude or phase of different state components; or by loading multiple frames sequentially within an iteration cycle in a time multiplexing manner and integrating them at the detector end. The electrical signal read from the output end is linearly scaled to obtain the continuous torque acting on the pendulum shaft, and the limiting range can be set according to the capability of the actuator.

[0045] In this embodiment, the reinforcement learning paradigm can employ a deterministic strategy Actor-Critic algorithm such as TD3: the Actor is provided by the optical network of this invention, which maps the state-encoded input light field to action 'a', and its trainable parameters correspond to the mean optical path parameter μ and the variance optical path parameter σ (σ participates in "parameterized noise exploration" during the training phase); the Critic is implemented electronically in the control unit or external processor, receiving (s,a) and reward r and outputting a Q estimate, which is used to construct optimization criteria (e.g., using a target network and experience replay, the specific implementation is not limited), wherein the reward definition can be selected according to task requirements, for example... in These are positive weighting coefficients. During each iteration, the variance optical path generates a new sampling perturbation ε before loading the input. The action of the optical network output is reported by the environment, and the control unit updates μ and σ accordingly and writes them back to the corresponding modulation unit.

[0046] In this embodiment, typical hyperparameter settings can be provided as examples as needed, such as the number of training epochs, learning rate, and reward weights, which can be selected within commonly used ranges. This invention is not limited to specific numerical values.

[0047] In some embodiments, the variance optical path can be physically shut down or σ can be set to zero during the deployment phase, leaving only the mean optical path for inference output; input encoding and output scaling can be kept consistent with the training phase to enable online inference for real-time control or other tasks.

[0048] Without departing from the inventive concept, the following modifications can be made: the algorithm genre can be replaced with PPO, SAC, A2C, DDPG or their variants; the input encoding method can be phase encoding, amplitude encoding or phase-amplitude joint encoding, or polarization / frequency division / spatial multiplexing can be used; the output can be expanded to multi-channel readout to adapt to multi-dimensional control quantities; the sampling strategy can change its statistical properties by adjusting the scatterer rotation speed, amplitude or programmable scattering mode on the basis of spontaneous randomness, or symmetric sampling or multi-sample sampling can be used to reduce gradient estimation variance; a calibration step can be added at the beginning of training or during the intermittent stage to compensate for slow drift, and this step does not change the core structure defined in the claims.

[0049] It should be noted that the above descriptions of structure, process and application can be used independently or in any combination; the terms "may / optional / for example" appearing in the text are intended to describe feasible methods and do not constitute a limitation on the claims.

[0050] The foregoing has provided a detailed description of a diffractive optical network system with parameterized noise exploration and its training method provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A diffractive optical neural network system with parameterized noise detection, characterized in that, The system includes: an input light field unit, a beam splitting unit, a first modulation unit, a second modulation unit, a physical sampling unit, a light field superposition unit, a photoelectric detection unit, and a control unit; wherein, a) The input light field unit is used to generate and inject an input light field under coherent illumination conditions; b) The beam splitting unit is used to divide the input light field into an independent mean light path and a variance light path; c) The first modulation unit is disposed in the mean optical path and is used to perform programmable phase and / or amplitude modulation on the passing optical field according to the first parameter set μ; d) The second modulation unit is disposed in the variance optical path and is used to perform programmable phase and / or amplitude modulation on the passing optical field according to the second parameter set σ; e) The physical sampling unit, disposed in the variance optical path, includes a dynamic scattering medium, used to randomly scatter the light field modulated by the second modulation unit to generate a sampling perturbation ε that satisfies preset statistical characteristics; f) The light field superposition unit is used to perform interference superposition of the light fields of the mean light path and the variance light path to obtain a superimposed light field; g) The photoelectric detection unit is used to read out the intensity of the superimposed light field and output an electrical signal; h) The control unit, electrically connected to the first modulation unit and the second modulation unit, is used to provide control signals corresponding to the parameter sets μ and σ, and / or set the sampling rhythm and statistical target of the physical sampling unit.

2. The system according to claim 1, characterized in that, The beam splitting unit and / or the optical field superposition unit are beam splitting prisms and / or beam splitting devices, which together with the two optical paths form a Mach-Zehnder interference structure, and / or the beam splitting and superposition adopt one or more of polarization multiplexing, time-division multiplexing, frequency-division multiplexing or spatial multiplexing.

3. The system according to claim 1 or 2, characterized in that, The first modulation unit and / or the second modulation unit are one or more of a liquid crystal spatial light modulator (SLM), a digital micromirror device (DMD), and an electro-optic modulator (EOM), and the modulation method is phase modulation, amplitude modulation, or phase-amplitude combined modulation.

4. The system according to any of the preceding claims, characterized in that, The dynamic scattering medium is a programmable scatterer, a rotating or vibrating frosted glass, a microelectromechanical microscope array, a multimode fiber, a liquid crystal scatterer, or a combination thereof. The control unit is optionally electrically connected to the dynamic scattering medium to adjust the scattering rate, correlation, or spectral density. When the dynamic scattering medium operates in a spontaneous random mode, the control unit does not apply control to it.

5. The system according to any of the preceding claims, characterized in that, The photoelectric detection unit includes a linear array or area array photoelectric detector, and the square law intensity readout is nonlinearly mapped by a readout circuit or processor to correspond to the nonlinear layer in the neural network.

6. The system according to any of the preceding claims, characterized in that, The parameter set μ corresponds to the equivalent transmission parameters of the mean optical path, the parameter set σ corresponds to the equivalent transmission parameters of the variance optical path, and the sampling perturbation ε is generated by the dynamic scattering medium in the physical domain so that the system as a whole forms a parameterized noise structure of μ + σ⊙ε.

7. The system according to any of the preceding claims, characterized in that, The control unit is configured to enable the variance optical path during the training phase, disable the variance optical path and / or set σ to zero during the inference phase, thereby completing low-latency inference only through the mean optical path.

8. A training method for a diffractive optical neural network with parameterized noise exploration, applied to the system described in any one of claims 1 to 7, characterized in that, Includes the following steps: S0. Initialization: Set the set of parameters to be optimized, μ and σ, and map them to the first modulation unit and the second modulation unit respectively by the control unit; set the sampling rhythm and / or sampling distribution parameters. S1. Spontaneous sampling update: Before loading the input, the dynamic scattering medium of the variance optical path is made to spontaneously and / or update the sampling state according to the sampling rhythm to generate a new sampling perturbation ε. S2, Input Loading: Provide the input light field unit with task-related input states and inject them into the system; S3, Optical path execution: The optical path of the system automatically completes the beam splitting of the input light field, the modulation of the mean optical path and the variance optical path, the physical sampling of the variance optical path, and the interference superposition. S4. Readout: The square-law intensity is read out by the photoelectric detection unit to obtain the observation signal; S5. Optimization: Based on the observed signal and the mission objective, construct optimization criteria, update the parameter sets μ and σ, and write the updated μ and σ into the first modulation unit and the second modulation unit, respectively; S6. Judgment: Make a judgment based on the preset training completion conditions; if not met, return to execute S1.

9. The method according to claim 8, characterized in that, The optimization criterion is a reinforcement learning objective aimed at long-term reward, employing one of the following: policy gradient, value function, or a combination of both in the Actor-Critic genre, and using the parameterized noise structure to achieve state-dependent exploration.

10. The method according to claim 8 or 9, characterized in that, The training completion conditions include at least one of the following: the validation set performance reaches a preset threshold, the objective function improvement is lower than a preset threshold for several consecutive rounds, or the policy stability index reaches a preset threshold; and after training is completed, a deployment switch is performed: the variance optical path is closed and / or the second modulation unit parameter σ is frozen or cleared, and only the mean optical path is retained for inference output.

Images