Spatial diffraction deep neural network asymmetric multifunctional imaging method and system

Abstract

The invention discloses a spatial diffraction deep neural network asymmetric multifunctional imaging method and system, and is suitable for the field of optical signal processing. The system is composed of a signal input module, an all-optical diffraction neural network module and a signal acquisition module. The signal input module is responsible for providing a light source and a modulation element and is used for generating and modulating a light field signal. And the all-optical diffraction neural network module adopts a double-loss function optimization strategy to carry out asymmetric training. Specifically, different loss functions are introduced in different transmission directions to train different functions at the same time, so that diversified processing of signals is realized. And the signal acquisition module is responsible for acquiring and storing a final calculation result. Different imaging functions can be realized according to different signal input directions, so that diversified requirements of optical signal processing are met, the blank of related fields is filled, and a brand new thought is provided for further research and application of the optical imaging field.

Description

Technical Field

[0001] The present invention belongs to the field of optical signal processing, and more specifically, relates to a spatial diffraction deep neural network asymmetric multifunctional imaging method and system. Background Art

[0002] The spatial diffraction deep neural network is a new deep learning model that combines the principles of optical diffraction with neural network algorithms. Optical diffraction is the phenomenon in which light waves are deflected and overlapped when encountering obstacles or the edges of objects. This phenomenon is an indispensable foundation for optical imaging and information processing. By analyzing the diffraction pattern of light waves, information about the shape, size, and structure of an object can be obtained. This characteristic of diffraction makes it valuable for applications in image processing, optical imaging, and pattern recognition. Furthermore, a neural network is a mathematical model that mimics the structure and function of neurons in the human brain. By adjusting the connections and weights between neurons, a neural network can learn and extract features from input data, thereby achieving functions such as classification, recognition, and prediction. Deep learning is one of the fastest-growing methods in machine learning. It utilizes multi-layer artificial neural networks implemented in computers to digitally learn and abstract data, capable of performing advanced functions and, in some cases, even outperforming human experts. In recent years, deep learning has made significant progress in medical image analysis, speech recognition, language translation, image classification, and other fields. Combining optical diffraction and deep learning creates a spatial diffraction deep neural network. This combination can fully utilize the information processing capabilities of optical diffraction and the learning capabilities of neural networks to achieve faster processing and analysis of optical signals and images. However, spatial diffraction deep neural networks have high environmental requirements and are easily affected by factors such as light, temperature, and humidity, which limits their practical applications. Therefore, exploring the application capabilities of spatial neural networks and further expanding their application scenarios is a current research focus. Summary of the Invention

[0003] This invention provides an asymmetric multifunctional imaging method and system using spatial diffraction deep neural networks. Its unique feature is that it can flexibly perform different functions depending on the signal input direction, thereby adapting to the diverse needs of optical signal processing. This design enables the imaging method to effectively respond to different functional requirements and has broad application prospects. This innovation fills a research gap in the field of spatial diffraction deep neural networks and brings new development opportunities to the fields of optical imaging and information processing.

[0004] To achieve the above objectives, the present invention provides a spatial diffraction deep neural network asymmetric multifunctional imaging method, which constructs a spatial diffraction neural network. The spatial diffraction neural network is used as a carrier for optical imaging, so that the input light beam is imaged after being transmitted through the spatial diffraction neural network; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2...θ n ), where θ i is the phase of the i-th phase layer, 1≤i≤n;

[0005] Using a first image output after the original input light beam is forward-input into the spatial diffraction neural network and a second image output after the original input light beam is backward-input into the spatial diffraction neural network as training data, using machine learning to train the spatial diffraction neural network, determine a phase combination, and obtain a trained spatial diffraction neural network;

[0006] The input light beam to be imaged is input forward and backward in sequence to the trained spatial diffraction neural network to obtain the first imaging and the second imaging.

[0007] Preferably, the use of machine learning to train the spatial diffraction neural network includes: using an asymmetric double-loss optimization strategy, wherein the double-loss optimization function of the asymmetric double-loss optimization strategy is:

[0008] Loss=αL1+βL2

[0009] Among them, L1 is the loss function of forward propagation, L2 is the loss function of back propagation, and α and β are weighting coefficients.

[0010] The present invention also provides a spatial diffraction deep neural network asymmetric multifunctional imaging system, comprising a signal input module, a spatial diffraction deep neural network, and a signal output module;

[0011] The signal input module is used to provide an input light beam carrying image information;

[0012] The spatial diffraction deep neural network is used as a carrier for optical imaging, so that the input light beam is sequentially transmitted forward and backward through the spatial diffraction neural network to form an image, thereby obtaining a first image and a second image; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2...θ n ), where θ i is the phase of the i-th phase layer, 1≤i≤n;

[0013] The signal output module is used to display the imaging of the spatial diffraction deep neural network.

[0014] Preferably, the phase layers are arranged according to a preset layer spacing, and the phase combinations (θ1, θ2...θ n ) is obtained by training using a machine learning method. The layer spacing is not optimized during the training process. Instead, the best spacing value is selected as a fixed parameter by comparing the experimental results under different spacing conditions;

[0015] Preferably, each phase layer includes m×m pixels, and each pixel is used to achieve phase modulation in the range from 0 to 2π.

[0016] Preferably, the processing material system of each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, and photosensitive polymer.

[0017] Preferably, the medium between each phase layer is vacuum, gas, liquid or solid.

[0018] Preferably, the input light field enters the spatial diffraction neural network after passing through a propagation medium, and the propagation medium includes air or a complex medium, and the complex medium includes but is not limited to a scattering medium, an absorbing medium or a nonlinear medium.

[0019] Preferably, the spatial diffraction deep neural network adapts to light beams of different wavelengths and supports asymmetric imaging operations of single-wavelength and multi-wavelength input light beams.

[0020] Preferably, the three-dimensional integration process of the spatial diffraction deep neural network can be realized in a variety of ways, including but not limited to electron beam lithography and maskless lithography.

[0021] Through the above technical solutions conceived by the present invention, compared with the existing technology, the present invention adopts a spatial diffraction neural network structure, multi-layer phase modulation, and an asymmetric loss optimization strategy, which successfully realizes the coordinated optimization of two imaging functions, and can simultaneously meet the imaging needs of different directions in one device, providing an efficient, accurate and multifunctional optical imaging solution. On the basis of building a spatial diffraction deep neural network, a dual loss function optimization strategy is adopted for asymmetric training, which expands the function of the model and can flexibly perform different functions according to the different signal input directions. This is essentially different from the previous invention. On the other hand, the spatial diffraction deep neural network is widely used in the fields of optical imaging, optical communication, information security, pattern recognition, optical sensing and detection. Therefore, the asymmetric multifunctional imaging method of a spatial diffraction deep neural network provided by the present invention will play a very important application value in many fields in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of a spatial diffraction deep neural network asymmetric multifunctional imaging method provided by the present invention.

[0023] Figure 2 This is a diagram of the physical experimental setup of a spatial diffraction deep neural network asymmetric multifunctional imaging system provided by the present invention.

[0024] Figure 3 This is a simulation flow chart of the all-optical diffraction deep neural network module in the asymmetric multifunctional imaging method of the spatial diffraction deep neural network provided by an embodiment of the present invention.

[0025] Figure 4(a) is a schematic diagram of an embodiment of a spatial diffraction deep neural network asymmetric multifunctional imaging method provided by the present invention.

[0026] Figure 4(b) is a schematic diagram of an embodiment of another spatial diffraction deep neural network asymmetric multifunctional imaging method provided by the present invention.

[0027] FIG4( c ) is a schematic diagram of the imaging effect of the embodiment FIG4( a ).

[0028] FIG4( d ) is a schematic diagram of the imaging effect of the embodiment FIG4( b ). DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0030] To achieve the above objectives, the present invention provides a spatial diffraction deep neural network asymmetric multifunctional imaging method, comprising the following steps:

[0031] A spatial diffraction neural network is constructed, which is used as a carrier for optical imaging, so that the input light beam is imaged after being transmitted through the spatial diffraction neural network; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2...θ n ), where θ i is the phase of the i-th phase layer, 1≤i≤n;

[0032] Using a first image output after the original input light beam is forward-input into the spatial diffraction neural network and a second image output after the original input light beam is backward-input into the spatial diffraction neural network as training data, using machine learning to train the spatial diffraction neural network, determine a phase combination, and obtain a trained spatial diffraction neural network;

[0033] The input light beam to be imaged is input forward and backward in sequence to the trained spatial diffraction neural network to obtain the first imaging and the second imaging.

[0034] Specifically, the spatial diffraction neural network is trained using machine learning, including: adopting an asymmetric double-loss optimization strategy, wherein the double-loss optimization function of the asymmetric double-loss optimization strategy is:

[0035] Loss=αL1+βL2

[0036] Among them, L1 is the loss function of forward propagation, L2 is the loss function of back propagation, and α and β are weighting coefficients.

[0037] The present invention also provides a spatial diffraction deep neural network asymmetric multifunctional imaging system, comprising a signal input module, a spatial diffraction deep neural network, and a signal output module;

[0038] The signal input module is used to provide an input light beam carrying image information;

[0039] The spatial diffraction deep neural network is used as a carrier for optical imaging, so that the input light beam is sequentially transmitted forward and backward through the spatial diffraction neural network to form an image, thereby obtaining a first image and a second image; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2...θ n ), where θ i is the phase of the i-th phase layer, 1≤i≤n;

[0040] The signal output module is used to display the imaging of the spatial diffraction deep neural network.

[0041] Specifically, the phase layers are arranged according to a preset layer spacing, and the phase combinations (θ1, θ2...θ n ) is obtained by training using a machine learning method. The layer spacing is not optimized during the training process. Instead, the best spacing value is selected as a fixed parameter by comparing the experimental results under different spacing conditions;

[0042] Specifically, each phase layer includes m×m pixels, and each pixel is used to achieve phase modulation in the range from 0 to 2π.

[0043] Specifically, the processing material system of each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, and photosensitive polymer.

[0044] Specifically, the medium between each phase layer is vacuum, gas, liquid or solid.

[0045] Specifically, the input light field enters the spatial diffraction neural network after passing through a propagation medium, where the propagation medium includes air or a complex medium, and the complex medium includes but is not limited to a scattering medium, an absorbing medium or a nonlinear medium.

[0046] Specifically, the spatial diffraction deep neural network adapts to light beams of different wavelengths and supports asymmetric imaging operations for single-wavelength and multi-wavelength input light beams.

[0047] Specifically, the three-dimensional integration process of spatial diffraction deep neural network can be realized in a variety of ways, including but not limited to electron beam lithography and maskless lithography.

[0048] The following is a description with reference to specific embodiments and drawings.

[0049] like Figure 1 As shown, the present invention provides a schematic diagram of a spatial diffraction deep neural network asymmetric multifunctional imaging method, and the specific implementation method is as follows:

[0050] It includes a signal input module, an all-optical diffraction neural network module and an information acquisition module. The signal input module is responsible for generating the input light field and can be composed of a laser, a collimator and a spatial light field control device. The specific implementation method is as follows: the laser emits Gaussian light, the collimator converts the divergent light beam into a parallel light beam, and then the spatial light field control device performs signal modulation according to the preset information. The all-optical diffraction neural network module is used to extract and transform the signal features. It consists of multiple phase layers, with both sides of the network as input ends, and introduces a dual-loss function optimization strategy simulation model, so that the model can perform different functions according to the different transmission directions of the input signal, and synchronizes the trained and optimized model parameters to the optical phase modulation layer during the experimental phase. The information acquisition module is responsible for collecting the output results of the spatial deep diffraction neural network. This module can be composed of a CMOS camera and a computer. The specific implementation method is: after placing the CMOS camera on the all-optical diffraction neural network module, the collected light spot is uploaded to the computer for storage, and then the signal acquisition after the model output is completed.

[0051] The following describes a specific embodiment of the spatial diffraction deep neural network asymmetric multifunctional imaging system provided by the present invention to illustrate Figure 2 The structural diagram shown in the figure is as follows:

[0052] The system comprises: a laser 1, a collimator 2, a spatial light field control device (SLM3), a first lens 4, a first objective lens 5, a first optical phase modulation layer 6, a second optical phase modulation layer 7, a third optical phase modulation layer 8, a fourth optical phase modulation layer 9, a second objective lens 10, a second lens 11, a CMOS camera 12, and a computer 13. The Gaussian light output by the laser 1 passes through the collimator 2 to ensure its stable transmission and precise alignment, and is then transmitted to the SLM3. The SLM3 loads the set signal information to modulate the light field, and then passes through the 4f system composed of the first lens 4 and the first objective lens 5. The signal light carrying the pre-modulated information is reduced and imaged into the all-optical diffraction neural network composed of the first optical phase modulation layer 6, the second optical phase modulation layer 7, the third optical phase modulation layer 8, and the fourth optical phase modulation layer 9. The light is then expanded and collimated by the second objective lens 10 and the second lens 11 and transmitted to the CMOS camera 12 for acquisition. The acquisition results are uploaded to the computer for storage.

[0053] The following describes a specific embodiment of the spatial diffraction deep neural network asymmetric multifunctional imaging method provided by the present invention to illustrate the process of establishing a full-optical deep diffraction neural network module. The various parameters of the module in the embodiment of the present invention are obtained by establishing a simulation model and optimizing it using a deep learning method. Specifically, Figure 3 A simulation flow chart of an asymmetric imaging method according to an embodiment of the present invention is shown below:

[0054] A numerical simulation model of a multifunctional optical calculator using a spatial diffraction deep neural network was established. The angular spectrum method was used to simulate the propagation and diffraction of light between phase modulation layers. Based on the principle of fast Fourier transform (FFT), the diffraction of light waves during propagation and the resulting diffraction were simulated. A deep learning network was then established based on the simulation model. A dual-loss optimization strategy was used for asymmetric training, and the training parameters of the spatial diffraction deep neural network were optimized using gradient descent and backpropagation methods.

[0055] Specifically, assuming the input light field is U(x, y, z), where x and y are spatial coordinates on a plane, and z = 0 represents the input surface, the propagation of the light field U at the far field z is expressed as follows:

[0056] U N (x,y,0)=U N-1 (x,y,0)e i*2π*tanh(phi)

[0057]

[0058] U N (x,y,z)=F -1 {F{U N (x,y,0)}*H(fx ,f y ,z,λ)}

[0059]

[0060] Where F represents the Fourier transform operator, is the wave number, λ is the wavelength, H(f x ,f y ,z,λ) is the transfer function of free space, where f x and f y denote the spatial frequencies along the x and y directions, respectively.

[0061] In order to achieve asymmetric and multifunctional imaging, a dual-loss optimization strategy is used for simulation during the training phase of the network, that is, one set of image mappings is trained in one propagation direction (e.g., forward: A→B), and another set of image mappings is trained in the opposite propagation direction (e.g., reverse: B→A). The training loss function of this asymmetric multifunctional imaging method can be defined as:

[0062] Loss(L 正 ,L 反 ,O 正 ,O 反 )=MSE(L 反 ,O 反 )+MSE(L 正 ,O 正 )

[0063]

[0064] Where (x, y) represents the coordinates, V represents the x *N y Pixel-defined input and output plane, L 前 , O 前 Indicates the forward Label and output result, L 后 , O 后 Indicates the backward label and output result.

[0065] The following describes a specific embodiment of the spatial diffraction deep neural network asymmetric multifunctional imaging method provided by the present invention, and the specific implementation method is as follows:

[0066] First, a numerical simulation model of the optical component was established based on the angular spectrum method. Two sets of training data were collected: one set was used for the network's forward calculation, where both input and output were images of cats. The other set was used for the network's reverse calculation, where input was an image of a cat and output was an image of a dog, as shown in Figure 4(a). The numerical simulation model was then trained using gradient descent and backpropagation algorithms. The model successfully achieved asymmetric imaging based on different transmission directions, as shown in Figure 4(c). To demonstrate the versatility of the method and device, two other sets of training data were collected: one set was used to map cat input images to chicken outputs through forward calculation, and the other set was used to map cat input images to dog outputs through reverse calculation, as shown in Figure 4(b). Similarly, the simulation model was trained. The training results demonstrated that the method can achieve a variety of functions based on different input directions, as shown in Figure 4(d).

[0067] The present invention is not limited to the above-mentioned specific embodiments. A person skilled in the art can implement the present invention in a variety of other specific embodiments based on the contents disclosed in the present invention. Therefore, any design that adopts the design structure and ideas of the present invention and makes some simple changes or modifications falls within the scope of protection of the present invention.

Claims

1. A spatial diffraction deep neural network asymmetric multifunctional imaging method, characterized in that: The following steps are involved: A spatial diffraction neural network is constructed, and the spatial diffraction neural network is used as a carrier of optical imaging, so that an input light beam is imaged after being transmitted through the spatial diffraction neural network; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2...θ n ), where θ i is the phase of the i-th phase layer, 1≤i≤n; Using a first image output after the original input light beam is forwardly input to the spatial diffraction neural network and a second image output after the original input light beam is backwardly input to the spatial diffraction neural network as training data, using machine learning to train the spatial diffraction neural network, determining a phase combination, and obtaining a trained spatial diffraction neural network; The input light beam to be imaged is input forward and backward in sequence to the trained spatial diffraction neural network to obtain the first imaging and the second imaging.

2. The spatial diffraction deep neural network asymmetric multifunctional imaging method according to claim 1, characterized in that: The adopting of machine learning to train the spatial diffraction neural network includes: adopting an asymmetric double loss optimization strategy, wherein the double loss optimization function of the asymmetric double loss optimization strategy is: Loss = αL1 + βL2 Among them, L1 is the loss function of forward propagation, L2 is the loss function of back propagation, and α and β are weighting coefficients.

3. A spatial diffraction deep neural network asymmetric multifunctional imaging system, characterized in that: It includes a signal input module, a spatial diffraction deep neural network, and a signal output module; The signal input module is used to provide an input light beam carrying image information; The spatial diffraction deep neural network is used as a carrier of optical imaging, so that the input light beam is imaged after being transmitted forward and backward through the spatial diffraction neural network in sequence, thereby obtaining a first image and a second image; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2...θ n ), where θ i is the phase of the i-th phase layer, 1≤i≤n; The signal output module is used to display the imaging of the spatial diffraction deep neural network.

4. The spatial diffraction deep neural network asymmetric multifunctional imaging system according to claim 3 is characterized in that: The n phase layers are arranged according to a preset layer spacing, and the phase combinations (θ1, θ2, ... θ n ) is trained using a machine learning method, and the layer spacing does not participate in the optimization during the training process.

5. The spatial diffraction deep neural network asymmetric multifunctional imaging system according to claim 3, characterized in that: Each phase layer includes m×m pixels, and each pixel is used to achieve phase modulation in the range from 0 to 2π.

6. The spatial diffraction deep neural network asymmetric multifunctional imaging system according to claim 3, characterized in that: The processing material system of each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, and photosensitive polymer.

7. The spatial diffraction deep neural network asymmetric multifunctional imaging system according to claim 3, characterized in that: The medium between each phase layer is vacuum, gas, liquid or solid.

8. The spatial diffraction deep neural network asymmetric multifunctional imaging system according to claim 3, characterized in that: The input light field enters the spatial diffraction neural network after passing through a propagation medium, wherein the propagation medium includes air or a complex medium, and the complex medium includes a scattering medium, an absorbing medium or a nonlinear medium.

9. The spatial diffraction deep neural network asymmetric multifunctional imaging system according to claim 3, characterized in that: The spatial diffraction deep neural network adapts to light beams of different wavelengths and supports asymmetric imaging operations of single-wavelength and multi-wavelength input light beams.

10. The spatial diffraction deep neural network asymmetric multifunctional imaging system according to claim 3, characterized in that: The three-dimensional integration processes for realizing spatial diffraction deep neural networks include electron beam lithography and maskless lithography.

Images