Spatially diffracted deep neural network asymmetric multifunctional imaging method and system

Abstract

The application discloses a kind of spatial diffraction deep neural network asymmetric multifunctional imaging method and system, it is suitable for optical signal processing field.The system is composed of signal input module, all-optical diffraction neural network module and signal acquisition module.Signal input module is responsible for providing light source and modulation element, for generating and modulating optical field signal.All-optical diffraction neural network module uses double loss function optimization strategy to carry out asymmetric training.Specifically, it introduces different loss functions in different transmission directions to train different functions simultaneously, thereby realizing the diversification of signal processing.Signal acquisition module is responsible for collecting and storing the final calculation result.The application can realize different imaging functions according to the different signal input directions, thereby adapting to the diversification needs of optical signal processing, not only filling the gap in the related field, but also providing a new way of thinking for further research and application in the field of optical imaging.

Description

Technical Field

[0001] This invention belongs to the field of optical signal processing, and more specifically, relates to a spatial diffraction depth neural network asymmetric multifunctional imaging method and system. Background Technology

[0002] Spatial diffraction deep neural networks are a novel deep learning model that integrates the principles of optical diffraction and neural network algorithms. Optical diffraction is the phenomenon where light waves bend and overlap when they encounter obstacles or the edges of objects. This phenomenon is an indispensable foundation in optical imaging and information processing. By analyzing the diffraction patterns of light waves, information about the shape, size, and structure of objects can be obtained. This characteristic of diffraction makes it of significant application value in image processing, optical imaging, and pattern recognition. Secondly, neural networks are mathematical models that mimic the structure and function of neurons in the human brain. Through the connections and weight adjustments between neurons, neural networks can learn and extract features from input data, thereby achieving functions such as classification, recognition, and prediction. Deep learning is one of the fastest-growing methods in the field of machine learning. It utilizes multi-layered artificial neural networks implemented in computers to digitally learn and abstract data, enabling it to perform advanced functions and even, in some cases, outperform human experts. In recent years, deep learning has made significant progress in fields such as medical image analysis, speech recognition, language translation, and image classification. Combining optical diffraction and deep learning forms spatial diffraction deep neural networks. This combination fully leverages the information processing capabilities of optical diffraction and the learning abilities of neural networks, enabling faster processing and analysis of optical signals and images. However, spatial diffraction deep neural networks are highly sensitive to environmental conditions and are significantly affected by factors such as light, temperature, and humidity, which limits their practical applications. Therefore, exploring the application capabilities of spatial neural networks and further expanding their application scenarios is a current research focus. Summary of the Invention

[0003] This invention provides an asymmetric multifunctional imaging method and system based on spatial diffraction deep neural networks. Its unique feature lies in its ability to flexibly execute different functions based on varying signal input directions, thereby adapting to diverse needs in optical signal processing. This design enables the imaging method to effectively address diverse functional requirements and offers broad application prospects. This innovation fills a research gap in the field of spatial diffraction deep neural networks, bringing new development opportunities to the fields of optical imaging and information processing.

[0004] To achieve the above objectives, this invention provides an asymmetric multifunctional imaging method using a spatial diffraction deep neural network. A spatial diffraction neural network is constructed, which serves as a carrier for optical imaging, allowing the input beam to be imaged after transmission through the neural network. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θ...). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n;

[0005] The first image output after the original input beam is forward-inputted into the spatial diffraction neural network and the second image output after the original input beam is backward-inputted into the spatial diffraction neural network are used as training data. The spatial diffraction neural network is trained by machine learning to determine the phase combination and obtain the trained spatial diffraction neural network.

[0006] The input beam to be imaged is sequentially input into the trained spatial diffraction neural network in the forward and backward directions to obtain the first image and the second image.

[0007] Preferably, training the spatial diffraction neural network using machine learning includes: employing an asymmetric double-loss optimization strategy, wherein the double-loss optimization function of the asymmetric double-loss optimization strategy is:

[0008] Loss = αL1 + βL2

[0009] Where L1 is the loss function for forward propagation, L2 is the loss function for backward propagation, and α and β are weighting coefficients.

[0010] The present invention also provides a spatial diffraction deep neural network asymmetric multifunctional imaging system, including a signal input module, a spatial diffraction deep neural network, and a signal output module;

[0011] The signal input module is used to provide an input beam carrying image information;

[0012] The spatial diffraction depth neural network is used as a carrier for optical imaging, allowing the input beam to be transmitted sequentially forward and backward through the spatial diffraction neural network before imaging, resulting in a first image and a second image. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θ). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n;

[0013] The signal output module is used to display the imaging of the spatial diffraction depth neural network.

[0014] Preferably, the phase layers are arranged according to a preset layer spacing, and the phase combinations (θ1, θ2, ... θ) on the phase layers are... n The interlayer spacing is obtained by training using machine learning methods. It is not involved in the optimization during the training process. Instead, the optimal spacing value is selected as a fixed parameter by comparing the experimental results under different spacing conditions.

[0015] Preferably, each phase layer includes m×m pixels, and each pixel is used to achieve phase modulation in the range of 0 to 2π.

[0016] Preferably, the processing material system for each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, and photosensitive polymer.

[0017] Preferably, the medium between each phase layer is a vacuum, gas, liquid, or solid.

[0018] Preferably, the input light field enters the spatial diffraction neural network after passing through a propagation medium, and the propagation medium includes air or a complex medium, including but not limited to scattering media, absorption media or nonlinear media.

[0019] Preferably, the spatial diffraction deep neural network is adapted to light beams of different wavelengths, supporting asymmetric imaging operations with single-wavelength and multi-wavelength input light beams.

[0020] Preferably, the three-dimensional integration process for realizing spatial diffraction deep neural networks can be carried out in various ways, including but not limited to electron beam lithography and maskless lithography.

[0021] Compared with existing technologies, the technical solution conceived in this invention, employing a spatial diffraction neural network structure, multi-layer phase modulation, and an asymmetric loss optimization strategy, successfully achieves synergistic optimization of two imaging functions. It can simultaneously meet imaging requirements in different directions within a single device, providing an efficient, accurate, and multifunctional optical imaging solution. Based on the construction of a spatial diffraction deep neural network, an asymmetric training strategy using a dual loss function is employed, expanding the model's functionality and allowing it to flexibly execute different functions based on the signal input direction. This is fundamentally different from previous inventions. Furthermore, spatial diffraction deep neural networks are widely used in optical imaging, optical communication, information security, pattern recognition, optical sensing and detection, and other fields. Therefore, the asymmetric multifunctional imaging method using a spatial diffraction deep neural network provided by this invention will play a vital role in various future applications. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a spatial diffraction depth neural network asymmetric multifunctional imaging method provided by the present invention.

[0023] Figure 2 This is a diagram of the physical experimental setup for a spatial diffraction depth neural network asymmetric multifunctional imaging system provided by the present invention.

[0024] Figure 3 This is a simulation flowchart of the all-optical diffraction deep neural network module in an asymmetric multifunctional imaging method of spatial diffraction deep neural network provided in an embodiment of the present invention.

[0025] Figure 4(a) is a schematic diagram of an embodiment of a spatial diffraction depth neural network asymmetric multifunctional imaging method provided by the present invention.

[0026] Figure 4(b) is a schematic diagram of an embodiment of another spatial diffraction depth neural network asymmetric multifunctional imaging method provided by the present invention.

[0027] Figure 4(c) is a schematic diagram of the imaging effect of Figure 4(a) in the embodiment.

[0028] Figure 4(d) is a schematic diagram of the imaging effect of Figure 4(b) in the embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0030] To achieve the above objectives, a spatial diffraction depth neural network asymmetric multifunctional imaging method according to the present invention includes the following steps:

[0031] A spatial diffraction neural network is constructed, which serves as a carrier for optical imaging, enabling the input beam to be imaged after transmission through the spatial diffraction neural network. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θn). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n;

[0032] The first image output after the original input beam is forward-inputted into the spatial diffraction neural network and the second image output after the original input beam is backward-inputted into the spatial diffraction neural network are used as training data. The spatial diffraction neural network is trained by machine learning to determine the phase combination and obtain the trained spatial diffraction neural network.

[0033] The input beam to be imaged is sequentially input into the trained spatial diffraction neural network in the forward and backward directions to obtain the first image and the second image.

[0034] Specifically, training the spatial diffraction neural network using machine learning includes: employing an asymmetric double-loss optimization strategy, wherein the double-loss optimization function of the asymmetric double-loss optimization strategy is:

[0035] Loss = αL1 + βL2

[0036] Where L1 is the loss function for forward propagation, L2 is the loss function for backward propagation, and α and β are weighting coefficients.

[0037] The present invention also provides a spatial diffraction deep neural network asymmetric multifunctional imaging system, including a signal input module, a spatial diffraction deep neural network, and a signal output module;

[0038] The signal input module is used to provide an input beam carrying image information;

[0039] The spatial diffraction depth neural network is used as a carrier for optical imaging, allowing the input beam to be transmitted sequentially forward and backward through the spatial diffraction neural network before imaging, resulting in a first image and a second image. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θ). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n;

[0040] The signal output module is used to display the imaging of the spatial diffraction depth neural network.

[0041] Specifically, the phase layers are arranged according to a preset layer spacing, and the phase combinations (θ1, θ2, ... θ) on the phase layers are... n The interlayer spacing is obtained by training using machine learning methods. It is not involved in the optimization during the training process. Instead, the optimal spacing value is selected as a fixed parameter by comparing the experimental results under different spacing conditions.

[0042] Specifically, each phase layer includes m×m pixels, and each pixel is used to achieve phase modulation in the range of 0 to 2π.

[0043] Specifically, the processing material system for each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, and photosensitive polymer.

[0044] Specifically, the medium between each phase layer is a vacuum, gas, liquid, or solid.

[0045] Specifically, the input light field enters the spatial diffraction neural network after passing through a propagation medium, which includes air or a complex medium, including but not limited to scattering media, absorption media, or nonlinear media.

[0046] Specifically, the spatial diffraction deep neural network is adapted to light beams of different wavelengths and supports asymmetric imaging operations with single-wavelength and multi-wavelength input light beams.

[0047] Specifically, the three-dimensional integration process for realizing spatial diffraction deep neural networks can be carried out in various ways, including but not limited to electron beam lithography and maskless lithography.

[0048] The following description is based on specific embodiments and accompanying drawings.

[0049] like Figure 1 As shown, this invention provides a schematic diagram of an asymmetric multifunctional imaging method using a spatial diffraction depth neural network. The specific implementation is as follows:

[0050] The system comprises a signal input module, an all-optical diffraction neural network module, and an information acquisition module. The signal input module generates the input light field and can consist of a laser, a collimator, and a spatial light field modulation device. Specifically, the laser emits Gaussian light, the collimator converts the diverging beam into a parallel beam, and then the spatial light field modulation device modulates the signal according to preset information. The all-optical diffraction neural network module extracts and transforms the signal features. It consists of multiple phase layers, with both sides of the network as input terminals. A dual-loss function optimization strategy simulation model is introduced, allowing the model to perform different functions based on the direction of the input signal transmission. During the experimental phase, the trained and optimized model parameters are synchronized to the optical phase modulation layer. The information acquisition module acquires the output results of the spatial depth diffraction neural network. This module can consist of a CMOS camera and a computer. Specifically, the CMOS camera is placed after the all-optical diffraction neural network module, and the acquired light spot is uploaded to the computer for storage, thus completing the signal acquisition after the model output.

[0051] The following describes a specific embodiment of the spatial diffraction depth neural network asymmetric multifunctional imaging system provided by the present invention, as an example. Figure 2 The structural diagram shown below illustrates the specific structure as follows:

[0052] The system includes: a laser 1, a collimator 2, a spatial light field modulation device (SLM3), a first lens 4, a first objective lens 5, a first optical phase modulation layer 6, a second optical phase modulation layer 7, a third optical phase modulation layer 8, a fourth optical phase modulation layer 9, a second objective lens 10, a second lens 11, a CMOS camera 12, and a computer 13. The Gaussian light output from the laser 1 is stably transmitted and precisely aligned by the collimator 2, and then transmitted to the SLM3. The SLM3 modulates the light field with pre-defined signal information, and then, through a 4f system composed of the first lens 4 and the first objective lens 5, the signal light carrying the pre-modulated information is reduced and imaged into a full-optical diffraction neural network composed of the first optical phase modulation layer 6, the second optical phase modulation layer 7, the third optical phase modulation layer 8, and the fourth optical phase modulation layer 9. The beam is then expanded and collimated by the second objective lens 10 and the second lens 11 and transmitted to the CMOS camera 12 for acquisition. The acquisition results are then uploaded to the computer for storage.

[0053] The following describes a specific embodiment of the spatial diffraction depth neural network asymmetric multifunctional imaging method provided by the present invention, illustrating the establishment process of the all-optical depth diffraction neural network module. The parameters of the module in this embodiment are obtained by establishing a simulation model and optimizing it using deep learning methods. Specifically, Figure 3 The simulation flowchart for an asymmetric imaging method according to an embodiment of the present invention is as follows:

[0054] A numerical simulation model of a multifunctional optical calculator with a spatial diffraction deep neural network was established. The angular spectral method was used to numerically simulate the propagation and diffraction of light between phase modulation layers. Based on the principle of Fast Fourier Transform (FFT), the diffraction of light waves during propagation and the resulting diffracted scene were simulated. A deep learning network was established based on the simulation model, and asymmetric training was performed using a double-loss optimization strategy. The training parameters of the spatial diffraction deep neural network were optimized using gradient descent and backpropagation methods.

[0055] Specifically, assuming the input light field is U(x, y, z), where x and y are spatial coordinates on the plane, and z = 0 represents the input surface, the propagation of the light field U at the far field z is shown in the following equation:

[0056] U N (x,y,0)=U N-1 (x,y,0)e i*2π*tanh(phi)

[0057]

[0058] U N (x,y,z)=F -1 {F{U N (x,y,0)}*H(fx ,f y ,z,λ)}

[0059]

[0060] Where F represents the Fourier transform operator. H(f) is the wave number, λ is the wavelength, and H(f) is the wavelength. x ,f y Let f(z,λ) be the transfer function in free space, where f(z,λ) = f(z,λ) x and f y These represent the spatial frequencies along the x and y directions, respectively.

[0061] To achieve asymmetric and multifunctional imaging, a dual-loss optimization strategy is employed during network training. This involves training one set of image mappings in one propagation direction (e.g., forward: A→B) and another set in the opposite propagation direction (e.g., backward: B→A). The training loss function for this asymmetric multifunctional imaging method can be defined as:

[0062] Loss(L 正 ,L 反 O 正 O 反 ) = MSE(L 反 O 反 )+MSE(L 正 O 正 )

[0063]

[0064] Where (x, y) represents the coordinates, and V represents the coordinates of N. x *N y The input / output plane defined by pixels, L 前 O 前 L represents the forward label and the output result. 后 O 后 This indicates the label and output result for the backward movement.

[0065] The following describes a specific embodiment of the spatial diffraction depth neural network asymmetric multifunctional imaging method provided by the present invention. The specific implementation method is as follows:

[0066] First, a numerical simulation model of the optical element was established based on the angular spectrum method. Two sets of training data were collected: one set was used to train the forward computation of the network, with the input and output being the same image of a cat; the other set was used to train the backward computation of the network, with the input being an image of a cat and the output being an image of a dog, as shown in Figure 4(a). Then, the numerical simulation model was trained using gradient descent and backpropagation algorithms. The model successfully achieved asymmetric imaging according to different transmission directions, as shown in Figure 4(c). To demonstrate the versatility of the method and device functions, two more sets of different training data were collected: one set was used to map the input image of a cat to the output image of a chicken through forward computation; the other set was used to map the input image of a cat to the output image of a dog through backward computation, as shown in Figure 4(b). Similarly, the simulation model was trained, and the results after training showed that the method can achieve rich functions according to different input directions, as shown in Figure 4(d).

[0067] This invention is not limited to the specific embodiments described above. Those skilled in the art can implement this invention using various other specific embodiments based on the content disclosed herein. Therefore, any design that adopts the design structure and concept of this invention and makes some simple changes or modifications falls within the protection scope of this invention.

Claims

1. A spatial diffraction deep neural network asymmetric multifunctional imaging method, characterized in that, Includes the following steps: A spatial diffraction neural network is constructed, which serves as a carrier for optical imaging, enabling the input beam to be imaged after being transmitted through the spatial diffraction neural network. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is ( θ 1, θ 2…… θ n ),in θ i Let i be the phase of the i-th phase layer, 1≤i≤n; The first image output after the original input beam is forward-inputted into the spatial diffraction neural network and the second image output after the original input beam is backward-inputted into the spatial diffraction neural network are used as training data. The spatial diffraction neural network is trained by machine learning to determine the phase combination and obtain the trained spatial diffraction neural network. The step of training the spatial diffraction neural network using machine learning includes: employing an asymmetric double-loss optimization strategy, wherein the double-loss optimization function of the asymmetric double-loss optimization strategy is: L us= αL 1+ β L 2 in, L 1 represents the loss function for forward propagation. L 2 represents the loss function for backpropagation. and These are weighting coefficients; The input beam to be imaged is sequentially input into the trained spatial diffraction neural network in the forward and backward directions to obtain the first image and the second image.

2. A spatial diffraction deep neural network asymmetric multifunctional imaging system, characterized in that, Includes a signal input module, a spatial diffraction deep neural network, and a signal output module; The signal input module is used to provide an input beam carrying image information; The spatial diffraction depth neural network is used as a carrier for optical imaging, allowing the input beam to be transmitted sequentially forward and backward through the spatial diffraction neural network before imaging, resulting in a first image and a second image; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is ( θ 1, θ 2…… θ n ),in θ i The phase of the i-th phase layer, 1≤i≤n; the n phase layers are arranged according to a preset layer spacing, and the phase combination on the phase layer ( θ 1, θ 2…… θ n The interlayer spacing is obtained by training using machine learning methods and is not included in the optimization during the training process. The step of training the spatial diffraction neural network using machine learning includes: employing an asymmetric double-loss optimization strategy, wherein the double-loss optimization function of the asymmetric double-loss optimization strategy is: L us= αL 1+ β L 2 in, L 1 represents the loss function for forward propagation. L 2 represents the loss function for backpropagation. and These are weighting coefficients; The signal output module is used to display the imaging of the spatial diffraction depth neural network.

3. The spatial diffraction depth neural network asymmetric multifunctional imaging system according to claim 2, characterized in that, Each phase layer consists of m×m pixels, with each pixel representing a value from 0 to 2. Phase modulation within the range.

4. The spatial diffraction depth neural network asymmetric multifunctional imaging system according to claim 2, characterized in that, The processing material system for each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, or photosensitive polymer.

5. The spatial diffraction depth neural network asymmetric multifunctional imaging system according to claim 2, characterized in that, The medium between each phase layer can be a vacuum, gas, liquid, or solid.

6. The spatial diffraction depth neural network asymmetric multifunctional imaging system according to claim 2, characterized in that, The input beam enters the spatial diffraction neural network after passing through a propagation medium, which includes air or a complex medium, including a scattering medium, an absorption medium, or a nonlinear medium.

7. The spatial diffraction depth neural network asymmetric multifunctional imaging system according to claim 2, characterized in that, The spatial diffraction deep neural network is adapted to beams of different wavelengths and supports asymmetric imaging operations with single-wavelength and multi-wavelength input beams.

8. The spatial diffraction depth neural network asymmetric multifunctional imaging system according to claim 2, characterized in that, The three-dimensional integration process for realizing spatial diffraction deep neural networks includes electron beam lithography and maskless lithography.

Images