Spatially diffracted deep neural network versatile optical computing method and system

Abstract

This invention discloses a multifunctional optical computation method and system using a spatial diffraction deep neural network, applicable to the field of optical signal processing. Combining feature construction techniques and utilizing a multi-phase surface structure, this invention solves various numerical computation problems with low loss and high efficiency. The method creates spatial features by dividing the output plane into regions and fusing them with the digital information of the input light field to construct new interactive features as training labels. By training the spatial diffraction deep neural network, a mapping relationship between digital information and computation results is established. Finally, the trained model parameters are synchronized to the spatial diffraction deep neural network component, enabling the solution of various numerical computation problems.

Description

Technical Field

[0001] This invention belongs to the field of optical signal processing, and more specifically, relates to a multifunctional optical computing method and system for spatial diffraction deep neural networks. Background Technology

[0002] Spatial diffraction deep neural networks represent a significant intersection of optics and computing, integrating the principles of optical diffraction with neural network algorithms. Optical diffraction is a natural phenomenon where light waves are deflected and overlapped as they pass through the edges of objects or obstacles. This phenomenon plays a crucial role in optical imaging and information processing because analyzing diffraction patterns can reveal information about an object's shape, size, and structure. Neural networks, on the other hand, are mathematical models inspired by biological nervous systems, using connections and weight adjustments between neurons to extract and learn features from input data. Deep learning, as a major branch of machine learning, has achieved tremendous success in recent years, particularly in areas such as medical image analysis, speech recognition, natural language processing, and image classification. The innovation of spatial diffraction deep neural networks lies in combining the principles of optical diffraction with deep learning. By leveraging the information processing capabilities of optical diffraction and the learning capabilities of neural networks, spatial diffraction deep neural networks can process and analyze optical signals and images more quickly. However, despite their immense potential, spatial diffraction deep neural networks still face some challenges in practical applications. First, they are highly sensitive to environmental conditions and easily affected by external factors such as light, temperature, and humidity. This can lead to network instability, thus limiting its reliability and stability in practical applications. Secondly, although spatial diffraction deep neural networks are theoretically capable of handling complex nonlinear problems, designing effective network structures and algorithms remains a challenge in practice. However, despite these challenges, researchers remain confident in the application potential of spatial diffraction deep neural networks. Future research directions include further optimizing network structures and algorithms, improving network robustness and stability, and exploring broader application scenarios. With continuous technological advancements and in-depth theoretical understanding, it is believed that spatial diffraction deep neural networks will play an increasingly important role in optical imaging, information processing, and other fields. Summary of the Invention

[0003] This invention provides a multifunctional optical computing method and system using spatial diffraction deep neural networks. Its unique feature lies in the ability of spatial diffraction neural networks to solve various linear and nonlinear operations. This design enables the optical computing method to flexibly address diverse computational needs and perform multiple operations at the speed of light. This innovation fills a research gap in the field of spatial diffraction deep neural networks and brings new development opportunities to the field of optical signal processing.

[0004] To achieve the above objectives, this invention provides a multifunctional optical computing method using spatial diffraction deep neural networks, comprising the following steps:

[0005] A spatial diffraction neural network is constructed, which serves as the carrier for optical computation. The input data is a light field containing two digital information points generated by a spatial light modulator. This light field is converted into a beam and transmitted to the spatial diffraction neural network to generate a light spot. The imaging position of the light spot represents the computational result of the two digital information points in the input light field. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θ). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n; the output plane consists of multiple grids, each grid representing a different computation result. The size of the light spot is smaller than the grid size, and the distance of the light spot from the edge in the grid is determined by the input digital information to avoid crosstalk.

[0006] The spatial diffraction neural network is trained using the input light field and its spot imaging position on the output plane as training data, and the phase combination is optimized to obtain a trained spatial diffraction neural network.

[0007] The digital information to be calculated is converted into a light field and input into a trained spatial diffraction neural network. The imaging position of the output light spot is the calculation result.

[0008] Furthermore, the determination of the light spot imaging position includes: dividing the output plane into multiple grid regions, each grid region representing a different calculation result, and determining the grid into which the light spot should fall based on the calculation result of two numbers in the input light field; establishing a local coordinate system within the grid, and further determining the specific position of the light spot within the grid based on the two numbers in the input light field as horizontal and vertical coordinate information.

[0009] Preferably, the spatial diffraction neural network is trained using machine learning, a loss function is used to measure the difference between the real target and the network output, and the gradient of the network parameters is obtained based on this difference using gradient descent. Then, the parameters are updated using backpropagation. The loss function can be the mean squared error (MSE) function.

[0010]

[0011] Among them, y pre The output of the network is y, where y is the target value.

[0012] The present invention also provides a multifunctional optical computing system for spatial diffraction deep neural networks, including a signal input module, a spatial diffraction deep neural network, and a signal output module;

[0013] The signal input module is used to provide an optical field containing two digital information;

[0014] The spatial diffraction deep neural network serves as a carrier for optical computation, enabling the generation of light spots after the light field is transmitted through the neural network. The imaging position of the light spot represents the computational result of two numbers in the input light field. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θ). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n; the output plane consists of multiple grids, each grid representing a different computation result. The size of the light spot is smaller than the grid size, and the distance of the light spot from the edge of the grid is determined by the input digital information.

[0015] The signal output module is used to display the light spot.

[0016] Preferably, the phase layers are arranged according to a preset layer spacing, and the phase combinations (θ1, θ2, ... θ) on the phase layers are... n The interlayer spacing is obtained by training using machine learning methods. It is not involved in the optimization during the training process. Instead, the optimal spacing value is selected as a fixed parameter by comparing the experimental results under different spacing conditions.

[0017] Preferably, each phase layer includes m×m pixels, and each pixel is used to achieve phase modulation in the range of 0 to 2π.

[0018] Preferably, the processing material system for each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, and photosensitive polymer.

[0019] Preferably, the medium between each phase layer is a vacuum, gas, liquid, or solid.

[0020] Preferably, the signal output module includes an output plane, where each region in the output plane represents a different calculation result, and the region division method can be dynamically adjusted according to actual task requirements.

[0021] Compared with existing technologies, this invention proposes a multifunctional optical computing method and system based on a spatial diffraction deep neural network. Through the construction of a spatial diffraction deep neural network, feature construction techniques, and machine learning optimization, it successfully applies spatial diffraction deep neural networks to various numerical computation problems, exhibiting significant technical advantages such as low loss, high efficiency, and scalability. Based on the construction of the spatial diffraction deep neural network, feature construction techniques are used to expand its functionality, enabling flexible execution of multiple types of computational tasks. This is fundamentally different from previous inventions. Furthermore, spatial diffraction deep neural networks are widely used in optical imaging, optical communication, information security, pattern recognition, and optical sensing and detection. The multifunctional optical computing method and system based on a spatial diffraction deep neural network proposed in this invention can solve multiple types of computations, providing a solid research foundation for the application of spatial diffraction deep neural networks. Therefore, the multifunctional optical computing method and system based on a spatial diffraction deep neural network provided by this invention will play a very important application role in various fields in the future. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a multifunctional optical computing method using a spatial diffraction deep neural network provided by the present invention.

[0023] Figure 2 This is a schematic diagram of the physical experiment of a multifunctional optical computing system for spatial diffraction deep neural networks provided by the present invention.

[0024] Figure 3 This is a simulation flowchart of the all-optical diffraction deep neural network module in a multifunctional optical computing method for spatial diffraction deep neural networks provided in an embodiment of the present invention.

[0025] Figure 4(a) is a schematic diagram of an output plane in a multifunctional optical computing method for spatial diffraction deep neural networks provided by the present invention.

[0026] Figure 4(b) is a feature engineering design diagram provided in an embodiment of the present invention.

[0027] Figure 4(c) is a schematic diagram of the output results provided by the embodiment of the present invention.

[0028] Figure 4(d) is a schematic diagram of a multifunctional optical computing method for spatial diffraction deep neural networks provided in an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0030] To achieve the above objectives, a multifunctional optical computing method using a spatial diffraction deep neural network, provided by this invention, includes the following steps:

[0031] A spatial diffraction neural network is constructed, which serves as the carrier for optical computation. The input data is a light field containing two digital information points generated by a spatial light modulator, which is transmitted to the spatial diffraction neural network to generate a light spot. The imaging position of the light spot represents the computational result of the two digital information points in the input light field. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θ...). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n; the output plane consists of multiple grids, each grid representing a different computation result. The size of the light spot is smaller than the grid size, and the distance of the light spot from the edge in the grid is determined by the input digital information to avoid crosstalk.

[0032] The spatial diffraction neural network is trained using the input light field and its spot imaging position on the output plane as training data, and the phase combination is optimized to obtain a trained spatial diffraction neural network.

[0033] The digital light field to be calculated is converted into a light beam and input into a trained spatial diffraction neural network. The imaging position of the output light spot is the calculation result.

[0034] Specifically, determining the imaging position of the light spot includes: dividing the output plane into multiple grid regions, each grid region representing a different calculation result, and determining the grid into which the light spot should fall based on the calculation result of two numbers in the input light field; establishing a local coordinate system within the grid, and further determining the specific position of the light spot within the grid based on the two numbers in the input light field as horizontal and vertical coordinate information.

[0035] Specifically, machine learning is used to train the spatial diffraction neural network. A loss function is used to measure the difference between the real target and the network output. Based on this difference, the gradient of the network parameters is obtained using gradient descent, and then the parameters are updated using backpropagation. The loss function can be the mean squared error (MSE) function.

[0036]

[0037] Among them, y pre The output of the network is y, where y is the target value.

[0038] The present invention also provides a multifunctional optical computing system for spatial diffraction deep neural networks, including a signal input module, a spatial diffraction deep neural network, and a signal output module;

[0039] The signal input module is used to provide an optical field containing two digital information;

[0040] The spatial diffraction deep neural network serves as a carrier for optical computation, enabling the generation of light spots after the light field is transmitted through the neural network. The imaging position of the light spot represents the computational result of two numbers in the input light field. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (θ1, θ2, ..., θ). n ), where θ i Let i be the phase of the i-th phase layer, 1≤i≤n; the output plane consists of multiple grids, each grid representing a different computation result. The size of the light spot is smaller than the grid size, and the distance of the light spot from the edge of the grid is determined by the input digital information.

[0041] The signal output module is used to display the light spot.

[0042] Specifically, the phase layers are arranged according to a preset layer spacing, and the phase combinations (θ1, θ2, ... θ) on the phase layers are... n The interlayer spacing is obtained by training using machine learning methods. It is not involved in the optimization during the training process. Instead, the optimal spacing value is selected as a fixed parameter by comparing the experimental results under different spacing conditions.

[0043] Specifically, each phase layer includes m×m pixels, and each pixel is used to achieve phase modulation in the range of 0 to 2π.

[0044] Specifically, the processing material system for each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, and photosensitive polymer.

[0045] Specifically, the medium between each phase layer is a vacuum, gas, liquid, or solid.

[0046] Specifically, the signal output module includes an output plane, where each region represents a different calculation result, and the region division method can be dynamically adjusted according to actual task requirements.

[0047] The following description is based on specific embodiments and accompanying drawings.

[0048] like Figure 1As shown, this invention provides a schematic diagram of a multifunctional optical computing method using a spatial diffraction deep neural network. The specific implementation is as follows:

[0049] This embodiment employs a spatial diffraction deep neural network to extract signal features, which consists of multiple phase layers. During the model simulation, features were constructed on the output data. Specifically, the output plane was divided into regions to construct spatial features representing the computation results. Then, the spatial features were combined with digital information from the input light field to construct new interactive features. Specifically, within the region, light spots can be generated based on the digital information from the input light field as horizontal and vertical coordinates, and the light spot map is used as a training label for a specific computation task. Subsequently, the spatial diffraction deep neural network establishes a mapping relationship between the digital information of the input light field and the training label. Finally, the trained model parameters are synchronized to the network components, enabling the final optical computing system to perform multiple types of linear and nonlinear computations relying solely on a linear model.

[0050] The following describes a specific embodiment of the spatial diffraction deep neural network multifunctional optical computing system provided by the present invention, as an example. Figure 2 The structural diagram shown below illustrates the specific structure as follows:

[0051] The device includes: a laser 1, a collimator 2, a spatial light field modulation device (SLM) 3, a first lens 4, a first objective lens 5, a first optical phase modulation layer 6, a second optical phase modulation layer 7, a third optical phase modulation layer 8, a fourth optical phase modulation layer 9, a second objective lens 10, a second lens 11, a CMOS camera 12, and a computer 13. The Gaussian light output from the laser 1 is stably transmitted and precisely aligned by the collimator 2, and then transmitted to the SLM 3. The SLM 3 modulates the light field with pre-defined signal information, and then, through a 4f system composed of the first lens 4 and the first objective lens 5, the signal light carrying the pre-modulated information is reduced and imaged into a full-optical diffraction neural network composed of the first optical phase modulation layer 6, the second optical phase modulation layer 7, the third optical phase modulation layer 8, and the fourth optical phase modulation layer 9. The beam is then expanded and collimated by the second objective lens 10 and the second lens 11 and transmitted to the CMOS camera 12 for acquisition. The acquisition results are then uploaded to the computer for storage.

[0052] The following describes a specific embodiment of the multifunctional optical computation method using a spatial diffraction deep neural network provided by this invention, illustrating the process of establishing an all-optical depth diffraction neural network module. The parameters of the module in this embodiment are obtained by establishing a simulation model and optimizing it using deep learning methods. Specifically, Figure 3 The simulation flowchart for a multifunctional optical computing method according to an embodiment of the present invention is as follows:

[0053] A numerical simulation model of a multifunctional optical computation method using a spatial diffraction deep neural network is established. During the simulation, the output plane is divided into regions to construct spatial features representing the computation results. Then, these spatial features are combined with digital information from the input light field to construct new interactive features, which are used as training labels for specific computational tasks. Subsequently, a spatial diffraction deep neural network model is established to construct the mapping relationship between the input and the labels. The model uses the angular spectrum method to numerically simulate the propagation and diffraction of light between phase modulation layers. Based on the principle of Fast Fourier Transform (FFT), the diffraction of light waves during propagation and the diffracted scene are simulated. Finally, the training parameters of the all-optical deep diffraction neural network are optimized using gradient descent and backpropagation methods.

[0054] Specifically, assuming the input light field is U(x, y, z), where x and y are spatial coordinates on the plane, and z = 0 represents the input surface, the propagation of the light field U at the far field z is shown in the following equation:

[0055] U N (x,y,0)=U N-1 (x,y,0)e i*2π*tanh(phi)

[0056]

[0057] U N (x,y,z)=F -1 {F{U N (x,y,0)}*H(f x ,f y ,z,λ)}

[0058]

[0059] Where F represents the Fourier transform operator. H(f) is the wave number, λ is the wavelength, and H(f) is the wavelength. x ,f y Let f(z,λ) be the transfer function in free space, where f(z,λ) = f(z,λ) x and f y These represent the spatial frequencies along the x and y directions, respectively.

[0060] Regarding the training strategy, in order to make the output results as close as possible to the training labels, the mean squared error can be selected as the loss function, and the calculation formula can be defined as:

[0061]

[0062] Where (x, y) represents the coordinates, and V represents the coordinates of N. x *N y The output plane region defined by pixels.

[0063] The following example, using addition as the calculation method, details an embodiment of the spatial diffraction deep neural network multifunctional optical calculation method provided by this invention.

[0064] Taking a 125*125 planar region as an example, it is divided into 25 squares as the output region, each square being 25*25 pixels in size. The output planar image is shown in Figure 4(a). Taking x+y=z as an example, if the size of the output spot is n*n, then the feature construction of the output label for this method should satisfy that within the square with region number z, the coordinates of the upper left corner of the target spot (x+y=z) are... 1 ,y 1 It should satisfy:

[0065] (x 1 ,y 1 )=n*(x,y)+m

[0066] m is the offset, used to ensure that no light spot falls on the edge of the output area, thus affecting the final decision-making effect. A schematic diagram of the feature construction is shown in Figure 4(b). Figure 4(c) is a schematic diagram of the output result provided by an embodiment of the present invention.

[0067] The following is a specific embodiment of the spatial diffraction deep neural network multifunctional optical computing method provided by the present invention, as shown in Figure 4(d). The specific implementation method is as follows:

[0068] First, a numerical simulation model of a spatial diffraction deep neural network is established based on the angular spectrum method. According to the calculation method and input data, the calculation results are used to construct features and serve as training labels. Then, the simulation model is used for training, and the trained model parameters are synchronized to the physical layer to complete the construction of the optical computing system. This enables the device to implement specific calculation methods based on the spatial diffraction deep neural network.

[0069] This invention is not limited to the specific embodiments described above. Those skilled in the art can implement this invention using various other specific embodiments based on the content disclosed herein. Therefore, any design that adopts the design structure and concept of this invention and makes some simple changes or modifications falls within the scope of protection of this invention.

Claims

1. A multifunctional optical computation method using spatial diffraction deep neural networks, characterized in that, The method includes the following steps: A spatial diffraction neural network is constructed, which serves as the carrier for optical computation. The input data is an optical field containing two numerical information values, which is transmitted to the spatial diffraction neural network to generate a light spot. The imaging position of the light spot represents the computational result of the two numerical values ​​in the input optical field. The spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is (…). θ 1, θ 2…… θ n ),in θ i Let be the phase of the i-th phase layer, 1≤i≤n; the output plane is composed of multiple grids, each grid representing a different computation result. The size of the light spot is smaller than the grid size, and the distance of the light spot from the edge of the grid is determined by the input digital information; the determination of the imaging position of the light spot includes: dividing the output plane into multiple grid regions, each grid region representing a different computation result, and determining the grid that the light spot should fall into based on the computation result of two numbers in the input light field; establishing a local coordinate system within the grid, and determining the specific position of the light spot within the grid based on the two numbers in the input light field as the horizontal and vertical coordinate information; The spatial diffraction neural network is trained using the input light field and its spot imaging position on the output plane as training data, and the phase combination is optimized to obtain a trained spatial diffraction neural network. The digital information to be calculated is converted into a light field and input into a trained spatial diffraction neural network. The imaging position of the output light spot is the calculation result.

2. The multifunctional optical computing method using a spatial diffraction deep neural network according to claim 1, characterized in that, The spatial diffraction neural network is trained using machine learning methods. A loss function is used to measure the difference between the real target and the network output. Based on this difference, the gradient of the network parameters is obtained using gradient descent, and then the parameters are updated using backpropagation. The loss function is the mean squared error (MSE) function. in, For network output results, This is the target value.

3. A multifunctional optical computing system based on spatial diffraction deep neural networks, characterized in that, It includes a signal input module, a spatial diffraction deep neural network, and a signal output module; The signal input module is used to provide an optical field containing two digital information; The spatial diffraction deep neural network is used as a carrier for optical computing, so that the light field is transmitted through the spatial diffraction neural network to generate light spots. The imaging position of the light spot represents the computational result of two numbers in the input light field; the spatial diffraction neural network includes n phase layers, and the phase combination of all phase layers is ( θ 1, θ 2…… θ n ),in θ i Let i be the phase of the i-th phase layer, 1≤i≤n; The signal output module includes an output plane, which is composed of multiple grid regions. Each grid region represents a different calculation result. The calculation result of two numbers in the input light field determines the grid where the light spot should fall. A local coordinate system is established within the grid, and the specific position of the light spot within the grid is determined based on the two numbers in the input light field as horizontal and vertical coordinate information. The size of the light spot is smaller than the grid size, and the distance of the light spot from the edge of the grid is determined by the input digital information. The signal output module is used to display the light spot.

4. The spatial diffraction deep neural network multifunctional optical computing system according to claim 3, characterized in that, The phase layers are arranged according to a preset layer spacing, and the phase combinations on the phase layers ( θ 1, θ 2…… θ n The interlayer spacing is obtained by training using machine learning methods and is not involved in the optimization during the training process.

5. The spatial diffraction deep neural network multifunctional optical computing system according to claim 3, characterized in that, Each phase layer consists of m×m pixels, with each pixel representing a value from 0 to 2. Phase modulation within the range.

6. The spatial diffraction deep neural network multifunctional optical computing system according to claim 3, characterized in that, The processing material system for each phase layer includes any one of gallium nitride, aluminum oxide, metal, silicon, or photosensitive polymer.

7. The spatial diffraction deep neural network multifunctional optical computing system according to claim 3, characterized in that, The medium between each phase layer can be a vacuum, gas, liquid, or solid.

8. The spatial diffraction deep neural network multifunctional optical computing system according to claim 3, characterized in that, The input light field enters the spatial diffraction neural network after passing through a propagation medium, which includes air or a complex medium, including a scattering medium, an absorption medium, or a nonlinear medium.

Images