Method for identifying unknown target in complex environment by using all-optical diffraction neural network

Abstract

A method for identifying unknown targets in complex environments using a full-optical diffraction neural network, the full-optical diffraction neural network adopted is composed of multiple layers of phase-type diffractive optical elements, which can perform multiple wavefront regulation on the light field corresponding to the input image, and after a certain distance of diffraction propagation, the classification and identification of the image are completed in the optical domain; on the basis of the diffraction neural network, an anti-interference recognition mechanism is proposed, the concepts of target objects and interference objects to be specifically classified are proposed during the training of the diffraction neural network, and loss functions and constraint conditions are set respectively; then combined with the wavelength multiplexing mechanism, different target classification tasks are independently processed under different wavelength channels, so as to realize the classification task of unknown targets in complex environments by integrating the classification results of all wavelength channels in the full-waveband range, and to play the advantages of high speed and low energy consumption of the full-optical diffraction neural network, and to overcome the limitations such as simple application scene and small number of targets of the optical neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical neural networks, and in particular to a method for identifying unknown targets in a complex environment using a full-optical diffractive neural network. BACKGROUND

[0002] Optical neural networks exhibit light-speed-level operation, ultra-low power consumption, and natural parallel processing capabilities in object recognition. New architecture mechanisms, training methods, and implementation platforms have been developed to maximize the advantages of optical computing and to play a role in practical applications. In the application of optical neural networks to target recognition, diffractive optical neural networks, on-chip optical neural networks, holographic networks, and optoelectronic hybrid neural networks have been applied to single-target recognition tasks. While each of the above methods has its own characteristics, they all have limitations such as the inability to implement multi-target recognition and the inability to recognize targets with varying sizes and spatial positions. SUMMARY

[0003] The present application relates to the field of optical neural networks, and in particular to a method for identifying unknown targets in a complex environment using a full-optical diffractive neural network.

[0004] The present application is achieved by the following technical solutions:

[0005] The method for identifying unknown targets in a complex environment using a full-optical diffractive neural network includes:

[0006] A wavelength-multiplexed anti-interference full-optical diffractive neural network is built, wherein the anti-interference full-optical diffractive network includes an input layer, a full-optical diffractive layer, and an output layer; the full-optical diffractive layer includes a plurality of wavelength channels, each wavelength channel performing a different anti-interference single-target classification task;

[0007] The anti-interference full-optical diffractive neural network is trained, and the phase coefficients of the neurons in the full-optical diffractive layer are determined;

[0008] Based on the phase coefficients of the neurons, design the structural parameters of the wavelength multiplexing metasurface;

[0009] Based on the structural parameters of the wavelength multiplexing metasurface, a wavelength multiplexing metasurface is prepared; the prepared wavelength multiplexing metasurface is verified to determine the target recognition performance of the wavelength multiplexing metasurface.

[0010] Optionally, constructing a wavelength-multiplexed, interference-resistant all-optical diffraction neural network includes:

[0011] Construct an input layer, a full-optical diffraction layer, and an output layer; among which,

[0012] The input layer is used to perform binarization processing on the multi-target scene image, so that the transmittance of the region where each target is located in the multi-target scene image is 1, and the transmittance of other regions is 0.

[0013] The all-optical diffraction layer is an array composed of several diffraction neurons, each corresponding to a wavelength channel; the diffraction neurons are capable of performing complex amplitude modulation on the light field from the input layer.

[0014] The output layer is used to output the energy distribution of the light field modulated by the complex amplitude.

[0015] Optionally, training the anti-interference all-optical diffraction neural network includes:

[0016] Set up a training dataset to train the anti-interference all-optical diffraction neural network;

[0017] A loss function and optimizer are set up to iterate the loss function of the anti-interference all-optical diffraction neural network in reverse for each training iteration, thereby completing the training of the anti-interference all-optical diffraction neural network.

[0018] Optionally, a loss function and optimizer are set to iteratively process the loss function of the anti-interference all-optical diffraction neural network for each training iteration, thereby completing the training of the anti-interference all-optical diffraction neural network, including:

[0019] Each time the anti-interference all-optical diffraction neural network is trained, the output result is obtained through forward propagation, and a loss function is set based on the output result and the target result;

[0020] The gradient derivative of the loss function is calculated to obtain the gradient information of the loss function relative to the phase coefficients of each layer of neurons. The backpropagation algorithm is then used to pass all the gradient information back layer by layer. The Adam optimizer is set to update and adjust the phase coefficients of each layer of neurons with the corresponding learning rate. This process is repeated iteratively to reduce the loss function, thereby completing the training of the anti-interference all-optical diffraction neural network and determining the phase coefficients of the neurons in the all-optical diffraction layer.

[0021] Optionally, the method further includes:

[0022] After the anti-interference all-optical diffraction neural network is trained, each wavelength channel of the anti-interference all-optical diffraction neural network executes different optical field modulation strategies for the corresponding target and interfering objects. The optical field corresponding to the target object, after being modulated by the wavelength channel, appears as a light spot focused in a specific area in the output plane. The optical field corresponding to the interfering object, after being modulated by the wavelength channel, appears as low-density noise uniformly distributed across the entire output plane.

[0023] Optionally, based on the phase coefficient of the neuron, the structural parameters of the wavelength multiplexing metasurface are designed, including:

[0024] Based on the phase coefficient of the neuron, the transmission phase response of micro-nano structures of different sizes at different wavelengths is scanned using rigorous coupled-wave analysis, thereby selecting micro-nano structures that meet the preset anti-rotation and transmittance conditions to form a candidate structure library.

[0025] Select matching micro / nano structures from the candidate structure library, and introduce corresponding geometric phases to the matching micro / nano structures so that the phase response of the matching micro / nano structures can achieve uniform coverage of 0-2π at different wavelengths.

[0026] Based on the structural parameters of the matched micro / nano structures, the structural parameters of the wavelength multiplexing metasurface are designed.

[0027] Optionally, a wavelength-multiplexing metasurface is fabricated based on the structural parameters of the wavelength-multiplexing metasurface, including:

[0028] Based on the structural parameters of the wavelength multiplexing metasurface, a master mask is prepared;

[0029] The substrate is pretreated and a hard mask is deposited on the substrate, and photoresist is spin-coated onto the hard mask;

[0030] Using the master mask, the substrate is subjected to ultraviolet exposure, development, and removal of photoresist in the exposed areas to form a pattern consistent with the master mask.

[0031] The substrate is subjected to ion etching and hard mask forming, followed by pattern transfer and residual photoresist removal to obtain a wavelength-reusable metasurface.

[0032] Optionally, before verifying the prepared wavelength-multiplexing metasurface, the method further includes:

[0033] Multiple wavelength-multiplexing metasurfaces are used to assemble optical computing devices;

[0034] A verification optical path is constructed; wherein, the verification optical path includes, in sequence along the light transmission direction, a light source, a mirror group, a first polarizer, a first quarter-wave plate, a deflecting mirror, a digital micro-modulator, a 4f beam-shrinking system, the optical computing device, a microscope objective, a second quarter-wave plate, a second polarizer, and a near-infrared camera.

[0035] Optionally, the prepared wavelength-multiplexing metasurface is verified to determine its target recognition performance, including:

[0036] The image output by the near-infrared camera is acquired using the verification optical path, and the target recognition performance of the wavelength multiplexing metasurface is determined based on the light field intensity distribution within the output image.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The method for identifying unknown targets in complex environments using an all-optical diffraction neural network provided by this invention employs an all-optical diffraction neural network composed of multi-layer phase-type diffraction optical elements. During propagation, optical image recognition is achieved by reconstructing the input image multiple times from the wavefront. Based on the diffraction neural network, an anti-interference recognition mechanism is proposed. During the training phase of the diffraction neural network, the concepts of target objects and interference objects to be specifically classified are proposed, and loss functions and constraints are set for each. An error backpropagation algorithm is used to optimize the output to match the expected value. Furthermore, combined with a wavelength multiplexing mechanism, different target object classification tasks are processed independently under different wavelength channels. Thus, by integrating the classification results of all wavelength channels across the entire wavelength range, the classification task of unknown targets in complex environments can be achieved. This leverages the advantages of high speed and low energy consumption of the all-optical diffraction neural network, overcoming the limitations of optical neural networks such as simple application scenarios and small number of targets. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0040] Figure 1 This is a flowchart illustrating the method for identifying unknown targets in complex environments using an all-optical diffraction neural network, as provided by the present invention.

[0041] Figure 2 This is a schematic diagram of an anti-interference all-optical diffraction neural network that uses wavelength multiplexing.

[0042] Figure 3 This is a schematic diagram of the architecture of an anti-interference all-optical diffraction neural network that uses wavelength multiplexing.

[0043] Figure 4 This is a schematic diagram of the training process for an anti-interference all-optical diffraction neural network that uses wavelength multiplexing.

[0044] Figure 5 A schematic diagram of the design process for wavelength-reused metasurfaces.

[0045] Figure 6 This is a schematic diagram of the fabrication process of a wavelength-reused metasurface.

[0046] Figure 7 This is a schematic diagram of the experimental verification of an anti-interference all-optical diffraction neural network with wavelength multiplexing.

[0047] Figure 8 It is the confusion matrix of the classification accuracy of the test set of the anti-interference all-optical diffraction neural network for recognizing handwritten digits using wavelength multiplexing.

[0048] Figure 9 It is the confusion matrix of the classification accuracy of the test set of fashion items identified by the wavelength-reused anti-interference all-optical diffraction neural network. Detailed Implementation

[0049] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and not for limiting the present invention. Furthermore, it should be noted that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, not all structures. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.

[0050] The terms "comprising" and "having," and any variations thereof, used in this invention are intended to cover non-exclusive inclusion. For example, a process, method, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0051] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] Please see Figure 1As shown, an embodiment of the present invention provides a method for identifying unknown targets in complex environments using an all-optical diffraction neural network. This method includes:

[0053] A wavelength-reused, interference-resistant all-optical diffraction neural network is constructed. This network comprises an input layer, an all-optical diffraction layer, and an output layer. Each all-optical diffraction layer includes several wavelength channels, each performing a different interference-resistant single-target classification task. For details, please refer to [link to relevant documentation]. Figure 2 The definition of a wavelength-multiplexed anti-interference all-optical diffraction neural network is as follows: Each wavelength channel performs a different anti-interference single-target classification task. For example, in a scene containing three objects belonging to different categories (numbers, fashion items, and letters), wavelength channel 1 can identify the subclasses (0-5) of object 1 (number objects), and treat objects 2 (fashion items) and 3 (letters) as "interference objects," neither distinguishing their specific subclasses nor affecting the recognition result of object 1; wavelength channel 2 can identify the subclasses of object 2, and treat objects 1 and 3 as "interference objects"; wavelength channel 3 can identify the subclasses of object 3, and treat objects 1 and 2 as "interference objects." Furthermore, under different wavelength channels, interference objects with arbitrarily changing positions and sizes do not affect the decision result of the target object. Therefore, by integrating the recognition results from the three wavelength channels, the subclasses of all target objects in the scene are obtained; please also refer to [further details omitted]. Figure 3 The wavelength-multiplexed anti-interference all-optical diffraction neural network consists of an input layer, an all-optical diffraction layer, and an output layer. The output layer contains a complex scene of 1+N objects (1 represents one target object, and N represents N interfering objects). The all-optical diffraction layer can contain 100*100 neurons, and each neuron can perform specific complex amplitude modulation on the input light field. The output layer represents the category of the target object, specifically represented by a light spot focused on a specific detection area. Taking six categories as an example, six small square areas with different spatial positions are set up accordingly. If the target object corresponds to the first subcategory, the light spot is focused on the first small square area.

[0054] The anti-interference all-optical diffraction neural network is trained, and the phase coefficients of neurons in the all-optical diffraction layer are determined. This network can transmit information to the predetermined target through multiple independent wavelength channels without crosstalk between them. Therefore, it can be designed as an independent anti-interference network for different input wavelengths. For example, in wavelength channel 1, in a complex scene with 1+N objects, "1" represents the number of target objects, which can be any number from 0 to 9; "N" represents the number of interfering objects from other categories. To ensure the all-optical diffraction neural network accurately outputs the target value in multi-object scenes... During the training phase, an "anti-interference" training mechanism is adopted for the subclasses of the target objects. That is, different light field modulation strategies are designed for the datasets of interference objects and target objects respectively. After the light field of the target object is modulated by the all-optical diffraction neural network, it appears as a light spot concentrated in a specific area in the output plane, corresponding to the category of the target object. After the light field of the interference object is modulated by the all-optical diffraction neural network, it appears as low-density noise evenly distributed throughout the output plane, so as not to affect the classification result of the target object. Similarly, the same principle applies to wavelength channel 2 and wavelength channel 3 to identify the target objects belonging to the corresponding subclasses.

[0055] Based on the phase coefficients of neurons, the structural parameters of the wavelength-reusing metasurface are designed. After the anti-interference all-optical diffraction neural network is trained on a computer, the phase array of neurons in the anti-interference all-optical diffraction neural network is numerically determined. Then, the parameters of the aforementioned neuron phase array are transferred to the optical device, enabling the identification of unknown targets in complex environments using an all-optical device. Considering that the metasurface can independently modulate orthogonal wavelength channels and has the characteristics of small size, low loss, and easy integration, the metasurface is chosen as the optical device to realize the wavelength-reusing anti-interference all-optical diffraction neural network. Taking m wavelength channels as an example, the 0-2π phase response is discretized into N orders. By changing the structural parameters of the metasurface (such as length, width, height, and rotation angle), N is selected. m A variety of metasurface structures with different parameters were used to achieve an Nth-order phase response in the range of 0-2π with m wavelength channels.

[0056] Based on the structural parameters of the wavelength-reusing metasurface, a wavelength-reusing metasurface was fabricated. The fabricated wavelength-reusing metasurface was then verified to determine its target recognition performance. In practical operation, the micro / nano structure array corresponding to the wavelength-reusing metasurface was exported as a GDSII format image file and processed using electron beam lithography. The fabricated wavelength-reusing metasurface was then experimentally verified, demonstrating that the above design can reuse different wavelength channels and achieve the recognition function of objects at different locations in complex environments.

[0057] In another embodiment, please refer to Figure 3 The construction of a wavelength-multiplexed anti-interference all-optical diffraction neural network includes:

[0058] Construct an input layer, a full-optical diffraction layer, and an output layer; among which,

[0059] The input layer is used to binarize multi-target scene images, so that the transmittance of each target area in the multi-target scene image is 1, and the transmittance of other areas is 0.

[0060] The all-optical diffraction layer is an array of diffraction neurons, with each diffraction neuron corresponding to a wavelength channel. The diffraction neurons can perform complex amplitude modulation on the light field from the input layer. The all-optical diffraction layer can be an array of 100*100 diffraction neurons, with each diffraction neuron capable of performing complex amplitude modulation on the light field.

[0061] The output layer is used to output the energy distribution of the optical field after complex amplitude modulation.

[0062] The propagation of light in an anti-interference all-optical diffraction neural network follows the Rayleigh-Sommerfeld diffraction equation. When the distance of diffraction propagation is z, then by the... The diffraction layer reaches the first The light field representation of the +1 diffraction layer is as follows:

[0063]

[0064] in, Indicates the first The diffraction layer is located in The light field, Indicates the first The diffraction layer is located in Modulation of diffractive neurons, This indicates the distance the diffraction propagates. , The spatial frequencies along the x-axis and y-axis are represented. The anti-interference all-optical diffraction neural network includes N diffraction layers. After the above process is repeated N times, a customized complex amplitude distribution of the optical field is formed in the output layer.

[0065] In another embodiment, training an interference-resistant all-optical diffraction neural network includes:

[0066] Set up a training dataset to train an anti-interference all-optical diffraction neural network.

[0067] Set a loss function and optimizer to iteratively process the loss function of the anti-interference all-optical diffraction neural network for each training iteration, thereby completing the training of the anti-interference all-optical diffraction neural network.

[0068] In another embodiment, a loss function and an optimizer are set to iteratively process the loss function of the anti-interference all-optical diffraction neural network for each training iteration, thereby completing the training of the anti-interference all-optical diffraction neural network, including:

[0069] Each time the anti-interference all-optical diffraction neural network is trained, the output result is obtained through forward propagation, and the loss function is set according to the output result and the target result.

[0070] The gradient derivative of the loss function is calculated to obtain the gradient information of the loss function relative to the phase coefficients of each layer of neurons. The backpropagation algorithm is then used to pass all the gradient information back layer by layer. The Adam optimizer is set up to update and adjust the phase coefficients of each layer of neurons with the corresponding learning rate. This process is repeated iteratively to reduce the loss function, thereby completing the training of the anti-interference all-optical diffraction neural network and determining the phase coefficients of the neurons in the all-optical diffraction layer.

[0071] In another embodiment, the method further includes:

[0072] After training the anti-interference all-optical diffraction neural network is completed, each wavelength channel of the anti-interference all-optical diffraction neural network executes different optical field modulation strategies for the corresponding target and interfering objects. The optical field corresponding to the target object, after being modulated by the wavelength channel, appears as a light spot focused in a specific area in the output plane. The optical field corresponding to the interfering object, after being modulated by the wavelength channel, appears as low-density noise uniformly distributed across the entire output plane.

[0073] Please see Figure 4 The training process of the anti-interference all-optical diffraction neural network mainly includes:

[0074] Set up the training dataset. Taking wavelength channel 1 as an example, the MNIST dataset is the target object dataset, which contains 20,000 images and 20,000 corresponding labels (0-5). The interference object dataset contains 20,000 images, each containing N interference objects of arbitrary spatial location and size, from the Fashion-MNIST and EMNIST datasets, with the labels uniformly set to 6. Then, mix and shuffle the target object dataset and the interference object dataset to form the training dataset.

[0075] A loss function is set up to measure the difference between the output of the anti-interference all-optical diffraction neural network and the ideal result. Therefore, different requirements are placed on the output light field of the interfering object and the target object, necessitating the formulation of different loss functions. For the target object, the output light field is characterized by a spot focused at a specific location, with no significant energy focusing at other locations. Therefore, the loss function is defined as the index O of the region with the highest energy. + One-hot encoded G with the target label + The mean squared error (MSE) function and the ratio E of the total energy of the light spot to the total energy of all regions on the output plane are used to form the target object's loss function, Loss, through a weighted average. targetThe calculation formula is as follows:

[0076]

[0077] Among them, MSE (O + G + ) is O + and G + The mean squared error function, where β1 and β2 are weighting coefficients.

[0078] For interference, the output light field exhibits uniform noise distributed across the entire output plane. Therefore, the loss function is defined as the Pearson correlation function (PCC) between the output plane light field and the standard uniform noise. The formula for the Pearson correlation coefficient is as follows:

[0079]

[0080] Where cov(X, Y) is the covariance of sample set X and sample set Y, and σ X and σ X These are the standard deviations of sample set X and sample set Y, respectively. i and Y i Let be the i-th sample in sample set X and sample set Y, respectively. and are the means of sample set X and sample set Y, respectively, and n is the number of samples in sample set X and sample set Y.

[0081] Loss function for interference inter It is expressed as follows:

[0082]

[0083] Where PCC(O,I) is the Pearson correlation coefficient between sample set O and sample set I, PCC(O,O) DET ) are sample set O and sample set O DET Pearson correlation coefficient, PCC(O,O) SFT ) are sample set O and sample set O SFT Pearson correlation coefficient, , and These are the weighting coefficients.

[0084] A training batch contains 8 images, including both the target object and the interference objects. Therefore, the loss function is calculated according to different labels, and the sum of the loss values ​​of the 8 images is used as the basis for subsequent iterative optimization of the adversarial all-optical diffraction neural network. The formula is as follows:

[0085]

[0086] Loss is the sum of the loss values.

[0087] For wavelength channel 1, in multi-target scenarios for handwritten digit recognition, handwritten digits possess rich low-frequency information, making it relatively easy to distinguish different subclasses and resulting in good energy focusing. Therefore, the weighting coefficients can be adjusted accordingly. Set them to 0.8, 0.2, 0.4, 0.4, and 0.2 respectively.

[0088] For wavelength channel 2, in the recognition of fashion items in multi-target scenarios, fashion items are rich in high-frequency information, which is easily lost after binarization, making it difficult to distinguish different subcategories, and the energy is not easily focused on a specific region. Therefore, the weighting coefficients can be adjusted. Set them to 0.5, 0.5, 0.1, 0.4, and 0.5 respectively.

[0089] For wavelength channel 3, in the recognition of handwritten letters in multi-target scenarios, the outlines of handwritten letters are obvious, and distinguishing different subclasses is relatively easy. Therefore, the weighting coefficients can be adjusted. Set them to 0.7, 0.3, 0.4, 0.4, and 0.2 respectively.

[0090] An optimizer is configured so that after each batch of data input, the output result is first calculated through forward propagation, and the loss function is calculated based on the difference between the output and the target result. By taking the gradient derivative of the loss function, the gradient information of the loss function with respect to the phase coefficients of each layer of neurons is obtained. Then, the backpropagation algorithm is used to propagate these gradients back layer by layer, and the Adam optimizer updates and adjusts the phase coefficients of each layer of neurons with a learning rate of 0.01. This process is repeated iteratively, continuously reducing the loss function while updating the parameters in each batch. The entire training process is set to 10 iterations, thereby improving the ability of the adversarial all-optical diffraction neural network to identify a single target in a multi-target environment.

[0091] In another embodiment, the structural parameters of the wavelength-reusing metasurface are designed based on the neuron phase coefficient, including:

[0092] Based on the phase coefficient of the neuron, the transmission phase response of micro- and nanostructures of different sizes at different wavelengths is scanned using rigorous coupled-wave analysis (RCWA) to select micro- and nanostructures that meet the preset anti-rotation and transmittance conditions to form a candidate structure library.

[0093] Select matching micro / nano structures from the candidate structure library, and introduce corresponding geometric phases to the matching micro / nano structures so that the phase response of the matching micro / nano structures can achieve uniform coverage of 0-2π at different wavelengths.

[0094] Based on the structural parameters of the matched micro / nano structures, the structural parameters of the wavelength multiplexing metasurface are designed.

[0095] Please see Figure 5 Taking the design of a metasurface structure with two wavelength channels multiplexed as an example, the specific process includes: using Rigorous Coupled Wave Analysis (RCWA) software to scan the phase response of micro / nano structures (such as nanopillar structures) with different length and width dimensions at two wavelengths; selecting micro / nano structures that meet the conditions of anti-rotation (anti-rotation angle) and transmittance as a candidate structure library; selecting matching micro / nano structures from the candidate structure library; rotating the matching micro / nano structures to introduce the corresponding geometric phase; and finally enabling the phase response of the matching micro / nano structures to achieve uniform coverage of 0-2π at different wavelengths.

[0096] Please continue reading. Figure 4 In the metasurface design process, the metasurface adopts a Si structure with a period of 500 nm and a height of 800 nm, fabricated on a SiO2 substrate with a thickness of 500 μm. To ensure that the metasurface structure meets the design requirements for wavelength multiplexing, a metasurface design method combining propagation phase and geometric phase is proposed. The specific steps are as follows:

[0097] First, with a fixed metasurface structure height, a metasurface phase transmission design is used. The phase transmission relies on the propagation of light within the unit structure to achieve phase accumulation, as shown below:

[0098]

[0099] Where, φ pp Let λ be the propagation phase, λ be the wavelength of the light wave, and n be the wavelength of the light wave. eff Where is the equivalent refractive index, and H is the height of the unit cell; the transmission phase of the metasurface structure is wavelength-dependent.

[0100] Using rigorous coupled-wave analysis (RCWA) software, the length and width of individual micro-nano structures (such as rectangular column structures) were set as variables to calculate the phase modulation effect of all rectangular column structures at m different wavelengths. Micro-nano structures that meet the preset anti-rotation and transmittance conditions were selected to form a candidate structure library.

[0101] Based on the transmission phase, a geometric phase is introduced. The geometric phase is introduced by the rotation angle of the micro / nano structure, and it is represented as follows:

[0102]

[0103] Where J(θ) is the geometric phase, t xx t xy t yx and t yyLet R(θ) represent the transmission coefficients for different wavelength channels, R(θ) be the matrix of rotation angles of the micro / nano structure, and R(-θ) be the inverse of the matrix of rotation angles of the micro / nano structure. It can be seen that the geometric phase of the metasurface structure is independent of wavelength and depends only on the rotation angle. Using the above method, the phase response of the candidate structure library in the m wavelength channels is more uniform in the range of 0-2π. Then, the N structure closest to the discrete phase is selected. m A micro / nano structure was developed to realize the neuronal function of an anti-interference all-optical diffraction neural network.

[0104] Based on the phase coefficients corresponding to neurons at the same location in the same diffraction layer, metasurface structures that can simultaneously satisfy m phases are selected. Each diffraction layer contains 100*100 neurons. To reduce the difficulty of aligning neurons between metasurfaces, neurons are represented as macropixels. A macropixel is 5μm in size and consists of 10*10 identical micro / nano structures. Therefore, the above steps are repeated 10,000 times to form a metasurface array, resulting in a final size of 500μm*500μm for the metasurface array.

[0105] In other words, the optimal structural parameters, such as size and rotation angle, are selected from the structure library. The number of structural arrays is consistent with the number of neurons in the diffraction layer of the all-optical diffraction network, and is used to form a wavelength multiplexing metasurface.

[0106] In another embodiment, a wavelength-multiplexing metasurface is fabricated based on its structural parameters, including:

[0107] Based on the structural parameters of the wavelength multiplexing metasurface, a master mask is prepared;

[0108] Pre-treat the substrate and deposit a hard mask on the substrate, then spin-coat photoresist onto the hard mask;

[0109] Using a master mask, the substrate is exposed to ultraviolet light, developed, and the photoresist in the exposed area is removed to form a pattern consistent with the master mask.

[0110] The substrate is subjected to ion etching and hard mask forming, followed by pattern transfer and residual photoresist removal to obtain a wavelength multiplexed metasurface.

[0111] Please see Figure 6 In practice, the metasurface array is exported as a GDSII format graphic file, and a master mask is prepared using an electron beam lithography system, with the nanostructure error controlled within 0.02 μm.

[0112] In the pretreatment of the substrate and the deposition of the hard mask on the substrate, a 6-inch silicon wafer was used as the substrate, and organic contaminants and metal ions were removed by a standard cleaning process; then, a 300 nm thick SiO2 hard mask layer was generated on the silicon substrate by PECVD.

[0113] In ultraviolet (UV) exposure, UV lithography pattern transfer is used to spin-coat a silicon wafer with photoresist, align the master mask with it, and then perform UV exposure using a lithography machine. After development, the photoresist in the exposed areas is removed, forming a resist pattern consistent with the master mask.

[0114] In the ion etching (reactive ion etching, RIE) operation, photoresist was used as a mask, and an inductively coupled plasma system was used to simultaneously etch the hard mask and the silicon layer. The process gas was a mixture of C4F8, SF6 and O2. The etching parameters were fixed at 80 sccm for C4F8, 50 sccm for SF6, and 5 sccm for O2. The proportion of SF6 was 37%, and the etching time was 10 min.

[0115] In the mask stripping operation, after the pattern is transferred to the silicon substrate using an optimized reactive ion etching process, residual photoresist is removed by ultrasonic cleaning with acetone, the SiO2 hard mask layer is removed by buffered oxide etching solution, and finally the wavelength multiplexing metasurface is fabricated by rinsing with ultrapure water and drying with nitrogen.

[0116] In another embodiment, prior to verifying the prepared wavelength-multiplexing metasurface, the following steps are also included:

[0117] Multiple wavelength-multiplexing metasurfaces are used to assemble optical computing devices;

[0118] A verification optical path is constructed; wherein, the verification optical path includes, in sequence along the direction of light transmission, a light source, a mirror group, a first polarizer, a first quarter-wave plate, a deflecting mirror, a digital micromodulator, a 4f beam-shrinking system (referred to as the 4f system), an optical computing device, a microscope objective, a second quarter-wave plate, a second polarizer, and a near-infrared camera.

[0119] After the metasurface array is fabricated, the processing quality is evaluated. Once the fabrication is confirmed to be error-free, multiple metasurfaces are precisely aligned and integrated. The metasurface spacing is 500 μm, and spacers of similar thickness are used to ensure the interlayer spacing. High-precision alignment between multiple metasurfaces is achieved through precisely designed alignment marks, with an alignment error of no more than 1 μm and a rotation error of no more than 1°. After being bonded with UV-cured adhesive, they are assembled into an optical computing device.

[0120] Please see Figure 7The verification optical path shown in the diagram is constructed as an example of the experimental verification process of a multi-target recognition network implemented with two wavelength channels. The specific process is as follows: A near-infrared light source is used, with a 780nm channel to implement the MNIST dataset six-class classification task and a 1064nm channel to implement the Fashion MNIST dataset six-class classification task. The light emitted from the near-infrared light source is circularly polarized after passing through a first polarizer and a first quarter-wave plate. After amplitude encoding of the input image using a digital micro-modulator (DMD), the light is beam-constricted by a 4f system, diffracted for 500µm, and then enters the optical computing device. An output surface with a size of 500µm is set at a distance of 3.5mm from the optical computing device. A near-infrared camera is used to record the output results. To match the camera's pixel size, an optical microscope objective is used to expand the light field of the output surface. The light then passes through a second quarter-wave plate and a second polarizer to filter out the circularly polarized light that is not accurately modulated by the metasurface, retaining the modulated circularly polarized light, thereby improving the signal-to-noise ratio of the output light field.

[0121] In another embodiment, the prepared wavelength-multiplexing metasurface is verified to determine its target recognition performance, including:

[0122] The image output by the near-infrared camera is obtained using the verification optical path, and the target recognition performance of the wavelength multiplexed metasurface is determined based on the light field intensity distribution in the output image.

[0123] Optionally, determining the target recognition performance of the wavelength multiplexing metasurface includes:

[0124] Images captured by a near-infrared camera are retrieved as the network's output.

[0125] The output image is segmented to obtain the target region and interference region corresponding to the output image. The target region refers to multiple pre-defined square regions, and the target category is determined based on the energy intensity of these regions. The interference region is the portion of the entire imaging area excluding the target region.

[0126] When the input image contains only the target to be identified, the energy intensity of each target region is calculated. The target region with the highest energy intensity is numbered according to the category of the identified target. When the input image contains only interfering objects, the mean and standard deviation of the light field intensity of the target region and the interfering region are calculated. The light field intensity of both the target region and the interfering region is close to 0, and the standard deviation is less than 10. -3 When the output image contains multiple objects, the light field energy of objects belonging to the target class will be concentrated in the target area, while the light field energy of objects belonging to the interference class will be evenly distributed in the interference area.

[0127] To verify the network's ability to effectively suppress interference and achieve accurate target classification, the signal-to-noise ratio and the energy ratio between target regions were used as two metrics to measure network performance.

[0128] The signal-to-noise ratio (SNR) is obtained using the following formula:

[0129]

[0130] Where P1 represents the energy intensity of the target region with the highest energy. S 目标 P represents the area of ​​the target region. 图像 P represents the total energy of the image captured by the near-infrared camera. 目标总 S represents the total energy within all target regions. 干扰 This indicates the area of ​​the interference region. The SNR should be no less than 20 to ensure that the network can accurately distinguish the target object category without being affected by the light field energy of the interfering object.

[0131] The target recognition confidence score ΔE, which is the energy ratio between target regions, is obtained by the following formula.

[0132]

[0133] Where P1 represents the energy intensity of the target region with the highest energy, and P2 represents the energy intensity of the target region with the second highest energy. ΔE is not greater than 1 to ensure that the network can correctly distinguish the correct category.

[0134] By collaboratively calculating target recognition confidence and signal-to-noise ratio, this approach overcomes the limitations of single-index evaluation. It accurately quantifies the clarity of target recognition while fully considering the impact of complex environmental interference on recognition, achieving multi-dimensional coverage of the "target recognition capability + anti-interference capability" of wavelength-multiplexed metasurfaces. This scheme can objectively and repeatably determine whether the target recognition performance of wavelength-multiplexed metasurfaces meets the standards, providing clear quantitative basis for iterative processes such as metasurface structure optimization and neuron phase coefficient adjustment. This ensures the reliability and stability of its recognition of unknown targets in complex environments, promoting the application effect of all-optical diffraction neural networks in practical scenarios (such as complex lighting and multi-interference source environments).

[0135] Please see Figures 8-9The values ​​shown are the classification accuracy confusion matrices for handwritten digit recognition test sets and fashion item test sets using a wavelength-multiplexed anti-interference all-optical diffraction neural network, respectively. It can be observed that different anti-interference target classification tasks are performed under different wavelength channels, enabling multi-target recognition. The aforementioned wavelength-multiplexed multi-target recognition diffraction neural network has been validated in the near-infrared and terahertz bands. Therefore, it can be seen that by scaling the diffraction features proportionally according to the wavelength range of interest and designing a wavelength-multiplexed metasurface structure, multi-target recognition tasks within any wavelength range can be achieved. This effectively solves the current limitation of diffraction neural network applications, making it suitable for fields such as image recognition and signal processing, and possessing significant application prospects and importance.

[0136] In summary, this method for identifying unknown targets in complex environments using an all-optical diffraction neural network employs an all-optical diffraction neural network composed of multi-layered phase-type diffraction optical elements. During propagation, optical image recognition is achieved by reconstructing the input image multiple times from the wavefront. Based on the diffraction neural network, an anti-interference recognition mechanism is proposed. During the training phase of the diffraction neural network, the concepts of target objects and interference objects to be specifically classified are proposed, and loss functions and constraints are set separately. An error backpropagation algorithm is used to optimize the output to match the expected value. Furthermore, combined with a wavelength multiplexing mechanism, different target object classification tasks are processed independently under different wavelength channels. Thus, by integrating the classification results of all wavelength channels across the entire wavelength range, the classification task of unknown targets in complex environments can be achieved. This leverages the advantages of high speed and low energy consumption of the all-optical diffraction neural network, overcoming the limitations of optical neural networks such as simple application scenarios and small number of targets.

[0137] The above is only one specific embodiment of the present invention, and any improvements made based on the concept of the present invention shall be considered within the scope of protection of the present invention.

Claims

1. A method for identifying unknown targets in complex environments using an all-optical diffraction neural network, characterized in that, include: An anti-interference all-optical diffraction neural network with wavelength multiplexing is constructed; wherein, the anti-interference all-optical diffraction network includes an input layer, an all-optical diffraction layer, and an output layer; the all-optical diffraction layer includes several wavelength channels, each wavelength channel performing a different anti-interference single-target classification task; Train the anti-interference all-optical diffraction neural network and determine the neuron phase coefficients of the all-optical diffraction layer; Based on the phase coefficients of the neurons, design the structural parameters of the wavelength multiplexing metasurface; Based on the structural parameters of the wavelength multiplexing metasurface, a wavelength multiplexing metasurface is fabricated; the fabricated wavelength multiplexing metasurface is then verified to determine its target recognition performance. Based on the phase coefficients of the neurons, the structural parameters of the wavelength multiplexing metasurface are designed, including: Based on the phase coefficient of the neuron, the transmission phase response of micro-nano structures of different sizes at different wavelengths is scanned using rigorous coupled-wave analysis, thereby selecting micro-nano structures that meet the preset anti-rotation and transmittance conditions to form a candidate structure library. Select matching micro / nano structures from the candidate structure library, and introduce corresponding geometric phases to the matching micro / nano structures so that the phase response of the matching micro / nano structures can achieve uniform coverage of 0-2π at different wavelengths. Based on the structural parameters of the matched micro / nano structures, the structural parameters of the wavelength multiplexing metasurface are designed.

2. The method for identifying unknown targets in complex environments using an all-optical diffraction neural network as described in claim 1, characterized in that: Building a wavelength-multiplexed, interference-resistant all-optical diffraction neural network includes: Construct an input layer, a full-optical diffraction layer, and an output layer; among which, The input layer is used to perform binarization processing on the multi-target scene image, so that the transmittance of the region where each target is located in the multi-target scene image is 1, and the transmittance of other regions is 0. The all-optical diffraction layer is an array composed of several diffraction neurons, each corresponding to a wavelength channel; the diffraction neurons are capable of performing complex amplitude modulation on the light field from the input layer. The output layer is used to output the energy distribution of the light field modulated by the complex amplitude.

3. The method for identifying unknown targets in complex environments using an all-optical diffraction neural network as described in claim 2, characterized in that: Training the anti-interference all-optical diffraction neural network includes: Set up a training dataset to train the anti-interference all-optical diffraction neural network; A loss function and optimizer are set up to iterate the loss function of the anti-interference all-optical diffraction neural network in reverse for each training iteration, thereby completing the training of the anti-interference all-optical diffraction neural network.

4. The method for identifying unknown targets in complex environments using an all-optical diffraction neural network as described in claim 3, characterized in that: Setting a loss function and optimizer, and then iteratively processing the loss function for each training iteration of the anti-interference all-optical diffraction neural network to complete the training of the anti-interference all-optical diffraction neural network, including: Each time the anti-interference all-optical diffraction neural network is trained, the output result is obtained through forward propagation, and a loss function is set based on the output result and the target result; The gradient derivative of the loss function is calculated to obtain the gradient information of the loss function relative to the phase coefficients of each layer of neurons. The backpropagation algorithm is then used to pass all the gradient information back layer by layer. The Adam optimizer is set to update and adjust the phase coefficients of each layer of neurons with the corresponding learning rate. This process is repeated iteratively to reduce the loss function, thereby completing the training of the anti-interference all-optical diffraction neural network and determining the phase coefficients of the neurons in the all-optical diffraction layer.

5. The method for identifying unknown targets in complex environments using an all-optical diffraction neural network as described in claim 4, characterized in that: The method further includes: After the anti-interference all-optical diffraction neural network is trained, each wavelength channel of the anti-interference all-optical diffraction neural network executes different optical field modulation strategies for the corresponding target and interfering objects. The optical field corresponding to the target object, after being modulated by the wavelength channel, appears as a light spot focused in a specific area in the output plane. The optical field corresponding to the interfering object, after being modulated by the wavelength channel, appears as low-density noise uniformly distributed across the entire output plane.

6. The method for identifying unknown targets in complex environments using an all-optical diffraction neural network as described in claim 5, characterized in that: Based on the structural parameters of the wavelength-multiplexing metasurface, a wavelength-multiplexing metasurface is prepared, including: Based on the structural parameters of the wavelength multiplexing metasurface, a master mask is prepared; The substrate is pretreated and a hard mask is deposited on the substrate, and photoresist is spin-coated onto the hard mask; Using the master mask, the substrate is subjected to ultraviolet exposure, development, and removal of photoresist in the exposed areas to form a pattern consistent with the master mask. The substrate is subjected to ion etching and hard mask forming, followed by pattern transfer and residual photoresist removal to obtain a wavelength-reusable metasurface.

7. The method for identifying unknown targets in complex environments using an all-optical diffraction neural network as described in claim 6, characterized in that: Before verifying the prepared wavelength-multiplexed metasurface, the method further includes: Multiple wavelength-multiplexing metasurfaces are used to assemble optical computing devices; A verification optical path is constructed; wherein, the verification optical path includes, in sequence along the light transmission direction, a light source, a mirror group, a first polarizer, a first quarter-wave plate, a deflecting mirror, a digital micro-modulator, a 4f beam-shrinking system, the optical computing device, a microscope objective, a second quarter-wave plate, a second polarizer, and a near-infrared camera.

8. The method for identifying unknown targets in complex environments using an all-optical diffraction neural network as described in claim 7, characterized in that: The prepared wavelength-multiplexing metasurface is verified to determine its target recognition performance, including: The image output by the near-infrared camera is acquired using the verification optical path, and the target recognition performance of the wavelength multiplexing metasurface is determined based on the light field intensity distribution within the output image.

Images