Super surface-based plug-in diffraction neural network optimization method and task identification device

Abstract

The application discloses a plug-in and plug-out diffractive neural network optimization method and a task recognition device based on a metasurface, and belongs to the technical field of optical neural networks, micro-nano optics and image recognition applications. The plug-in and plug-out diffractive neural network is composed of an input layer, a modulation layer and an output layer, and the modulation layer is divided into a shared layer and a plug-in and plug-out layer. Gradient descent algorithm and back propagation algorithm are combined with a transfer learning algorithm to optimize the modulation layer parameters of the plug-in and plug-out diffractive neural network. According to the phase distribution of the optimized modulation layer of the plug-in and plug-out diffractive neural network, a metasurface nano pillar structure is selected, and the metasurface is prepared. A plug-in and plug-out diffractive neural network multi-task recognition device based on the metasurface is manufactured, and the multi-task recognition device comprises a mask plate, a detector and a metasurface used for constructing the plug-in and plug-out diffractive neural network. By switching the plug-in and plug-out components realized by the metasurface in the network, various recognition tasks can be switched, the flexibility of the plug-in and plug-out diffractive neural network is improved, and the consumption of computing resources and the training time are reduced.

Description

Technical Field

[0001] This invention relates to an optimization method for a metasurface-based pluggable diffraction neural network, mainly involving the training method of the pluggable diffraction neural network and the construction method of the optical system, belonging to the fields of optical neural networks, micro-nano optics, and image recognition application technology. Background Technology

[0002] The rapid development of deep learning has led to ever-increasing computational demands, which have rendered existing computing architectures inadequate and resulted in significant energy consumption. Optical neural networks (ONNs), with their advantages of high computational speed, high parallelism, and low energy consumption, have been proven to be an effective architecture for accelerating AI model computations. Replacing traditional electronic neural networks with ONNs can greatly improve computational speed and reduce energy consumption, making them a promising new computing framework. Traditional ONN architectures typically utilize optical devices such as lenses and spatial light modulators, resulting in large and complex optical systems. While these networks exhibit the advantages of ONNs, the size of the optical systems makes them difficult to integrate into a single system. Therefore, achieving highly integrable ONNs is a key area of ​​research.

[0003] Metasurfaces are novel two-dimensional structures that offer a promising platform for ultrathin planar optics compared to traditional diffractive optical elements. By altering the size, arrangement, and shape of the metasurface unit cells, the amplitude and phase of light can be controlled simultaneously. Numerous studies have already explored metasurface-based optical neural networks, primarily applied to tasks such as differential and cross-correlation mathematical operations, solving differential equations, image edge extraction, and image recognition. Due to the ultrathin and lightweight nature of metasurfaces, using them to implement optical neural networks facilitates the integration of optical systems.

[0004] Diffractive neural networks (DNs) are a popular architecture in optical neural networks in recent years. They simulate neurons in electronic neural networks using optical unit structures, enabling massive parallel data processing and further exploring the potential of optical neural networks. Combining DNs with metasurfaces can achieve high-performance computing while maintaining the miniaturization and integration of optical systems. However, most DNs are currently implemented using passive devices, including metasurfaces, making dynamic tunability difficult and unable to meet the need for multiple functions within a single device in practical applications. While some work has addressed the limitation of single-function DNs by using programmable electromagnetic metasurfaces or optoelectronic fusion computing architectures, these methods introduce additional energy consumption and increase the complexity of experimental setups. Summary of the Invention

[0005] To address the issues of dynamic control limitations and limited functionality in diffraction neural networks, one of the main objectives of this invention is to provide an optimization method for pluggable diffraction neural networks based on metasurfaces. The pluggable diffraction neural network consists of three parts: an input layer, a modulation layer, and an output layer. The modulation layer is divided into a shared layer and a pluggable layer. The modulation layer parameters are optimized using gradient descent and backpropagation algorithms combined with transfer learning. Based on the phase distribution of the optimized modulation layer, a metasurface nanopillar structure is selected and fabricated. By switching the pluggable components implemented with metasurfaces in the network, various recognition tasks can be switched, improving the flexibility of the pluggable diffraction neural network while reducing computational resource consumption and training time.

[0006] The second main objective of this invention is to provide a multi-task recognition device based on a metasurface-based pluggable diffraction neural network. The modulation layer phase distribution is obtained according to the optimization method of the metasurface-based pluggable diffraction neural network. The corresponding phase modulation nanopillar structure is selected according to the phase distribution, and a multilayer metasurface is prepared. Then, the multi-task recognition device based on the metasurface-based pluggable diffraction neural network is fabricated, realizing the miniaturization and multifunctionality of the diffraction neural network.

[0007] The objective of this invention is achieved through the following technical solution.

[0008] This invention discloses an optimization method for a metasurface-based pluggable diffraction neural network. The pluggable diffraction neural network consists of three parts: an input layer, a modulation layer, and an output layer. The modulation layer is divided into a shared layer and a pluggable layer. The pluggable layer of the pluggable diffraction neural network is composed of pluggable layer plugs for different classification tasks. Switching between pluggable layer plugs enables switching between various recognition tasks. The modulation layer parameters of the pluggable diffraction neural network are trained using gradient descent and backpropagation algorithms combined with transfer learning algorithms. The modulation layer parameters of the pluggable diffraction neural network are trained using a recognition task dataset to determine the number of shared and pluggable layers and obtain the phase distribution of the modulation layer of the pluggable diffraction neural network. The geometric dimensions of the nanopillar units are determined according to the transmittance requirements. Rectangular dielectric nanopillars of the same size but different azimuth angles are selected to generate corresponding metasurface structure processing files and fabricate the metasurface. When polarized light carrying object information through amplitude encoding is incident on the metasurface, the incident information can be modulated, and the energy of the outgoing beam can be focused on the corresponding detection plane sub-region. The category represented by the region with the highest energy is selected as the recognition result based on its energy distribution. Switching between pluggable layers enables the classification of various objects, improving the flexibility of network design and increasing the tunability of diffractive neural networks.

[0009] The present invention discloses a metasurface-based pluggable diffraction neural network optimization method, which achieves switching between multiple recognition tasks by switching pluggable layer plugs, including the following steps:

[0010] Step 1: The plug-and-play diffraction neural network based on the metasurface consists of three parts: an input layer, a modulation layer, and an output layer. The input layer uses a mask template to encode the information of the object image to be recognized, encodes the image information into the beam amplitude channel, and inputs the information into the modulation layer; the modulation layer consists of multiple phase distributions and is physically implemented by multiple metasurfaces, where: the number of modulation layers is set to N, the first N - M layers are shared layers, and the last M (M < N) layers are plug-and-play layers. The plug-and-play layer is composed of plug-and-play layer plugins for implementing different classification tasks. The output layer is a pre-set detection plane. If the number of recognized objects is C, then C equal-sized regions need to be selected on the detection plane as sub-detection regions. By dividing the modulation layer into a shared layer and a plug-and-play layer, multiple recognition tasks can be switched by switching the plug-and-play layer plugins. The number of modulation layers N and the number of plug-and-play layers M are determined by the performance of the plug-and-play diffraction neural network in the recognition task.

[0011] Step 2: Construct a propagation model based on the plug-and-play diffraction neural network described in Step 1. The propagation model is divided into a forward propagation module and a backward propagation module. The forward propagation module of the plug-and-play diffraction neural network is constructed according to the Rayleigh-Sommerfeld diffraction theory. Through the forward propagation module, the propagation of input information to the detection plane is realized, which is used to detect the recognition accuracy of the plug-and-play diffraction neural network. The backward propagation module uses the gradient descent algorithm and the backpropagation algorithm to optimize the phase parameters of the modulation layer according to the gradient of the loss function, and then obtains the phase distribution of the modulation layer. Suppose there are a total of Y datasets. First, select one from the Y datasets and train the plug-and-play diffraction neural network for the first time. Determine the number of modulation layers N required in Step 1 according to the phase distribution of the trained plug-and-play diffraction neural network; then, in combination with the transfer learning algorithm, select another dataset to continue training the phase parameters of the plug-and-play layer of the diffraction neural network, and fix the phase parameters of the shared layer. Determine the number of plug-and-play layers M according to the newly trained phase distribution of the plug-and-play layer. Finally, determine that the number of modulation layers is N, the number of plug-and-play layers is M, and determine the phase distribution of the plug-and-play diffraction neural network.

[0012] The specific implementation method of Step 2 includes the following steps:

[0013] Step 2.1: Implement the forward propagation module of the plug-and-play diffraction neural network according to the Rayleigh-Sommerfeld diffraction theory. Its propagation form is expressed as:

[0014] U(r l+1 )=t(r l )∫∫ S U(r l )·h(r l+1 -r l )dxdy (1)

[0015] Formula (1) shows the change in the optical field distribution from the l-th layer to the l + 1-th layer. Let a represent the transmittance of the l-th layer, where a and These represent amplitude and phase, respectively. When training only a pure phase-type plug-in diffraction neural network, a = 1 is set.

[0016] In formula (1), h(r) l+1 -r l The impulse response during the transmission process is represented as:

[0017]

[0018] in λ is the incident wavelength, which can be used in the design of visible light, near-infrared and microwave bands as needed.

[0019] When l = N+1, it represents the detection plane. Based on the energy distribution of the sub-detection regions of the detection plane, the corresponding energy distribution of the sub-detection regions can be expressed as s. i =|U i | 2 , where i represents the detector number, and the category corresponding to the region with the highest energy is selected as the output recognition result.

[0020] Step 2.2: In the first training iteration, select a dataset as the training dataset, such as the MNIST dataset, and use it as the training set to train the phase parameters of the modulation layer of the pluggable diffraction neural network. The training objective is to maximize the energy distribution of the corresponding category's sub-detector region while minimizing the energy outside the sub-detector region. During the optimization process, the mean squared error loss function E is used to evaluate the difference between the energy distribution of different sub-detector regions and the target energy distribution.

[0021]

[0022] Where g c This represents the target energy distribution, and C is the number of objects identified.

[0023] The phase parameters of the modulation layer are optimized by calculating the gradient of the loss function and using stochastic gradient descent and backpropagation algorithms. This optimization is repeated multiple times until the desired result is achieved. Training stops when convergence fails and accuracy cannot be improved. Repeat the above steps, adjusting the number of modulation layers N to find the optimal number of modulation layers for achieving the designer's desired object recognition accuracy. The number of modulation layers N is determined based on the phase distribution of the trained plug-in diffraction neural network.

[0024] Step 2.3: In the second training, select another dataset as the training dataset, such as the Fashion-MNIST dataset. Use the transfer learning algorithm to fix the parameters of the first NM layers of the network as shared layers, and continue to optimize the phase parameters of the plugged layers (the last M layers) using the Fashion-MNIST dataset. By adjusting the number of plugged layers M, find the number of modulation layers that can achieve the object recognition accuracy required by the designer. Determine the number of plugged layers M based on the phase distribution of the plugged layers in the newly trained plugged diffraction neural network. Determine the number of modulation layers as N, the number of plugged layers as M, and determine that the phase distribution of the plugged diffraction neural network includes NM phase distribution shared layers and 2×M plugged layer phase distributions.

[0025] Step 2.4: Select other datasets from the Y datasets and repeat step 2.3. Finally, the phase distribution of the plug-in diffraction neural network containing NM phase distribution shared layers and Y×M plug-in layer phase distributions can be obtained.

[0026] The training simulation software described in step two uses the Python-based TensorFlow algorithm framework.

[0027] Step 3: Based on the phase distribution of the modulation layer determined in Step 2, select the metasurface nanopillars required for the phase distribution of the modulation layer of the pluggable diffraction neural network. Based on the phase arrangement of the selected metasurface nanopillars determined in Step 2, fabricate the modulation layer, and construct the pluggable diffraction neural network based on the metasurface according to Step 1.

[0028] To reduce the fabrication difficulty of metasurfaces, a preferred approach is to use a multilayer metasurface composed of rectangular dielectric nanopillar arrays of the same size but different azimuth angles to realize a pluggable diffraction neural network. The Berry phase principle is used as the design principle for the metasurface; Berry phase modulation enables dispersion-free azimuth-angle full-phase control, where circularly polarized light is converted into its opposite spiral and carries a 2θ phase delay. The metasurface unit structure parameters include the length L, width W, and height H of the nanopillars, as well as the unit period length P.

[0029] When scanning the metasurface unit structure, the height H of the nanopillars and the unit period length P are fixed, and the electromagnetic response of each nanostructure can be calculated using electromagnetic full-wave simulation software. By performing a two-dimensional scan of the nanopillar length L and width W, the transmission coefficients trr(tll) and trl(tlr) of nanopillars of different sizes under polarized light incident light are obtained for co-polarized light and cross-circularly polarized light. During the simulation, by reasonably selecting the incident wavelength λ, nanopillar material, and nanopillar substrate material, nanopillar structures with a transmission coefficient of cross-circularly polarized light as close to 1 as possible can be selected, thereby ensuring the highest energy utilization efficiency of the entire metasurface. Then, according to the Berry phase principle, the nanopillar structure is rotated to complete the phase modulation corresponding to the phase distribution of the modulation layer. The nanopillar structure is arranged according to the phase distribution of the modulation layer of the insertion and extraction diffraction neural network trained in step two, generating the corresponding metasurface processing file and fabricating a multilayer metasurface. To ensure that the dimensions of the unit structure within the metasurface are consistent with the design, as a preferred method, standard electron beam etching technology is used to fabricate the multilayer metasurface.

[0030] The simulation software described in step three uses RCWA, which is based on the rigorous coupled-wave analysis method, and FDTD, which is based on the finite-difference time-domain method.

[0031] It also includes step four: based on step three, implement a metasurface-based pluggable diffraction neural network. When switching pluggable layer plugs, it can achieve classification tasks of various objects, improve the flexibility of pluggable diffraction neural network design, and increase the tunability of diffraction neural network.

[0032] This invention discloses a metasurface-based insertion-and-extraction diffraction neural network multi-task recognition device, comprising a mask, a detector, and a metasurface for constructing the insertion-and-extraction diffraction neural network. The metasurface for constructing the insertion-and-extraction diffraction neural network is prepared by obtaining the modulation layer phase distribution according to a metasurface-based insertion-and-extraction diffraction neural network optimization method, selecting corresponding phase-modulated nanopillar structures based on the phase distribution, and fabricating a multilayer metasurface, thereby obtaining the metasurface-based insertion-and-extraction diffraction neural network multi-task recognition device.

[0033] In autonomous driving systems, the metasurface-based pluggable diffraction neural network multi-task recognition device, combined with a CMOS imaging chip, can achieve real-time road obstacle detection by switching pluggable layers, and realize a miniaturized integrated device. In medical image recognition systems, the metasurface-based pluggable diffraction neural network multi-task recognition device can switch the corresponding pluggable diffraction neural network according to the type of input medical image to achieve medical image recognition, assisting in basic medical research and clinical experiments.

[0034] Beneficial effects:

[0035] 1. This invention discloses an optimization method and task recognition device for a pluggable diffraction neural network based on metasurfaces. The pluggable diffraction neural network consists of three parts: an input layer, a modulation layer, and an output layer. The modulation layer is divided into a shared layer and a pluggable layer. The modulation layer parameters of the pluggable diffraction neural network are optimized using gradient descent, backpropagation, and transfer learning algorithms. A metasurface nanopillar structure is selected based on the phase distribution of the optimized modulation layer of the pluggable diffraction neural network, and the metasurface is fabricated. By switching the pluggable components realized by the metasurface in the network, multiple recognition tasks can be switched, improving the flexibility of network design, while reducing computational resource consumption and training time, thus realizing a multifunctional diffraction neural network and a multi-task recognition device.

[0036] 2. The metasurface-based pluggable diffraction neural network optimization method and task recognition device disclosed in this invention use gradient descent algorithm, backpropagation algorithm, and transfer learning algorithm to optimize the phase parameters of the modulation layer of the pluggable diffraction neural network according to the gradient of the loss function, thereby obtaining the phase distribution of the modulation layer, determining the number of modulation layers and the number of pluggable layers, completing the phase distribution optimization of the modulation layer of the metasurface-based pluggable diffraction neural network, improving the flexibility of network design, and increasing the tunability of the diffraction neural network.

[0037] 3. The pluggable diffraction neural network optimization method and task recognition device based on metasurface disclosed in this invention determine the processing file of the metasurface nanopillar structure based on the phase distribution of the modulation layer of the optimized pluggable diffraction neural network, and prepare multi-layer metasurfaces to obtain a multi-task recognition device based on metasurface pluggable diffraction neural network, thereby realizing the miniaturization and multi-functionality of diffraction neural network.

[0038] 4. The metasurface-based pluggable diffraction neural network optimization method and task recognition device disclosed in this invention are applicable to the visible light, near-infrared and microwave bands. By selecting a reasonable nanopillar structure and designing and optimizing the geometry of the nanopillar, pluggable diffraction neural networks for each band can be realized. Attached Figure Description

[0039] Figure 1 This is a flowchart of the metasurface-based insertion and extraction diffraction neural network optimization method;

[0040] Figure 2 This is a schematic diagram of the propagation model of the metasurface-based insertion-extraction diffraction neural network optimization method;

[0041] Figure 3 This is an optimization flowchart based on transfer learning for the plug-in diffraction neural network in an embodiment of the present invention;

[0042] Figure 4 These are the two-dimensional scanning results of the transmission coefficients of nanopillars of different sizes and the phase modulation relationships corresponding to different rotations in the embodiments of the present invention;

[0043] Where: (a) - Schematic diagram of the rectangular nanocolumn structure, (b) - Amplitude scan results of the transmission coefficients trr (tll) and trl (tlr), (c) - Schematic diagram of the rotation angle of the rectangular nanocolumn, (d) - Relationship between the rotation angle and the additional phase;

[0044] Figure 5 is the optical path diagram used in the experiment of the embodiment of the present invention;

[0045] Where: 1 - Linear polarizer LP1, 2 - Quarter-wave plate QWP1, 3 - Digital micromirror device (DMD), 4, 5 - Lenses L1, L2, 6 - Cascade metasurface, 7 - Microscope objective, 8 - Quarter-wave plate QWP2, 9 - Transmission linear polarizer LP2, 10 - CCD;

[0046] Figure 6 are the simulation and experimental effects of digital data recognition in the embodiment of the present invention.

[0047] Figure 7 are the simulation and experimental effects of fashion item data recognition in the embodiment of the present invention. Specific Embodiment

[0048] The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments. At the same time, the technical problems and beneficial effects solved by the technical solution of the present invention are also described. It should be noted that the described embodiments are only for facilitating the understanding of the present invention and do not impose any limitation on it.

[0049] As Figure 1 shown, the optimized method for plug-and-play diffraction neural network based on metasurface disclosed in this embodiment is specifically implemented as follows:

[0050] Step 1: The plug-and-play diffraction neural network constructed based on metasurface consists of an input layer, a modulation layer, and an output layer. The input layer uses a mask template to implement the information encoding of the object image to be recognized, encodes the image information into the beam amplitude channel, and inputs the information into the modulation layer; the modulation layer consists of multiple phase distributions and is physically implemented by multiple metasurfaces. Where: the number of modulation layers is set to N, the first N - M layers are shared layers, and the last M (M < N) layers are plug-and-play layers. The plug-and-play layer is composed of plug-and-play layer plugins for implementing different classification tasks. The output layer is a pre-set detection plane. If the number of recognized objects is 6, then 6 equal-sized regions need to be selected on the detection plane as sub-detection regions. By dividing the modulation layer into a shared layer and a plug-and-play layer, multiple recognition tasks can be switched by switching the plug-and-play layer plugins. The number of modulation layers N and the number of plug-and-play layers M are determined by the performance of the plug-and-play diffraction neural network in the recognition task.

[0051] Figure 2 is the schematic diagram of the propagation model of the optimized method for plug-and-play diffraction neural network based on metasurface

[0052] The input layer uses a mask to implement the information encoding of the object image to be recognized, encodes the image information into the beam amplitude channel, and realizes the image encoding of the digital "3" to be classified and the fashion item "bag". Preset N layers of metasurfaces as the modulation layer, and select the first N - M layers as the shared layer, and the last M (M < N) layers as the pluggable layer. Use red metasurfaces to represent the shared layer plug-ins, and blue and green to represent the two types of pluggable layer plug-ins. The first N - M layers of the shared layer are used to preprocess the input information, and the last M (M < N) layers of the pluggable layer select specific plug-ins for specific tasks. The output layer is a preset detection plane. The detection plane of the handwritten digits is divided into 6 discrete regions distributed horizontally to represent the digits "0" - "5", and the detection plane of fashion is divided into 6 discrete regions distributed vertically to represent 6 fashion items (T-shirts, trousers, coats, sports shoes, bags, and ankle boots). When a plane wave is incident, amplitude modulation is performed through the mask to obtain an equal-phase input image "3". After passing through the modulation layer, the diffracted light is focused on the sub-region corresponding to the digit "3" in the detection area, realizing the handwritten digit classification task. Replace the pluggable layer plug-ins to achieve fashion item classification. The image of the "bag" is focused on the corresponding sub-region of the "bag" in the detection area after passing through the modulation layer to achieve the fashion item classification task

[0053] Figure 3 It is the optimization flowchart based on transfer learning of the pluggable diffraction neural network in the embodiment of the present invention. The specific implementation method of step two includes the following steps:

[0054] Step 2.1: Implement the forward propagation module of the pluggable diffraction neural network according to the Rayleigh - Sommerfeld diffraction theory, obtain the energy distribution of the detection plane. According to the energy distribution of the sub-detection regions of the detection plane, the energy distribution corresponding to the sub-detection region can be expressed as s i =|U i | 2 , where i represents the detector serial number, and select the category corresponding to the region with the maximum energy as the output recognition result.

[0055] Step 2.2: In the first training, select the MNIST dataset as the training set to train the phase parameters of the modulation layer of the pluggable diffraction neural network. The training objective is to maximize the energy distribution of the sub-detection regions corresponding to the categories, and at the same time minimize the energy outside the sub-detection regions. In the optimization process, use the mean square error loss function E to evaluate the difference between the energy distributions of different sub-detector regions and the target energy distribution. Use the random gradient descent and error backpropagation algorithms by calculating the gradient of the loss function to optimize the phase parameters of the modulation layer. Stop training after multiple iterations until convergence and the accuracy rate cannot be improved. Determine the number of modulation layers as 2 according to the trained phase distribution of the pluggable diffraction neural network.

[0056] Step 2.3: In the second training, the Fashion-MNIST dataset is selected. Using a transfer learning algorithm, the parameters of the first layer of the network are fixed as a shared layer. The phase parameters of the second insertion / extraction layer are further optimized using the Fashion-MNIST dataset. Based on the phase distribution of the newly trained insertion / extraction diffraction neural network, the number of insertion / extraction layers is determined to be 1. The number of modulation layers is determined to be 2, and the number of insertion / extraction layers is determined to be 1. The phase distribution of the insertion / extraction diffraction neural network is determined to include one phase distribution shared layer and two insertion / extraction layer phase distributions.

[0057] Step 3: Based on the phase distribution of the modulation layer determined in Step 2, select metasurface nanopillars with the required phase distribution for realizing the modulation layer of the pluggable diffraction neural network. Using the selected metasurface nanopillars with the phase arrangement determined in Step 2, fabricate the modulation layer and construct the metasurface-based pluggable diffraction neural network according to Step 1. Preferably, a multilayer metasurface composed of rectangular dielectric nanopillar arrays of the same size but different azimuth angles is used to realize the pluggable diffraction neural network. Using the Berry phase principle as the design principle of the metasurface, the Berry phase modulation method can achieve dispersion-free azimuth angle full-phase control. When circularly polarized light is incident, it is converted into its opposite spiral and carries a 2θ phase delay. The metasurface unit structure parameters include the length L, width W, height H of the nanopillars, and the unit period length P.

[0058] Figure 4 This presents the two-dimensional scanning results of the transmission coefficients of nanopillars of different sizes and the phase modulation relationships corresponding to different rotations in the embodiments of this invention. Rigorous Coupled Wave Analysis (RCWA) simulation was used to design and optimize the metasurface nanopillars, and the electromagnetic response of the periodic array of the nanostructure was calculated. The nanopillar material was amorphous silicon with a fixed height H of 600 nm and a unit period length P of 500 nm. The incident wavelength was set to 800 nm. By scanning the length L and width W of the nanopillars in 5 nm increments from 70 nm to 300 nm, the transmission coefficients trr(tll) and trl(tlr) of co-polarized and cross-polarized light were obtained. During the simulation, the refractive indices of the amorphous silicon nanopillars and the fused silica substrate were set to... and n sub =1.5. Nanopillars with a length L of 210 nm and a width W of 135 nm are selected to make the metasurface transmittance close to 1. The angle between the long axis of the nanopillar and the x-axis of the substrate is φ. When the incident light is set to circularly polarized light, the phase modulation value obtained by the transmitted cross-circularly polarized light can cover the range from 0 to 2π.

[0059] Next, based on the Berry phase principle, the rotating nanopillar structure completes phase modulation corresponding to the phase distribution of the modulation layer. Following the training in step two, the nanopillar structure is arranged according to the phase distribution of the modulation layer of the insertion-extraction diffraction neural network, generating the corresponding metasurface fabrication file. To ensure that the dimensions of the fabricated unit structures within the metasurface match the design, standard electron beam etching technology is used to prepare the multilayer metasurface.

[0060] The process also includes step four: implementing a metasurface-based pluggable diffraction neural network based on step three. This allows for the classification of various objects by switching pluggable layers, improving the flexibility of the pluggable diffraction neural network design and increasing its tunability.

[0061] Figure 5 This is the optical path diagram used in the experiment of this invention embodiment. Circularly polarized light is generated using polarizer LP1 and a quarter-wave plate, and incident on a digital micromirror device (DMD) to obtain a beam carrying target information. To avoid diffraction of the input beam during transmission, two lenses are used to form a 4f system, imaging the coded target image 4 mm in front of the metasurface. The image of the probe plane is acquired and magnified using a microscope objective, and the polarization state is filtered using a second set of quarter-wave plates and polarizer LP2. The CCD is moved to the probe plane to capture the image.

[0062] Figure 6 , 7 This is a simulation and experimental effect of identification from digital and fashion product data in the embodiments of the present invention.

[0063] In the digit classification simulation, the handwritten digit classification component achieved a simulation accuracy of 91.8% after 10 rounds of iterative training. Using 6000 handwritten digit images as the test dataset, and randomly selecting 300 images for experimental verification, the results, represented by a confusion matrix, showed an experimental accuracy of 91.3%. The experimental results agree well with the simulation results, indicating the effectiveness of the design theory. The handwritten digit images were encoded into the amplitude channel as input, and the light accurately focused onto the preset sub-detection region on the probe plane. Based on the energy distribution obtained from the simulation and experiments, it is proven that the system can correctly recognize handwritten digits.

[0064] The pluggable layer plugin was replaced with a fashion recognition plugin, which achieved 90.2% accuracy after five training epochs. A test dataset of 6000 fashion images was used, with 300 images randomly selected for experimental verification. The results, represented by the confusion matrix, showed an accuracy of 90%. The pluggable diffraction neural network successfully focused the light beam onto a specific sub-detection region to identify fashion items. In experimental testing, the metasurface-based pluggable diffraction neural network successfully identified both digital and fashion item data as designed, by switching the pluggable layer plugin.

[0065] This invention discloses a metasurface-based insertion-and-extraction diffraction neural network multi-task recognition device, comprising a mask, a detector, and a metasurface for constructing the insertion-and-extraction diffraction neural network. The metasurface for constructing the insertion-and-extraction diffraction neural network is prepared by obtaining the modulation layer phase distribution according to a metasurface-based insertion-and-extraction diffraction neural network optimization method, selecting corresponding phase-modulated nanopillar structures based on the phase distribution, and fabricating a multilayer metasurface, thereby obtaining the metasurface-based insertion-and-extraction diffraction neural network multi-task recognition device.

[0066] In autonomous driving systems, the metasurface-based pluggable diffraction neural network multi-task recognition device, combined with a CMOS imaging chip, can achieve real-time road obstacle detection by switching pluggable layers, and realize a miniaturized integrated device. In medical image recognition systems, the metasurface-based pluggable diffraction neural network multi-task recognition device can switch the corresponding pluggable diffraction neural network according to the type of input medical image to achieve medical image recognition, assisting in basic medical research and clinical experiments.

[0067] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A metasurface-based insertion-extraction diffraction neural network optimization method, characterized in that: The switching of multiple recognition tasks is achieved by switching the plug-and-play layer plug-ins, including the following steps: Step 1: The plug-and-play diffraction neural network constructed based on the metasurface consists of three parts: an input layer, a modulation layer, and an output layer; the input layer uses a mask template to encode the information of the object image to be recognized, encodes the image information into the beam amplitude channel, and inputs the information into the modulation layer; the modulation layer consists of multiple phase distributions and is physically implemented by multiple metasurfaces, where: the number of modulation layers is set to N, the first N-M layers are shared layers, and the last M layers are plug-and-play layers, M < N, and the plug-and-play layer consists of plug-and-play layer plug-ins for implementing different classification tasks; the output layer is a pre-set detection plane, and if the number of recognized objects is C, then C equal-sized regions need to be selected on the detection plane as sub-detection regions; by dividing the modulation layer into a shared layer and a plug-and-play layer, the switching of multiple recognition tasks is achieved by switching the plug-and-play layer plug-ins; the number of modulation layers N and the number of plug-and-play layers M are determined by the performance of the plug-and-play diffraction neural network in the recognition task; Step 2: A propagation model is constructed based on the plug-and-play diffraction neural network described in Step 1, and the propagation model is divided into a forward propagation module and a backward propagation module; the forward propagation module of the plug-and-play diffraction neural network is constructed according to the Rayleigh-Sommerfeld diffraction theory, and the forward propagation module is used to realize the propagation of the input information to the detection plane for detecting the recognition accuracy of the plug-and-play diffraction neural network; the backward propagation module uses the gradient descent algorithm and the backpropagation algorithm to optimize the phase parameters of the modulation layer according to the gradient of the loss function, and then obtain the phase distribution of the modulation layer; suppose there are a total of Y datasets, first select one from the Y datasets, train the plug-and-play diffraction neural network for the first time, and determine the number of modulation layers N required in Step 1 according to the phase distribution of the trained plug-and-play diffraction neural network; then combine the transfer learning algorithm to select another dataset to continue training the phase parameters of the plug-and-play layer of the diffraction neural network, and fix the phase parameters of the shared layer, and determine the number of plug-and-play layers M according to the newly trained phase distribution of the plug-and-play layer; finally, determine that the number of modulation layers is N, the number of plug-and-play layers is M, and determine the phase distribution of the plug-and-play diffraction neural network; Step 3: According to the phase distribution of the modulation layer determined in Step 2, select the metasurface nanocolumns for realizing the phase distribution required by the modulation layer of the plug-and-play diffraction neural network, arrange the selected metasurface nanocolumns according to the phase arrangement determined in Step 2, fabricate the modulation layer, and construct the plug-and-play diffraction neural network based on the metasurface according to Step 1.

2. The metasurface-based insertion and extraction diffraction neural network optimization method as described in claim 1, characterized in that: The specific implementation method of Step 2 includes the following steps: Step 2.1: Implement the forward propagation module of the plug-and-play diffraction neural network according to the Rayleigh-Sommerfeld diffraction theory, and its propagation form is represented as: U(r l+1 )=t(r l )∫∫ S U(r l )·h(r l+1 -r l )dxdy (1) Equation (1) illustrates the change in the light field distribution propagating from layer l to layer l+1. Let a represent the transmittance of the l-th layer, where a and These represent amplitude and phase, respectively. When training only a pure phase-type plug-in diffraction neural network, we set a = 1. In formula (1), h(r) l+1 -r l The impulse response during the transmission process is represented as: in λ is the incident wavelength; When l = N+1, it represents the detection plane. Based on the energy distribution of the sub-detection regions of the detection plane, the corresponding energy distribution of the sub-detection regions can be expressed as s. i =|U i | 2 , i represents the detector number, and the category corresponding to the region with the highest energy is selected as the output recognition result; Step 2.2: Select one dataset as the training dataset in the first training, and use it as the training set to train the phase parameters of the modulation layer of the plug-and-play diffraction neural network. The training goal is to maximize the energy distribution in the sub-detection regions corresponding to the categories while minimizing the energy outside the sub-detection regions; during the optimization process, the mean square error loss function E is used to evaluate the difference between the energy distributions in different sub-detector regions and the target energy distribution: Where g c This represents the target energy distribution, and C is the number of objects identified. The phase parameters of the modulation layer are optimized by calculating the gradient of the loss function and using stochastic gradient descent and backpropagation algorithms. This optimization is repeated multiple times until the desired result is achieved. Training is stopped when convergence occurs and accuracy cannot be improved. The above steps are repeated to find the number of modulation layers N that can achieve the object recognition accuracy required by the designer by adjusting the number of modulation layers N. The number of modulation layers N is determined according to the phase distribution of the trained plug-in diffraction neural network. Step 2.3: In the second training, another dataset is selected as the training dataset. The parameters of the first NM layers of the network are fixed as shared layers using the transfer learning algorithm. The phase parameters of the plug-in layer are further optimized using the Fashion-MNIST dataset. By adjusting the number of plug-in layers M, the number of modulation layers that can achieve the object recognition accuracy required by the designer is found. The number of plug-in layers M is determined according to the phase distribution of the plug-in diffraction neural network. The number of modulation layers is determined to be N, the number of insertion layers is determined to be M, and the phase distribution of the insertion diffraction neural network is determined to include NM phase distribution sharing layers and 2×M insertion layer phase distributions. Step 2.4: Select other datasets from the Y datasets and repeat step 2.

3. Finally, the phase distribution of the plug-in diffraction neural network containing NM phase distribution shared layers and Y×M plug-in layer phase distributions can be obtained.

3. The metasurface-based insertion and extraction diffraction neural network optimization method as described in claim 2, characterized in that: A pluggable diffraction neural network is realized using a multilayer metasurface composed of rectangular dielectric nanopillar arrays of the same size but different azimuth angles. The Berry phase principle is used as the design principle of the metasurface. The Berry phase modulation method can realize dispersionless azimuth angle full phase control. When circularly polarized light is incident, it will be converted into its opposite spiral and carry a 2θ phase delay. The metasurface unit structure parameters include the length L, width W, height H of the nanopillars and the unit period length P. When scanning the metasurface unit structure, the height H of the nanopillars and the period length P of the unit are fixed. The electromagnetic response of each nanostructure can be calculated using electromagnetic full-wave simulation software. By performing a two-dimensional scan of the length L and width W of the nanopillars, the transmission coefficients trr(tll) and trl(tlr) of nanopillars of different sizes under polarized light incident light are obtained for co-polarized light and cross-circularly polarized light. During the simulation, by reasonably selecting the incident wavelength λ, the nanopillar material, and the nanopillar substrate material, a nanopillar structure with a transmission coefficient of cross-circularly polarized light as close to 1 as possible can be selected, thereby ensuring the highest energy utilization efficiency of the entire metasurface. Then, according to the Berry phase principle, the nanopillar structure is rotated to complete the phase modulation corresponding to the phase distribution of the modulation layer. The nanopillar structure is arranged according to the phase distribution of the modulation layer of the insertion and extraction diffraction neural network trained in step two, generating the corresponding metasurface processing file and fabricating a multilayer metasurface.

4. The metasurface-based insertion and extraction diffraction neural network optimization method as described in claim 1, 2, or 3, characterized in that: It also includes step four, which implements a metasurface-based pluggable diffraction neural network based on step three. When switching pluggable layer plugs, it can perform classification tasks for various objects, improve the flexibility of the pluggable diffraction neural network design, and increase the tunability of the diffraction neural network.

5. The metasurface-based insertion and extraction diffraction neural network optimization method as described in claim 4, characterized in that: Multilayer metasurfaces are fabricated using standard electron beam etching techniques.

6. A metasurface-based insertion-extraction diffraction neural network multi-task recognition device, implemented based on the metasurface-based insertion-extraction diffraction neural network optimization method as described in claim 1, 2, or 3, characterized in that: The device includes a mask, a detector, and a metasurface for constructing a pluggable diffraction neural network. The metasurface for constructing the pluggable diffraction neural network is obtained by using a metasurface-based pluggable diffraction neural network optimization method to obtain the modulation layer phase distribution, selecting the corresponding phase modulation nanopillar structure according to the phase distribution, and preparing a multilayer metasurface to obtain a metasurface-based pluggable diffraction neural network multi-task recognition device.

7. The multi-task recognition device based on a metasurface insertion-extraction diffraction neural network as described in claim 6, characterized in that: In autonomous driving systems, the metasurface-based pluggable diffraction neural network multi-task recognition device, when combined with a CMOS imaging chip, can achieve real-time road obstacle detection by switching pluggable layers, and can also realize a miniaturized integrated device.

8. The multi-task recognition device based on a metasurface-based insertion-extraction diffraction neural network as described in claim 6, characterized in that: In a medical image recognition system, the metasurface-based pluggable diffraction neural network multi-task recognition device can switch the corresponding pluggable diffraction neural network according to the type of the input medical image to achieve medical image recognition, thus assisting in basic medical research and clinical experiments.

Images