Design method and device of metasurface optical neural network

Abstract

The invention discloses a design method and device of a metasurface optical neural network, and the method comprises the steps: taking a geometrical optical hyper-parameter, a metasurface parameter and a neural network parameter as samples at the same time, fitting a corresponding relation between the parameters and a judgment parameter based on the prior of a probabilistic agent model, and carrying out the design of the metasurface optical neural network. New to-be-observed geometrical optical hyper-parameters are determined through an acquisition function, iteration is continuously carried out before retrieval is not finished, the geometrical optical hyper-parameters are gradually made to approach to values meeting requirements, and metasurface parameters and neural network parameters can also be iterated into reasonable values; finally, the three parameters of the geometrical optical hyper-parameter, the metasurface parameter and the neural network parameter of the target are obtained at the same time, the effects of high design efficiency and good classification precision of the metasurface optical neural network are achieved through a soft and hard integrated design method, and the design requirement is better met.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and optoelectronic material design, and in particular to a design method and device for a supersurface optical neural network. Background Art

[0002] The metasurface optical neural network includes an optical system and a deep neural network. It can use light as an information carrier and two-dimensional metasurface materials as light controllers. The metasurface optical neural network has the characteristics of high bandwidth, high connectivity and parallel processing, and its computing speed is faster than that of artificial neural networks.

[0003] In the process of forming the present invention, the inventors found that the design of the supersurface optical neural network has the following technical problems:

[0004] On the one hand, it is difficult to determine the appropriate parameters of the various hardware parameters of the optical system to match other parameters, making it difficult to ensure imaging quality and classification accuracy; on the other hand, the number of adjustable units on the metasurface of the optical system is in the tens of millions or even billions, and the design of the metasurface relies on a large amount of numerical simulation calculations and the experience of designers, which is time-consuming and inefficient; on the other hand, it is also uncertain whether the design of the hardware parameters can meet the use requirements of deep neural networks. If it cannot meet the requirements, the hardware parameters need to be further adjusted according to the feedback of the deep neural network.

[0005] Therefore, the existing metasurface optical neural networks have low design efficiency and low precision, and these problems restrict the development of metasurface optical neural networks. Summary of the invention

[0006] In order to solve at least one of the above-mentioned problems in the prior art, the purpose of the present invention is to provide a method and device for designing a metasurface optical neural network in which multiple parameters can be collaboratively designed, with high design efficiency and good precision.

[0007] To achieve the above-mentioned purpose of the invention, an embodiment of the present invention provides a design method of a metasurface optical neural network, wherein the metasurface optical neural network includes an optical system and a deep neural network, wherein the optical system includes a metasurface, and the design method includes the following steps:

[0008] Step S10: Acquire a sample set, wherein the sample set includes parameter samples, each group of parameters in the parameter samples includes geometric optics hyperparameters, hypersurface parameters and neural network parameters, and judgment parameters corresponding to the sample;

[0009] Step S20: using a probabilistic proxy model to fit the independent variables of the geometric optical hyperparameters, the hypersurface parameters and the neural network parameters, and the dependent variable being the approximate distribution function of the judgment parameter;

[0010] Step S30: using an acquisition function to determine the geometrical optical hyperparameters to be observed for the approximate distribution function;

[0011] Step S40: Determine whether the termination condition is reached;

[0012] If the termination condition is not reached, the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed are calculated, the parameter samples are updated, and steps S20 to S40 are looped;

[0013] If the termination condition is reached, the corresponding geometric optical hyperparameters, hypersurface parameters and neural network parameters are determined as target parameters of the hypersurface optical neural network according to the judgment parameters;

[0014] Step S50: Designing the hypersurface optical neural network according to the target primitive distribution.

[0015] As a further improvement of the present invention, the optical system further comprises an aperture and an imaging surface, and the geometric optical hyperparameter comprises at least one of a distance parameter between the target and the aperture, a distance parameter between the aperture and the hypersurface, and a distance parameter between the hypersurface and the imaging surface;

[0016] The hypersurface includes a plurality of primitives, and the hypersurface parameters include size parameters of the primitives;

[0017] The neural network parameters include weight parameters of the deep neural network;

[0018] The judgment parameter is the accuracy and / or loss value generated by the deep neural network.

[0019] As a further improvement of the present invention, the method for generating parameter samples comprises the following steps:

[0020] A set of geometric optical hyperparameters is selected, and the deep neural network is trained using a gradient optimization method based on the set of geometric optical hyperparameters, and the obtained gradient optimal values ​​are determined as the hypersurface parameters and neural network parameters corresponding to the geometric optical hyperparameters, as well as the judgment parameters.

[0021] As a further improvement of the present invention, the probabilistic proxy model uses a Gaussian process;

[0022] The acquisition function is a maximized expected improvement function, a probability increment function, an expected increment function, a confidence upper bound function, or an information entropy function;

[0023] The process of step S30 includes:

[0024] Determine the promotion probability corresponding to each group of independent variables according to the normal distribution;

[0025] The geometrical optics hyperparameter among the independent variables with the greatest improvement probability is determined as the observed geometrical optics hyperparameter.

[0026] As a further improvement of the present invention, the sample set further includes image samples, and the image samples include a plurality of training images and true values ​​corresponding to the training images, and the step S40 includes:

[0027] Based on the geometric optical hyperparameters to be observed, the deep neural network is trained according to the image samples, and the deep neural network is trained using a gradient optimization method, and the obtained gradient optimal values ​​are determined as the hypersurface parameters and neural network parameters corresponding to the geometric optical hyperparameters, as well as the judgment parameters.

[0028] As a further improvement of the present invention, the step of calculating the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed includes:

[0029] Calculate the classification result of the training image obtained by the hypersurface optical neural network, the calculation formula of the classification result y is y=f(g(φ,x),θ), wherein φ is the hypersurface parameter, x is the feature of the training image, g function is the encoding of the training image by the hypersurface, θ is the neural network parameter, and f function is the interaction between the deep neural network and the image encoded by the hypersurface;

[0030] The difference between the classification result and the true value corresponding to the training image is calculated to obtain the judgment parameter.

[0031] As a further improvement of the present invention, the termination condition includes that the acquisition times of the acquisition function reaches the maximum acquisition times, or the judgment parameter meets the target condition.

[0032] As a further improvement of the present invention, the image in the image sample is a finger vein image, and the hypersurface optical neural network is used to identify the finger vein image.

[0033] As a further improvement of the present invention, based on the finger vein image recognition task, the loss function of the deep neural network is: loss = α*L1+β*L2, wherein L1 is the cosine boundary softmax loss function, L2 is the triplet loss function, and α and β are weighted coefficients for balancing the cosine boundary softmax loss function and the triplet loss function, respectively.

[0034] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a design device for a metasurface optical neural network, wherein the metasurface optical neural network includes an optical system and a deep neural network, wherein the optical system includes a metasurface, and the design device includes:

[0035] An acquisition module, used for acquiring a sample set, wherein the sample set includes parameter samples, each group of parameters in the parameter samples includes geometric optics hyperparameters, hypersurface parameters and neural network parameters, and a judgment parameter corresponding to the sample;

[0036] A priori module, used for fitting an approximate distribution function of the judgment parameter using a probabilistic proxy model, in which the independent variables are the geometric optical hyperparameters, the hypersurface parameters and the neural network parameters, and the dependent variable is the judgment parameter;

[0037] An acquisition module, used for determining a geometrical optical hyperparameter to be observed using an acquisition function on the approximate distribution function;

[0038] A judgment module is used to judge whether the termination condition is reached;

[0039] If the termination condition is not reached, the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed are calculated, the parameter samples are updated, and the contents in the a priori module, the a priori module, and the judgment module are circulated;

[0040] If the termination condition is reached, the corresponding geometric optical hyperparameters, hypersurface parameters and neural network parameters are determined as target parameters of the hypersurface optical neural network according to the judgment parameters;

[0041] A design module is used to design the supersurface optical neural network according to the target primitive distribution.

[0042] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides an electronic device, including:

[0043] A storage module storing a computer program;

[0044] The processing module can implement the steps in the above-mentioned method for designing a supersurface optical neural network when executing the computer program.

[0045] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by a processing module, can implement the steps in the above-mentioned method for designing a metasurface optical neural network.

[0046] Compared with the prior art, the present invention has the following beneficial effects: during the design process, the method and device for designing the metasurface optical neural network use the three parameters of geometric optical hyperparameters, hypersurface parameters and neural network parameters as samples at the same time, fit the corresponding relationship between these parameters and the judgment parameters based on the prior of the probabilistic proxy model, and then find a balance between the unexplored area and the area with the potential to approach the target parameter through the acquisition function, determine the new geometric optical hyperparameter to be observed, and continuously iterate before the retrieval is completed to gradually make the geometric optical hyperparameter approach the value that meets the requirements. On this basis, the hypersurface parameters and the neural network parameters can also be iterated to reasonable values, thereby obtaining the three parameters of the target geometric optical hyperparameters, hypersurface parameters and neural network parameters at the same time, and obtaining reasonable values ​​for each parameter with relatively small restrictions. The design method of integrating hardware and software achieves the effects of high design efficiency and good classification accuracy of the metasurface optical neural network, thereby better meeting the design requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic diagram of a supersurface optical neural network according to an embodiment of the present invention;

[0048] Figure 2 is a schematic structural diagram of a super surface according to an embodiment of the present invention;

[0049] Figure 3 is a flow chart of a method for designing a supersurface optical neural network according to an embodiment of the present invention;

[0050] Figure 4 It is a module schematic diagram of a design device for a supersurface optical neural network according to an embodiment of the present invention;

[0051] Among them, 100, metasurface optical neural network; 10, optical system; 11, aperture; 12, metasurface; 121, substrate; 122, primitive; 13, imaging surface; 20, deep neural network; 21, convolution layer; 22, pooling layer; 23, fully connected layer; 30, classification result. DETAILED DESCRIPTION

[0052] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0053] An embodiment of the present invention provides a method and device for designing a metasurface optical neural network with multiple parameters capable of collaborative design, high design efficiency and good precision.

[0054] Supersurface Optical Neural Network

[0055] The supersurface optical neural network 100 of this embodiment includes an optical system 10 and a deep neural network 20, such as Figure 1 As shown, the optical system 10 includes an aperture 11, a metasurface 12 and an imaging surface 13. The aperture 11 is used to limit the size of the imaging beam or the imaging space range, and limit the width of the beam passing through the optical system 10; the metasurface 12 is an artificial medium with a sub-wavelength structure, which modulates the optical signal through the nano-scale primitives 122, or controls the propagation direction of light at different positions; the imaging surface 13 can be a CMOS (Complementary Metal Oxide Semiconductor), which is used to complete the photoelectric conversion and record the distribution of light intensity. After the target passes through the optical neural network 100, the image processed by the metasurface 12 is obtained, and then the features are extracted by the deep neural network 20 for further identification, and the classification result 30 is obtained.

[0056] The metasurface 12 may be composed of a single lens or a plurality of lens groups, such as Figure 2 As shown, the metasurface 12 includes a substrate 121 and a primitive 122. The size and shape of each primitive 122 can be designed separately, and the metasurface parameters include the shape and period of the primitive, as well as the size parameters and rotation angle parameters corresponding to the shape. Through the design method and device of the metasurface optical neural network 100 of this embodiment, the parameters of each primitive 122 can be designed, and then the process parameters are generated based on these parameters.

[0057] The shape of the primitive 122 can be linear, cylindrical, cuboid, elliptical cylinder, hollow elliptical body, hollow cuboid, etc. Different shapes can correspond to some of the same and some of different size parameters and rotation angle parameters, such as the same period parameter and height parameter, and different parameters such as the radius of the cylinder, the constant and width of the cuboid, the major diameter and minor diameter of the elliptical cylinder, etc. Different primitives 122 can have different rotation angle parameters. The parameters of these primitives 122 together constitute the geometric parameters of the hypersurface 12.

[0058] A deep neural network 20 can be Figure 1As shown, it includes a convolution layer 21, a pooling layer 22 and a fully connected layer 23, and may also include some convolution blocks. Specifically, the convolution layer 21 can use a 7x7 filter and a 2x2 step size for initial convolution processing, and then add batch normalization and ReLU activation function. The pooling layer 22 can be a maximum pooling layer with a 3x3 filter and a 2x2 step size. The core part consists of 4 sequences of residual blocks, each of which contains two blocks. The characteristics of these residual blocks are that the input of the block is directly added to the output through "skip connection", thereby optimizing gradient propagation. The first residual block of each sequence uses a step size of 2 for downsampling in the first convolution operation to reduce the width and height of the feature map, and reduce the number of parameters and calculation amount of subsequent network layers. The fully connected layer 23 is at the end of the network, further reducing the feature map to a single vector, and connecting a fully connected layer 23 to complete the classification task. The deep neural network 20 not only enhances the representation ability of the model, but also effectively avoids the gradient disappearance problem in the deep network, thereby achieving efficient and stable model training and inference.

[0059] The metasurface optical neural network 100 of this embodiment can be used for image classification tasks, such as face recognition, smart cabin detection, finger vein recognition, etc. Finger vein recognition is used as an example for explanation below.

[0060] Finger vein recognition performs identity authentication by reading the vein distribution inside the finger, and has strong anti-counterfeiting capabilities. Finger vein recognition is usually based on near-infrared light technology. When near-infrared light penetrates the finger, the hemoglobin in the blood absorbs the light, making the vein information appear in the image, that is, obtaining the venous blood vessel image inside the finger. Existing finger vein recognition requires comparing the image with a standard template to give a verification result, which is difficult to detect and costly. This embodiment uses the super-surface optical neural network 100 for recognition, and the recognition cost is low and the accuracy is high.

[0061] When the design method of the metasurface optical neural network 100 described below is applied to the field of finger vein recognition, the entire process from the acquisition of the finger vein image to the recognition can be simulated. The simulation content includes the simulation of the geometric dimensions of the optical system 10, the simulation of the propagation of light in the near field and the far field, and the simulation of the operation of the deep convolutional neural network after the optical signal is converted into an electrical signal. Finally, the design process is realized through the method of hardware and software integration. In the design process, the Bayesian optimization algorithm and the deep neural network model are combined to realize the design process with smaller constraints and more optimized search strategies for each parameter, which ultimately reduces the design cost and improves the detection accuracy.

[0062] For the finger vein recognition task, the loss function of the deep neural network 20 of this embodiment is adaptively designed, for example, the cosine margin softmax loss function and the triplet loss function are combined at the same time. It has been verified that the loss function designed in this way can show different capabilities in intra-class compactness and inter-class separability, which can not only preserve the overall category structure of the samples, ensure that similar samples are close in the feature space, and dissimilar samples are separated, but also make the training process more stable. Specifically, the specific formula of the loss function loss is:

[0063] loss = α*L1+β*L2,

[0064] Among them, L1 is the cosine boundary softmax loss function, L2 is the triple loss function, α and β are the weighted coefficients for balancing the cosine boundary softmax loss function and the triple loss function. More specifically:

[0065]

[0066] Among them, cosθj is the equivalent form of the inner product between the normalized features and the weights, θj is the angle between the feature vector xi and the weight vector Wj, and s and m are the scaling hyperparameter and margin hyperparameter, respectively.

[0067]

[0068] in, is a set of triplets, Is a collection The cardinality of , γ is a marginal hyperparameter, and [·] = max(0, ·) represents the rectification function. A triplet consists of an anchor sample, a positive sample from the same class as the anchor sample, and a negative sample from a different class from the anchor sample. The features of the anchor sample, positive sample, and negative sample are denoted as xa, xp, and xn, respectively.

[0069] In addition, the optimizer of the deep neural network 20 selects the SGD optimizer, which takes the parameters of the neural network, the learning rate and the weight decay coefficient as input. The false recognition rate, the false rejection rate and the equal error rate indicators are used to evaluate the model performance.

[0070] Design method of optical neural network on super surface

[0071] The following is a design method for the metasurface optical neural network mentioned above. The design parameters of the metasurface optical neural network include three parts, namely: geometric optical hyperparameters, hypersurface parameters and neural network parameters. The design of geometric optical hyperparameters is mainly completed through Bayesian optimization, and the design of hypersurface parameters and neural network parameters is completed through the training of the neural network nested in Bayesian optimization. The whole training process includes three aspects: sample set generation, model training, and determination of design parameters according to the model.

[0072] Combine the following Figure 3 , illustrating a method for designing a supersurface optical neural network provided by an embodiment of the present invention. Although the present application provides method operation steps as shown in the following implementation manner or flow chart, based on conventional or no creative labor, the method does not logically have steps with necessary causal relationships, and the execution order of these steps is not limited to the execution order provided in the implementation manner of the present application.

[0073] Generation of sample sets

[0074] The sample set includes parameter samples and image samples, wherein each group of parameters in the parameter samples includes geometric optics hyperparameters, hypersurface parameters and neural network parameters, as well as judgment parameters corresponding to the sample; the image samples include multiple training images, and true values ​​corresponding to the training images.

[0075] The method for generating parameter samples includes the following steps:

[0076] A set of geometric optical hyperparameters is selected, and the deep neural network is trained using a gradient optimization method based on the set of geometric optical hyperparameters, and the obtained gradient optimal values ​​are determined as the hypersurface parameters and neural network parameters corresponding to the geometric optical hyperparameters, as well as the judgment parameters.

[0077] Among them, the geometric optical hyperparameters include at least one of the distance parameters between the target and the aperture, the distance parameters between the aperture and the hypersurface, and the distance parameters between the hypersurface and the imaging surface; the hypersurface parameters include the size parameters of the primitives; the neural network parameters include the weight parameters of the deep neural network; the judgment parameters are the accuracy acc and / or loss value loss generated by the deep neural network.

[0078] The value of the geometric optics hyperparameter can be randomly generated, and based on the randomly generated geometric optics hyperparameter, the hypersurface parameters and the neural network parameters are trained until the parameters with the optimal gradient value are obtained, that is, the hypersurface parameters and the neural network parameters corresponding to the geometric optics hyperparameter at this time, and the judgment parameters. The specific gradient optimization and training process can be described below. By generating multiple sets of geometric optics hyperparameters and training the corresponding other parameters, multiple parameter samples can be generated, for example, 50 sets of parameter samples are generated.

[0079] The method for generating image samples includes the following steps:

[0080] Import the original dataset of image data, and preprocess it based on the original dataset to generate more images to meet the requirements of model training and improve the generalization ability of the model. Specifically, preprocessing can include random cropping, random scaling, random rotation, random perspective transformation, random distortion and inter-class data enhancement, etc., to generate image samples with more data variants.

[0081] Taking the above-mentioned hypersurface optical neural network for finger vein recognition as an example, the image samples can be finger vein images, and these finger vein images are respectively matched with corresponding finger vein classification labels.

[0082] Model training

[0083] Before model training, the training principle of the model is explained:

[0084] The model simulates the entire process from the training image through the optical system to the deep neural network recognition, so all the parameters in this process are simulated, so the relationship between the geometric optical hyperparameters, hypersurface parameters and the neural network parameters and the judgment parameters is used as the objective function. Since the analytical expression of the specific relationship of the objective function cannot be given, the objective function is also a black box function. The purpose of model training is to determine the specific parameters of the hypersurface optical neural network used to classify the above image samples based on the parameter samples above.

[0085] This embodiment nests the training of the neural network on the basis of the Bayesian optimization model to meet the requirements of generating specific parameters of the hypersurface optical neural network. During the model training process, the number of initial parameter samples n and the number of model iterations m can be used as hyperparameters and pre-set according to requirements.

[0086] The design method of the hypersurface optical neural network of this embodiment is as follows:

[0087] Step S10: Obtain a sample set.

[0088] This includes the above parameter samples and image samples.

[0089] Step S20: Use a probabilistic proxy model to fit the independent variables ({x}, φ, θ) to the geometric optical hyperparameters {x}, the hypersurface parameters φ and the neural network parameters θ, and the dependent variable is the approximate distribution function of the judgment parameters.

[0090] Among them, the x in the geometric optics hyperparameter {x} may include the distance parameter between the target and the aperture, the distance parameter between the aperture and the hypersurface, the distance parameter between the hypersurface and the imaging surface, etc.; the hypersurface parameter φ is the size parameter of the primitive, such as the height, size, shape, etc. of the primitive. When all primitives are cylindrical primitives of equal height, the hypersurface parameter φ is the radius parameter of the primitive; the neural network parameter θ is the weight parameter of the deep neural network.

[0091] The probabilistic proxy model can use the Gaussian process to fit the distribution of the target function based on the correspondence between some initial independent variables and dependent variables, and provide an estimate of the uncertainty of the dependent variable corresponding to each independent variable. During the fitting process, the probabilistic proxy model assumes that each point in the continuous input space is associated with a normally distributed random variable. Each finite set of these random variables has a multivariate normal distribution, that is, any finite linear combination of them is a normal distribution, and each predicted value obeys the normal distribution. Finally, the posterior probability distribution corresponding to all independent variables ({x}, φ, θ) is obtained.

[0092] Step S30: using an acquisition function to determine the geometrical optical hyperparameters to be observed for the approximate distribution function.

[0093] The acquisition function is used to determine the next observation point, that is, the next set of independent variables. The acquisition function can be a function that maximizes the expected improvement, a probability increment function, an expected increment function, a confidence upper bound function, or an information entropy function. In the following, the acquisition function is described by taking the expected increment function (EI) as an example.

[0094] Step S31: determining the lift probability corresponding to each group of independent variables according to the normal distribution;

[0095] Step S32: Determine the geometrical optics hyperparameter among the independent variables with the largest probability of improvement as the observed geometrical optics hyperparameter.

[0096] The calculation formula of the acquisition function EI is: Among them, f min is the minimum value observed so far, μ(x) and σ(x) are the predicted mean and standard deviation at point x, respectively, Φ(Z) is the cumulative distribution function of the normal distribution, φ(Z) is the probability density function of the normal distribution, Z = (f min -μ(x)) / σ(x).

[0097] The above steps can be used to determine the next set of independent variables to be observed. Only the geometric optical hyperparameters among the independent variables to be observed are taken as the geometric optical hyperparameters to be observed. The hypersurface parameters and neural network parameters are further calculated in subsequent steps based on the geometric optical hyperparameters to be observed.

[0098] Step S40: Determine whether the termination condition is reached;

[0099] Step S41: If the termination condition is not reached, the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed are calculated, the parameter samples are updated, and steps S20 to S40 are looped.

[0100] Specifically, based on the geometric optical hyperparameters to be observed, the deep neural network is trained according to the image samples, and the deep neural network is trained using a gradient optimization method, and the obtained gradient optimal values ​​are determined as the hypersurface parameters and neural network parameters corresponding to the geometric optical hyperparameters, as well as the judgment parameters.

[0101] The step of calculating the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed includes:

[0102] Calculate the classification result of the training image obtained by the hypersurface optical neural network, the calculation formula of the classification result y is y=f(g(φ,x),θ), wherein φ is the hypersurface parameter, x is the feature of the training image, g function is the encoding of the training image by the hypersurface, θ is the neural network parameter, and f function is the interaction between the deep neural network and the image encoded by the hypersurface;

[0103] The difference between the classification result and the true value corresponding to the training image is calculated to obtain the judgment parameter.

[0104] Here, the feature of the training image corresponding to x is the intensity value of the pixel array of each channel of the training image. The hypersurface is equivalent to an optical convolution layer, which can perform a convolution operation on the input image. The g function is used to calculate the convolved feature image obtained by convolving the training image with the hypersurface. Here, the hypersurface parameter φ and the neural network parameter θ are trained simultaneously during the training process of the training image. By comparing the difference between the classification result and the true value corresponding to the training image, the accuracy acc and the loss value loss can be calculated respectively, where the loss value loss can be calculated based on the loss function mentioned above.

[0105] In the process of calculating the hypersurface parameters and neural network parameters for the observed geometric optical hyperparameters, p calculations can be performed. If the number of iterations of the Bayesian optimization model is m, and the neural network is trained p times in each iteration, then when the Bayesian optimization is finally completed, the neural network has been trained m*p times.

[0106] After the first iteration of the model, the number of updated parameter samples is n+1, and when the next loop runs to step S30, it is used to determine the n+2th group of geometrical optical hyperparameters to be observed.

[0107] Step S42: if the termination condition is reached, the corresponding geometric optical hyperparameters, hypersurface parameters and neural network parameters are determined as target parameters of the hypersurface optical neural network according to the judgment parameters;

[0108] The termination condition includes that the current sampling number reaches the maximum sampling number, or the approximate distribution function meets the convergence condition, wherein the current sampling number is the number of times the step S40 is executed. The maximum sampling number is the number of hyperparameter model iterations m above, which is determined according to computing resources. The convergence condition can be that the accuracy acc is higher than the minimum accuracy value, and / or the loss value loss is lower than the maximum loss value. When the maximum sampling number is reached, the parameters of a set of independent variables with the highest accuracy acc can be determined as the target parameters.

[0109] Determine design parameters based on the model

[0110] Step S50: Designing the hypersurface optical neural network according to the target primitive distribution.

[0111] After the above training is completed, a photomask data format file (GDS, Graphical Data System) can be generated according to the obtained metasurface parameters. This is a file format used to describe and record the metasurface layout, including information such as the physical layout and characteristic parameters of the metasurface. This serves as the basis and guidance for subsequent process manufacturing. Then, the corresponding optical neural network hardware structure can be produced, and the process department can create a metasurface lens that matches the metasurface parameters.

[0112] Subsequently, the appropriate lens group can be further selected according to the geometric optical hyperparameters, the spatial position of the CMOS sensor can be set, etc., so that the imaging target, resolution, field of view, and spectral range meet the imaging requirements of the metasurface, and the weight of the deep neural network can be determined according to the neural network parameters, which can then be used to complete image classification and recognition tasks.

[0113] Compared with the prior art, this embodiment has the following beneficial effects:

[0114] (1) In the design process of the design method and device of the metasurface optical neural network, the geometric optical hyperparameters, metasurface parameters and neural network parameters are taken as samples at the same time. Based on the prior of the probabilistic proxy model, the corresponding relationship between these parameters and the judgment parameters is fitted. Then, the acquisition function is used to find a balance between the unexplored area and the area with the potential to approach the target parameter, and the new geometric optical hyperparameter to be observed is determined. Before the retrieval is completed, the geometric optical hyperparameter is continuously iterated to gradually approach the value that meets the requirements. On this basis, the hypersurface parameters and the neural network parameters can also be iterated to reasonable values, thereby obtaining the three parameters of the target geometric optical hyperparameters, hypersurface parameters and neural network parameters at the same time. Reasonable values ​​are obtained for each parameter with relatively small restrictions. With the design method of hardware and software integration, the effect of high design efficiency and good classification accuracy of the metasurface optical neural network is achieved, which better meets the design requirements.

[0115] (2) This design method achieves an end-to-end design effect and realizes the optimal solution for the overall design. While the design efficiency is high, the final product has strong anti-interference ability and is not easily affected by external factors such as different light, ensuring stable recognition in different environments.

[0116] (3) When the metasurface optical neural network designed based on the design method of the metasurface optical neural network is applied to the task of finger vein image recognition, it can better meet the recognition requirements of finger vein images by adaptively improving the loss function.

[0117] Design device for optical neural network on super surface

[0118] In one embodiment, a design device for a supersurface optical neural network is provided, such as Figure 4 The design device of the super surface optical neural network includes modules, and the specific functions of each module are as follows:

[0119] An acquisition module, used for acquiring a sample set, wherein the sample set includes a plurality of parameter samples, each parameter sample includes a set of geometric optics hyperparameters, hypersurface parameters and neural network parameters, and a judgment parameter corresponding to the sample;

[0120] A priori module, used for fitting an approximate distribution function of the judgment parameter using a probabilistic proxy model, in which the independent variables are the geometric optical hyperparameters, the hypersurface parameters and the neural network parameters, and the dependent variable is the judgment parameter;

[0121] An acquisition module, used for determining a geometrical optical hyperparameter to be observed using an acquisition function on the approximate distribution function;

[0122] A judgment module is used to judge whether the termination condition is reached;

[0123] If the termination condition is not reached, the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed are calculated, the parameter samples are updated, and the contents in the a priori module, the a priori module, and the judgment module are circulated;

[0124] If the termination condition is reached, the corresponding geometric optical hyperparameters, hypersurface parameters and neural network parameters are determined as target parameters of the hypersurface optical neural network according to the judgment parameters;

[0125] A design module is used to design the supersurface optical neural network according to the target primitive distribution.

[0126] It should be noted that for details not disclosed in the design device of the supersurface optical neural network in the embodiment of the present invention, please refer to the details disclosed in the design method of the supersurface optical neural network in the embodiment of the present invention.

[0127] Those skilled in the art will understand that the module schematic diagram is merely an example of a design device for a metasurface optical neural network, and does not constitute a limitation on the terminal device of the design device for a metasurface optical neural network. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the design device for a metasurface optical neural network may also include input and output devices, network access equipment, buses, etc.

[0128] The design device of the supersurface optical neural network may also include computing devices such as computers, notebooks, PDAs, and cloud servers, as well as including but not limited to a processing module, a storage module, and a computer program stored in the storage module and executable on the processing module, such as the above-mentioned design method program for the supersurface optical neural network. When the processing module executes the computer program, the steps in the above-mentioned design method embodiments for the supersurface optical neural network are implemented, such as Figure 3 Steps shown.

[0129] In addition, the present invention also proposes an electronic device, which includes a storage module and a processing module. When the processing module executes the computer program, it can implement the steps in the above-mentioned method for designing a metasurface optical neural network, that is, implement the steps in any one of the technical solutions in the above-mentioned method for designing a metasurface optical neural network.

[0130] The electronic device can be a part of a design device integrated into a metasurface optical neural network, or a local terminal device, or a part of a cloud server.

[0131] The processing module can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processing module is the control center of the design device of the super-surface optical neural network, and uses various interfaces and lines to connect the various parts of the design device of the entire super-surface optical neural network.

[0132] The storage module can be used to store the computer program and / or module, and the processing module realizes various functions of the design device of the super surface optical neural network by running or executing the computer program and / or module stored in the storage module, and calling the data stored in the storage module. The storage module can mainly include a storage program area and a storage data area, wherein the storage program area can store an operating system, at least one application required for a function, etc. In addition, the storage module can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0133] Exemplarily, the computer program can be divided into one or more modules / units, which are stored in a storage module and executed by a processing module to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the design device of the metasurface optical neural network.

[0134] Furthermore, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by a processing module, can implement the steps in the above-mentioned method for designing a metasurface optical neural network, that is, implement the steps in any one of the technical solutions in the above-mentioned method for designing a metasurface optical neural network.

[0135] If the module integrated in the design method of the metasurface optical neural network is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processing module, the steps of each of the above-mentioned method embodiments can be implemented.

[0136] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0137] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0138] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for designing a supersurface optical neural network, characterized in that: The supersurface optical neural network includes an optical system and a deep neural network, the optical system includes a supersurface, and the design method includes the following steps: Step S10: Acquire a sample set, wherein the sample set includes parameter samples, each group of parameters in the parameter samples includes geometric optics hyperparameters, hypersurface parameters and neural network parameters, and judgment parameters corresponding to the sample; Step S20: using a probabilistic proxy model to fit the independent variables of the geometric optical hyperparameters, the hypersurface parameters and the neural network parameters, and the dependent variable being the approximate distribution function of the judgment parameter; Step S30: using an acquisition function to determine the geometrical optical hyperparameters to be observed for the approximate distribution function; Step S40: Determine whether the termination condition is reached; If the termination condition is not reached, the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed are calculated, the parameter samples are updated, and steps S20 to S40 are looped; If the termination condition is reached, the corresponding geometric optical hyperparameters, hypersurface parameters and neural network parameters are determined as target parameters of the hypersurface optical neural network according to the judgment parameters; Step S50: Designing the hypersurface optical neural network according to the target primitive distribution.

2. The method for designing a supersurface optical neural network according to claim 1, characterized in that: The optical system further comprises an aperture and an imaging surface, and the geometric optical hyperparameter comprises at least one of a distance parameter between the target and the aperture, a distance parameter between the aperture and the hypersurface, and a distance parameter between the hypersurface and the imaging surface; The hypersurface includes a plurality of primitives, and the hypersurface parameters include size parameters of the primitives; The neural network parameters include weight parameters of the deep neural network; The judgment parameter is the accuracy and / or loss value generated by the deep neural network.

3. The method for designing a supersurface optical neural network according to claim 1, characterized in that: The method for generating the parameter sample comprises the following steps: A set of geometric optical hyperparameters is selected, and the deep neural network is trained using a gradient optimization method based on the set of geometric optical hyperparameters, and the obtained gradient optimal values ​​are determined as the hypersurface parameters and neural network parameters corresponding to the geometric optical hyperparameters, as well as the judgment parameters.

4. The method for designing a supersurface optical neural network according to claim 1, characterized in that: The probabilistic agent model uses a Gaussian process; The acquisition function is a maximized expected improvement function, a probability increment function, an expected increment function, a confidence upper bound function, or an information entropy function; The process of step S30 includes: Determine the promotion probability corresponding to each group of independent variables according to the normal distribution; The geometrical optics hyperparameter among the independent variables with the greatest improvement probability is determined as the observed geometrical optics hyperparameter.

5. The method for designing a supersurface optical neural network according to claim 1, characterized in that: The sample set also includes image samples, and the image samples include a plurality of training images and true values ​​corresponding to the training images. Step S40 includes: Based on the geometric optical hyperparameters to be observed, the deep neural network is trained according to the image samples, and the deep neural network is trained using a gradient optimization method, and the obtained gradient optimal values ​​are determined as the hypersurface parameters and neural network parameters corresponding to the geometric optical hyperparameters, as well as the judgment parameters.

6. The method for designing a supersurface optical neural network according to claim 5, characterized in that: The step of calculating the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometrical optical hyperparameters to be observed includes: Calculate the classification result of the training image obtained by the hypersurface optical neural network, the calculation formula of the classification result y is y=f(g(φ,x),θ), wherein φ is the hypersurface parameter, x is the feature of the training image, g function is the encoding of the training image by the hypersurface, θ is the neural network parameter, and f function is the interaction between the deep neural network and the image encoded by the hypersurface; The difference between the classification result and the true value corresponding to the training image is calculated to obtain the judgment parameter.

7. The method for designing a supersurface optical neural network according to claim 1, characterized in that: The termination condition includes that the acquisition times of the acquisition function reaches the maximum acquisition times, or the judgment parameter meets the target condition.

8. The method for designing a supersurface optical neural network according to claim 1, characterized in that: The image in the image sample is a finger vein image, and the hypersurface optical neural network is used to identify the finger vein image.

9. The method for designing a supersurface optical neural network according to claim 8, characterized in that: Based on the finger vein image recognition task, the loss function of the deep neural network is: loss = α*L1+β*L2, wherein L1 is the cosine boundary softmax loss function, L2 is the triplet loss function, and α and β are weighted coefficients for balancing the cosine boundary softmax loss function and the triplet loss function, respectively.

10. A design device for a supersurface optical neural network, characterized in that: The supersurface optical neural network includes an optical system and a deep neural network, the optical system includes a supersurface, and the design device includes: An acquisition module, used for acquiring a sample set, wherein the sample set includes parameter samples, each group of parameters in the parameter samples includes geometric optics hyperparameters, hypersurface parameters and neural network parameters, and a judgment parameter corresponding to the sample; A priori module, used for fitting an approximate distribution function of the judgment parameter using a probabilistic proxy model, in which the independent variables are the geometric optical hyperparameters, the hypersurface parameters and the neural network parameters, and the dependent variable is the judgment parameter; An acquisition module, used for determining a geometrical optical hyperparameter to be observed using an acquisition function on the approximate distribution function; A judgment module is used to judge whether the termination condition is reached; If the termination condition is not reached, the hypersurface parameters, neural network parameters, and judgment parameters corresponding to the geometric optical hyperparameters to be observed are calculated, the parameter samples are updated, and the contents in the a priori module, the a priori module, and the judgment module are circulated; If the termination condition is reached, the corresponding geometric optical hyperparameters, hypersurface parameters and neural network parameters are determined as target parameters of the hypersurface optical neural network according to the judgment parameters; A design module is used to design the supersurface optical neural network according to the target primitive distribution.

11. An electronic device, characterized in that: include: A storage module storing a computer program; A processing module, which can implement the steps in the method for designing a supersurface optical neural network as described in any one of claims 1 to 9 when executing the computer program.

12. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processing module, the steps in the method for designing a supersurface optical neural network described in any one of claims 1 to 9 can be implemented.

Images