Design method and device of optical system based on metasurface image encryption

Abstract

The invention discloses a design method and device for an optical system based on metasurface image encryption, the optical system comprises an optical neural network and an artificial neural network, the optical neural network comprises a plurality of layers of metasurfaces, and the design method comprises the following steps: calculating imaging data obtained after image data passes through the optical neural network; processing the imaging data through an artificial neural network to obtain a classification result; confirming a loss value through a loss function according to the classification result and an actual value corresponding to the image data; meanwhile, trainable parameters of the optical neural network and the artificial neural network are trained and updated. According to the method, on one hand, an encrypted image can be directly obtained and stored in a disk, encryption is fast, original unencrypted data do not need to be stored in the disk, and information leakage is avoided from the source; on the other hand, the optical neural network and the artificial neural network which are jointly optimized are generated through an end-to-end method, and the image recognition or classification effect is better than that of a pure artificial neural network model.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and optical technology, and in particular to a design method and device for an optical system based on metasurface image encryption. Background Art

[0002] The privacy protection of image data has attracted more and more attention. For example, face data collected in public places involves everyone's privacy. Although the collection of faces has many benefits, such as better protection for social security, these original image data have been stored on disk. Once leaked or improperly used, it will cause great security problems, which in turn has caused people to worry about the privacy protection of image data. Even if the algorithm is encrypted, the problem of the original data being stored on disk cannot be avoided. It is difficult to resist the third party to decode and restore the original image through technical means. In addition, the use of algorithm encryption also has speed and energy consumption issues.

[0003] In the process of realizing the present invention, the inventors found that an optical system based on metasurface image encryption can help solve this problem. The optical system uses a metasurface to obtain images that are blurred and cannot be recognized by the human eye. It is difficult to restore them to clear images through software means later. Therefore, such images have the advantage of good natural confidentiality. As for how to use these images encrypted by the metasurface for subsequent data recognition or classification, the present invention has made customized algorithm model optimization. Summary of the invention

[0004] In order to solve the privacy protection problem of image data in the above-mentioned prior art problems, the purpose of the present invention is to provide a design method and device for an optical system based on metasurface image encryption with good encryption effect and good image recognition or classification effect. 。

[0005] To achieve the above-mentioned purpose of the invention, an embodiment of the present invention provides a design method of an optical system based on metasurface image encryption, wherein the optical system includes an optical neural network and an artificial neural network, wherein the optical neural network includes several layers of the metasurface, and the design method includes the following steps:

[0006] Step S10: Initializing the hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of several layers of primitives on the hypersurface;

[0007] Step S20: Acquire image data for training;

[0008] Step S30: calculating imaging data obtained after the image data passes through the optical neural network;

[0009] Step S40: obtaining a classification result by applying an artificial neural network to the imaging data;

[0010] Step S50: confirming a loss value through a loss function according to the classification result and the actual value corresponding to the image data;

[0011] Step S60: looping the above steps S30 to S50, and simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value;

[0012] Step S70: confirming the model parameters of the artificial neural network that has completed training, and confirming the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

[0013] As a further improvement of the present invention, the optical neural network further includes an aperture, a lens group and a COMS, and the aperture, the lens group, a plurality of layers of the metasurface and the COMS are arranged in sequence; the step S30 further includes:

[0014] Calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group;

[0015] Performing FDTD calculation on the optical signal passing through the plurality of layers of the metasurface;

[0016] The imaging data obtained when the optical signal leaves the metasurface and enters the COMS is calculated.

[0017] As a further improvement of the present invention, the lens group includes several layers of metalenses, and the hardware parameters include metalens geometric parameters of primitives on several layers of the metalenses;

[0018] The step S10 further includes:

[0019] Initializing the metalens geometric parameters;

[0020] The step S70 further includes:

[0021] Based on the optical neural network that has completed the training, the geometric parameters of the metalens are confirmed.

[0022] As a further improvement of the present invention, the step of calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group includes:

[0023] Calculating the imaging of the light signal when it reaches the superlens through the aperture;

[0024] Performing FDTD calculation on the optical signal passing through several layers of the metalens;

[0025] The imaging of the optical signal when it leaves the metalens and reaches the metasurface is calculated.

[0026] As a further improvement of the present invention, the hardware parameters include aperture parameters and COMS parameters;

[0027] The step S10 further includes:

[0028] Initialize the aperture parameters and the COMS parameters;

[0029] The step S70 further includes:

[0030] Based on the optical neural network that has completed training, at least one of the above-mentioned aperture parameters, COMS parameters, and the distances between adjacent devices such as the aperture, lens group, metasurface, and CMOS is confirmed.

[0031] As a further improvement of the present invention, the artificial neural network uses a deep residual network model as a keel model;

[0032] The step S40 comprises:

[0033] Inputting the imaging data into the deep residual network model for feature extraction, and outputting a predicted probability value through a fully connected layer;

[0034] The highest probability is selected as the classification result.

[0035] As a further improvement of the present invention, step S50 includes:

[0036] The classification result is compared with the actual value corresponding to the image data, and the loss value after comparison is calculated by using binary cross entropy or mean square error as the loss function.

[0037] As a further improvement of the present invention, the trainable parameters of the optical neural network include amplitude and phase determined according to the hardware parameters, the trainable parameters of the artificial neural network include weights and biases, the image data also includes a verification set for verification, and the step S60 further includes:

[0038] By means of an optimization algorithm, the amplitude and phase, the weight and the deviation are gradient updated, and based on the updated optical neural network and the artificial neural network, a new classification result is obtained through the steps S30 and S40, and a new loss value is calculated through step S50 until the loss value converges to a stable level, and the parameters of each network are recorded in the process;

[0039] Verify the accuracy and loss rate of each network through the verification set, and confirm the optimal network as the optical neural network and the artificial neural network that have completed training;

[0040] The step S70 further includes:

[0041] Based on the amplitude and phase of the trained optical neural network, the metasurface geometric parameters are reversely deduced.

[0042] As a further improvement of the present invention, the hypersurface geometric parameters include the shape and period of the primitive, and the size parameters and rotation angle parameters corresponding to the shape;

[0043] The design method also includes the steps of: coordinating optical system design and process iteration with the metasurface geometric parameters to obtain the structural parameters of the optical neural network.

[0044] To achieve one of the above-mentioned objects of the invention, an embodiment of the present invention provides a design device for an optical system based on metasurface image encryption, wherein the optical system includes an optical neural network and an artificial neural network, wherein the optical neural network includes several layers of the metasurface, and the design device includes:

[0045] An initialization module, used to initialize the hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of primitives on several layers of the hypersurface;

[0046] An acquisition module, used for acquiring image data for training;

[0047] An optical neural network calculation module, used to calculate imaging data obtained after the image data passes through the optical neural network;

[0048] An artificial neural network classification module, used for obtaining a classification result by applying an artificial neural network to the imaging data;

[0049] A loss function calculation module, used to determine a loss value through a loss function according to the classification result and an actual value corresponding to the image data;

[0050] A model training module, used for simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value;

[0051] A parameter generation module is used to confirm the model parameters of the artificial neural network that has completed training, and to confirm the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

[0052] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides an electronic device, including:

[0053] A storage module storing a computer program;

[0054] A processing module, when executing the computer program, can implement the steps in the above-mentioned method for designing an optical system based on metasurface image encryption.

[0055] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by a processing module, can implement the steps in the above-mentioned method for designing an optical system based on metasurface image encryption.

[0056] Compared with the prior art, the present invention has the following beneficial effects: the design method proposes an end-to-end optimization implementation scheme. On the one hand, the encrypted image can be directly obtained by hardware. The image finally obtained after the optical signal passes through the metasurface is not a clear and directly recognizable image. The encrypted image can be directly stored on the disk. The encryption is undertaken by the hardware metasurface. The encryption process is an optical process with fast speed and low energy consumption, and there is no need to store the original unencrypted data on the disk, thus avoiding information leakage from the source. On the other hand, a jointly optimized optical neural network and artificial neural network are generated through an end-to-end method. The optical neural network can be better identified or classified through the artificial neural network, and the image recognition or classification effect is better than that of the simple artificial neural network model, which not only meets the public's demand for privacy protection of image data, but also meets the needs of image users for the purpose of image acquisition. In addition, the operation speed of the artificial neural network is also faster. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of a method for designing an optical system according to an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of the structure of an optical neural network according to an embodiment of the present invention;

[0059] Figure 3 is a schematic structural diagram of a metasurface or a metalens according to an embodiment of the present invention;

[0060] Figure 4 is a schematic diagram of a partial structure of one layer of a metasurface or a metalens according to an embodiment of the present invention;

[0061] Figure 5 It is a schematic diagram of a module of a design device for an optical system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0063] An embodiment of the present invention provides a method and device for designing an optical system based on metasurface image encryption with good encryption effect and good image recognition or classification effect.

[0064] The optical system of this embodiment includes an optical neural network and an artificial neural network, wherein the optical neural network includes several layers of the metasurface. The imaging of the light signal after passing through the metasurface cannot obtain a clear image, which is equivalent to encryption preprocessing before imaging. As a result, it is difficult to store a clear image when storing on a disk. Therefore, the optical system can achieve encryption effect at the hardware level.

[0065] In addition, the programmable characteristics of the metasurfaces of several layers form a structure similar to a neural network, which can also undertake some image feature extraction tasks and further accelerate the calculation speed of subsequent artificial neural networks.

[0066] The design method of the optical system based on metasurface image encryption of this embodiment encrypts the image information through the metasurface, and simultaneously trains the trainable parameters of the optical neural network and the artificial neural network through an end-to-end method, continuously updates and optimizes the models of the optical neural network and the artificial neural network, and controls the hardware parameters of the optical neural network and the weights and deviations of the artificial neural network, and finally achieves the goal of judgment and classification, that is, better decrypts the image information through the artificial neural network. Therefore, this design method meets both the needs of image encryption and the needs of image decryption, and the entire design method pursues global optimization.

[0067] The optical system can be optimized for a specific target, for example, it can be an optical system mainly used for face recognition, taking photos of passers-by in public places, and matching target objects in the crowd. In this process, images are acquired through an optical neural network, and image analysis is performed through an artificial neural network, i.e., face recognition. The optimization direction is that relatively fuzzy images for human eye recognition can be relatively accurately classified by the artificial neural network. Other scenes can also be classifications such as male and female classification, human fatigue status, human and vehicle classification, license plate number recognition, etc. Different scenes can have different more optimized parameters. The following steps are further explained using face recognition as an example.

[0068] The following is divided into three embodiments to illustrate a design method of an optical system based on metasurface image encryption provided by an embodiment of the present invention. The main difference between the three embodiments is that in embodiment 1, the optical neural network is mainly designed and trained for the parameters of the metasurface. In embodiment 2, the lens of the optical neural network is set as a superlens. In addition to the training of the parameters of the metasurface in the optical neural network in embodiment 1, the training of the parameters of the superlens of the optical neural network is also included. Embodiment 3 can be based on embodiment 1 or embodiment 2, and the parameters of the aperture and / or CMOS can also be trained.

[0069] Example 1

[0070] Combine the following Figure 1 to Figure 4 , describing a design method of an optical system based on metasurface image encryption provided by an embodiment of the present invention, although the present application provides the method operation steps as shown in the following implementation or flow chart, based on routine or no creative labor, the execution order of the steps in the method that do not have a necessary causal relationship in logic is not limited to the execution order provided in the implementation of the present application. For example, the acquisition order of steps S10 and S20 below can be adjusted arbitrarily, without distinguishing the order in time sequence.

[0071] A design method of an optical system based on metasurface image encryption in this embodiment is as follows: Figure 1 As shown, the following steps S10 to S70 are included:

[0072] Step S10: Initialize the hardware parameters of the optical neural network, wherein the hardware parameters include the hypersurface geometric parameters of several layers of primitives on the hypersurface.

[0073] Schematic diagram of the structure of the optical neural network Figure 2 As shown, it includes an aperture, a lens group, a metasurface and a CMOS (Complementary Metal Oxide Semiconductor). The aperture is used to collect light field information, the lens group is used for imaging, the metasurface is used to control the propagation of light, and the CMOS is used to complete photoelectric conversion and record the distribution of light intensity. The aperture, the lens group, several layers of metasurfaces and CMOS are set in sequence.

[0074] The structure of the super surface can be Figure 3 As shown in , the hypersurface can have a multi-layer structure. The value of n in the figure can be 2, 3, 4, 5 or even more. There are many primitives to be trained on each layer of the hypersurface, such as Figure 4As shown, a structure similar to a neural network is formed by a multi-layer metasurface, which greatly improves its expression ability and enables the optical neural network to have a classification function. In other words, the programmable metasurface structure has an encryption effect on the one hand, and assists in data classification operations on the other.

[0075] Other apertures, lenses and CMOS can be used off-the-shelf or can be deeply customized by participating in the training of the model through the following Examples 2 and 3, which will be explained in detail below.

[0076] The shape of the primitive can be linear, cylindrical, cuboid, elliptical cylinder, hollow elliptical body, hollow cuboid, etc. Different shapes can correspond to some identical and some different size parameters and rotation angle parameters, such as period parameters and height parameters, and different parameters such as the radius of the cylinder, the constant and width of the cuboid, the major diameter and minor diameter of the elliptical cylinder, etc. Different primitives can have different rotation angle parameters. The parameters of these primitives together constitute the geometric parameters of the metasurface. The control effects of these different primitives on the light field can be accurately numerically calculated by algorithms such as FDTD.

[0077] The initialization process can be to generate a random matrix, which is the assignment of hardware parameters of the optical neural network. In this embodiment 1, if the trainable parameters in the optical neural network are mainly for the metasurface, the random matrix can be a parameter simulation of each element of the metasurface structure.

[0078] Step S20: Acquire image data for training.

[0079] In combination with the above, the design method of the optical system of this embodiment can optimize the model more specifically for some specific scenes to make the classification result more accurate. This embodiment takes face recognition as an example, and the acquired image data is face data.

[0080] The step S20 may include steps S21 to S22:

[0081] Step S21: unify the size of the face data images, for example, upsample or downsample images of different sizes to a uniform size.

[0082] Step S22: Process the color head portrait into a gray image. This can make the image data more consistent and meet the classification requirements of different scenes in different weather and different times of the day, such as day and night.

[0083] In subsequent steps, the grayscale images of uniform size are used for model training, verification, testing, etc.

[0084] Step S30: Calculate imaging data obtained after the image data passes through the optical neural network.

[0085] The step S30 may include steps S31 to S32:

[0086] Step S31: calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group;

[0087] Step S32: performing FDTD calculation on the optical signal passing through the plurality of layers of the metasurface;

[0088] Step S33: Calculate the imaging data obtained when the light signal leaves the metasurface and enters the COMS.

[0089] When calculating the propagation of light signals in the far field, that is, the order of magnitude of the propagation distance is much greater than the order of magnitude of the wavelength of light, and the light signals propagate between optical devices, the Rayleigh-Sommerfield formula or the Green function vector diffraction formula can be used for calculation. When calculating the propagation of light signals in the near field, that is, the order of magnitude of the propagation distance is roughly equivalent to the order of magnitude of the wavelength of light, and the light signals propagate within optical devices, such as between several layers of a metasurface, FDTD can be used for calculation. Thus, the imaging data when the image data reaches COMS through the optical neural network can be simulated and calculated.

[0090] Step S40: obtaining a classification result by applying an artificial neural network to the imaging data.

[0091] The artificial neural network uses a deep residual network model (ResNet, Residual Neural Network) as a keel model. ResNet builds a deep network by introducing residual connections, which can enable the network to learn and optimize more effectively. In the present embodiment, more specifically, ResNet18, a relatively shallow ResNet model, is used, which consists of a convolutional layer, a pooling layer, and a fully connected layer. Its main parts are convolutional layers and residual blocks, and the fully connected layer is located at the end of the network. The convolutional layer and the residual block are used to extract image features, and gradually reduce the size of the feature map, and finally perform classification or regression tasks through the fully connected layer. The relatively shallow structure of ResNet18 makes it easier to train when computing resources are limited, and performs well on the image classification task of the design method of this embodiment.

[0092] The step S40 further includes:

[0093] Inputting the imaging data into the deep residual network model for feature extraction, and outputting a predicted probability value through a fully connected layer;

[0094] The highest probability is selected as the classification result.

[0095] Taking the ResNet18 above as an example, the fully connected layer in the last few layers of ResNet18 plays the role of converting high-dimensional features into categories or regression outputs. It receives the feature vector of the previous layer as input, performs linear transformation through a series of weight matrices and bias terms, and then applies the activation function to generate the final output. The activation function can be softmax, sigmoid or tanh, etc. The fully connected layer introduces nonlinearity in the convolutional neural network and maps high-dimensional features to specific task outputs, such as the task of face recognition in this embodiment, to obtain multiple classification results and the corresponding probability of each classification. Finally, the highest probability is used as the output category.

[0096] Step S50: confirming a loss value through a loss function according to the classification result and the actual value corresponding to the image data.

[0097] Specifically, the classification result may be compared with an actual value corresponding to the image data, and a loss value after comparison may be calculated by using binary cross entropy or mean square error as a loss function.

[0098] The loss function can be reasonably weighed and adjusted according to specific task requirements, data characteristics and model design, so as to guide the trainable parameters more effectively during the training process. Here, the loss function can be the weighted sum of the losses in different models such as optical neural networks and artificial neural networks as the loss function of the entire network.

[0099] Step S60: looping the above steps S30 to S50, and simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value.

[0100] In this step, the loss value of step S50 can be back-propagated back to the network, and then the gradients of all parameters involved in the calculation in the optical neural network and the artificial neural network can be calculated by the chain rule. Wherein, the trainable parameters of the optical neural network in this embodiment 1 include the amplitude and phase determined according to the hardware parameters, and the trainable parameters of the artificial neural network include weights and biases. The trainable parameters of the optical neural network and the artificial neural network can be gradient updated by a model training optimizer, and the model training optimizer can use AdamW.

[0101] The step S60 specifically includes:

[0102] Step S61: Through the optimization algorithm, the amplitude and phase, the weight and the deviation are gradient updated, and based on the updated optical neural network and the artificial neural network, a new classification result is obtained through the steps S30 and S40, and then a new loss value is calculated through step S50 until the loss value converges to a stable level, and the parameters of each network are recorded in the process.

[0103] Step S61 is used to update the model, monitor and optimize the performance of the model through the two important evaluation indicators of loss value and accuracy, and update the model until these two indicators converge to a stable level. During the training process, the model file after each round of optimization can be saved.

[0104] The step S60 further includes:

[0105] Step S62: verifying the accuracy and loss rate corresponding to each network through the verification set, and confirming the optimal network as the optical neural network and the artificial neural network that have completed training.

[0106] Step S62 is used to select the optimal model and verify the model by cross-validation. The validation set is used for the hyperparameter selection and tuning of the model. By observing the loss and accuracy on the validation set, appropriate hyperparameters such as learning rate, regularization parameters, etc. can be selected, or the best model version can be determined. In addition, part of the image data can be set as a test set, for example, 70-80% of the samples are used for training, 10-15% for verification, and 10-15% for testing. The specific ratio can be adjusted according to the scale of face recognition and data set.

[0107] Through the above steps S61 and S62, the model will update parameters according to the data of the training set, and then evaluate the performance of the model on the validation set.

[0108] Step S70: confirming the model parameters of the artificial neural network that has completed training, and confirming the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

[0109] In this embodiment, the hypersurface geometric parameters are reversely derived based on the amplitude and phase of the trained optical neural network. In the following embodiments 2 and 3, other hardware parameters can also be reversely derived.

[0110] The hypersurface geometric parameters include the shape and period of the primitive, and the size parameters and rotation angle parameters corresponding to the shape. The design method also includes the step of: coordinating the hypersurface geometric parameters with the optical system design and process iteration to obtain the structural parameters of the optical neural network.

[0111] Specifically, the structural parameters of the optical neural network can be a photomask data format file (GDS, Graphical Data System), which is a file format used to describe and record the layout of the metasurface, including information such as the physical layout and characteristic parameters of the metasurface, as a basis and guide for subsequent process manufacturing. Then, the corresponding hardware structure of the optical neural network can be produced. For example, the process department can create a metasurface lens that matches the parameters of the metasurface of Example 1.

[0112] After applying the above method to obtain the trained optical neural network and artificial neural network, the hardware structure of the optical neural network and the matching software are obtained. In actual use, by inputting new samples, the corresponding probability can be output to achieve the classification goal.

[0113] Example 2

[0114] The difference between Example 2 and Example 1 is that the lens group may include several layers of metalenses, and the hardware parameters include metalens geometric parameters of the primitives on the several layers of the metalenses.

[0115] The super lens can have a more ultra-thin effect. The structure of the super lens can refer to Figure 3 As shown, the metalens has a multi-layer structure, and many primitives are also arranged on each layer of the metalens. The specific structure of the primitives can refer to the description of the primitives of the metasurface above. The metalens of this embodiment can process both monochromatic light, such as 940nm wavelength, and multicolor, such as RGB three center wavelengths, and the metalens can also optimize the dispersion effect.

[0116] Based on Example 1, the steps of the design method of Example 2 can be adaptively adjusted as follows:

[0117] The step S10 further includes:

[0118] Initialize the metalens geometric parameters.

[0119] The specific method of initialization can refer to the method in Example 1, which uses a random matrix to generate corresponding parameters. These parameters can be continuously updated and adjusted in subsequent model training and optimization.

[0120] Step S31 in Embodiment 2 includes:

[0121] Step S311: calculating the imaging of the light signal when it reaches the superlens through the aperture;

[0122] Step S312: performing FDTD calculation on the optical signals passing through the plurality of layers of the superlens;

[0123] Step S313: Calculate the imaging of the light signal when it leaves the metalens and reaches the metasurface.

[0124] Since the light signal propagates between several layers of metalenses, it belongs to the near-field propagation mentioned above, so the Rayleigh-Sommerfield formula or the Green's function vector diffraction formula is also used for calculation in step S311, the FDTD calculation is used in step S312, and the Rayleigh-Sommerfield formula or the Green's function vector diffraction formula is continued to be used for calculation in step S313.

[0125] In step S60, the metalens geometric parameters are also trained as part of the optical neural network. Such a more comprehensive end-to-end design is more conducive to the subsequent decryption of the artificial neural network.

[0126] The step S70 further includes:

[0127] Based on the optical neural network that has completed the training, the geometric parameters of the metalens are confirmed.

[0128] This embodiment 2 not only obtains the geometric parameters of the metasurface in embodiment 1, but also obtains the geometric parameters of the metalens through an end-to-end design method.

[0129] Finally, the process department creates a superlens that matches the training parameters of Example 2.

[0130] Example 3

[0131] In Example 3, the hardware parameters include aperture parameters, COMS parameters, and at least one of the parameters such as the distance between adjacent devices such as the aperture, lens group, metasurface, and CMOS. Preferably, the aperture parameters, COMS parameters, the distance between adjacent devices such as the aperture, lens group, metasurface, and CMOS can all be used as part of the model training.

[0132] Example 3 can be implemented on the basis of Example 1 or on the basis of Example 2. Taking the implementation based on Example 2 as an example, not only the parameters of the super-lens and super-surface of the optical neural network are designed end-to-end, but also the aperture parameters, COMS parameters, and the distances between adjacent devices such as the aperture, lens group, super-surface, and CMOS are designed, thereby further realizing the pursuit of global optimization for the entire design method.

[0133] On the basis of Example 2, the steps of the design method of Example 3 can be adaptively adjusted as follows: Step S10 further includes:

[0134] Initialize the aperture parameters and the COMS parameters.

[0135] The initialization here can still be obtained by assigning random values.

[0136] In step S60, the aperture parameters, COMS parameters, distances between components of the optical neural network, that is, the parameters of each optical component of the entire optical neural network are all part of the training.

[0137] The step S70 further includes:

[0138] Based on the optical neural network that has completed training, the aperture parameters and the COMS parameters are confirmed.

[0139] Finally, the process department creates or purchases the aperture and CMOS that match the aperture parameters and COMS parameters.

[0140] Compared with the prior art, this embodiment has the following beneficial effects:

[0141] On the one hand, the optical system designed by this design method can directly obtain an encrypted image. The image finally obtained after the light signal passes through the metasurface is not a clear and directly recognizable image. The encrypted image can be directly stored on the disk, with fast encryption and no need to store the original unencrypted data on the disk, thus avoiding information leakage from the source. On the other hand, a matching optical neural network and artificial neural network are generated through an end-to-end method. The artificial neural network can be used to better identify or classify the optical neural network, and the image recognition or classification effect is better than that of a simple artificial neural network model, which not only meets the public's demand for privacy protection of image data, but also meets the needs of image users for the purpose of image acquisition.

[0142] In one embodiment, a design device for an optical system based on metasurface image encryption is provided, such as Figure 5 The design device of the optical system based on metasurface image encryption includes modules, and the specific functions of each module are as follows:

[0143] An initialization module, used to initialize the hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of primitives on several layers of the hypersurface;

[0144] An acquisition module, used for acquiring image data for training;

[0145] An optical neural network calculation module, used to calculate imaging data obtained after the image data passes through the optical neural network;

[0146] An artificial neural network classification module, used for obtaining a classification result by applying an artificial neural network to the imaging data;

[0147] A loss function calculation module, used to determine a loss value through a loss function according to the classification result and an actual value corresponding to the image data;

[0148] A model training module, used for simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value;

[0149] A parameter generation module is used to confirm the model parameters of the artificial neural network that has completed training, and to confirm the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

[0150] It should be noted that for details not disclosed in the design device of the optical system of the embodiment of the present invention, please refer to the details disclosed in the design method of the optical system of the embodiment of the present invention.

[0151] The optical system design device may also include computing devices such as computers, notebooks, PDAs, and cloud servers, as well as including but not limited to a processing module, a storage module, and a computer program stored in the storage module and executable on the processing module, such as the optical system design method program described above. When the processing module executes the computer program, the steps in the optical system design method embodiments described above are implemented, such as Figure 1 Steps shown.

[0152] In addition, the present invention also proposes an electronic device, which includes a storage module and a processing module. When the processing module executes the computer program, it can implement the steps in the above-mentioned optical system design method, that is, implement the steps in any one of the technical solutions in the above-mentioned optical system design method.

[0153] The electronic device may be a part of a design device integrated into the optical system, or a local terminal device, or a part of a cloud server.

[0154] The processing module can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processing module is the control center of the design device of the optical system, and uses various interfaces and lines to connect the various parts of the design device of the entire optical system.

[0155] The storage module can be used to store the computer program and / or module, and the processing module realizes various functions of the optical system design device by running or executing the computer program and / or module stored in the storage module, and calling the data stored in the storage module. The storage module can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc. In addition, the storage module can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0156] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in a storage module and executed by a processing module to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the optical system design device.

[0157] Furthermore, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by a processing module, can implement the steps in the above-mentioned method for designing an optical system, that is, implement the steps in any one of the technical solutions in the above-mentioned method for designing an optical system.

[0158] If the module integrated in the optical system design method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processing module, the steps of each of the above-mentioned method embodiments can be implemented.

[0159] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0160] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0161] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A design method for an optical system based on metasurface image encryption, characterized in that: The optical system includes an optical neural network and an artificial neural network, the optical neural network includes several layers of the metasurface, and the design method includes the following steps: Step S10: Initializing the hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of several layers of primitives on the hypersurface; Step S20: Acquire image data for training; Step S30: calculating imaging data obtained after the image data passes through the optical neural network; Step S40: obtaining a classification result by applying an artificial neural network to the imaging data; Step S50: confirming a loss value through a loss function according to the classification result and the actual value corresponding to the image data; Step S60: looping the above steps S30 to S50, and simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value; Step S70: confirming the model parameters of the artificial neural network that has completed training, and confirming the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

2. The method for designing an optical system according to claim 1, characterized in that: The optical neural network further includes an aperture, a lens group and a COMS, wherein the aperture, the lens group, a plurality of layers of the metasurface and the COMS are arranged in sequence; the step S30 further includes: Calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group; Performing FDTD calculation on the optical signal passing through the plurality of layers of the metasurface; The imaging data obtained when the optical signal leaves the metasurface and enters the COMS is calculated.

3. The method for designing an optical system according to claim 2, characterized in that: The lens group includes several layers of metalenses, and the hardware parameters include metalens geometric parameters of primitives on the several layers of metalenses; The step S10 further includes: Initializing the metalens geometric parameters; The step S70 further includes: Based on the optical neural network that has completed the training, the metalens geometric parameters are confirmed.

4. The method for designing an optical system according to claim 3, characterized in that: The step of calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group comprises: Calculating the imaging of the light signal when it reaches the superlens through the aperture; Performing FDTD calculation on the optical signal passing through several layers of the superlens; The image of the optical signal when it leaves the metalens and reaches the metasurface is calculated.

5. The method for designing an optical system according to claim 2, characterized in that: The hardware parameters include aperture parameters and COMS parameters; The step S10 further includes: Initialize at least one of the aperture parameters, the CMOS parameters, and the distances between adjacent devices such as the aperture, the lens group, the metasurface, and the CMOS; The step S70 further includes: Based on the optical neural network that has completed training, at least one of the above-mentioned aperture parameters, COMS parameters, and the distances between adjacent devices such as the aperture, lens group, metasurface, and CMOS is confirmed.

6. The method for designing an optical system according to claim 1, characterized in that: The artificial neural network uses a deep residual network model as a keel model; The step S40 comprises: Inputting the imaging data into the deep residual network model for feature extraction, and outputting a predicted probability value through a fully connected layer; The highest probability is selected as the classification result.

7. The method for designing an optical system according to claim 1, characterized in that: The step S50 comprises: The classification result is compared with the actual value corresponding to the image data, and the loss value after comparison is calculated using binary cross entropy or mean square error as the loss function.

8. The method for designing an optical system according to claim 1, characterized in that: The trainable parameters of the optical neural network include amplitude and phase determined according to the hardware parameters, the trainable parameters of the artificial neural network include weights and biases, the image data also includes a verification set for verification, and the step S60 further includes: By means of an optimization algorithm, the amplitude and phase, the weight and the deviation are gradient updated, and based on the updated optical neural network and the artificial neural network, a new classification result is obtained through the steps S30 and S40, and a new loss value is calculated through step S50 until the loss value converges to a stable level, and the parameters of each network are recorded in the process; Verify the accuracy and loss rate of each network through the verification set, and confirm the optimal network as the optical neural network and the artificial neural network that have completed training; The step S70 further includes: Based on the amplitude and phase of the trained optical neural network, the metasurface geometric parameters are reversely deduced.

9. The method for designing an optical system according to claim 1, characterized in that: The hypersurface geometric parameters include the shape and period of the primitive, and the size parameters and rotation angle parameters corresponding to the shape; The design method also includes the steps of: coordinating optical system design and process iteration with the metasurface geometric parameters to obtain the structural parameters of the optical neural network.

10. A design device for an optical system based on metasurface image encryption, characterized in that: The optical system includes an optical neural network and an artificial neural network, the optical neural network includes several layers of the metasurface, and the design device includes: An initialization module, used to initialize the hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of primitives on several layers of the hypersurface; An acquisition module, used for acquiring image data for training; An optical neural network calculation module, used to calculate imaging data obtained after the image data passes through the optical neural network; An artificial neural network classification module, used for obtaining a classification result by applying an artificial neural network to the imaging data; A loss function calculation module, used to determine a loss value through a loss function according to the classification result and an actual value corresponding to the image data; A model training module, used for simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value; A parameter generation module is used to confirm the model parameters of the artificial neural network that has completed training, and to confirm the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

11. An electronic device, characterized in that: include: A storage module storing a computer program; A processing module, when executing the computer program, can implement the steps in the method for designing an optical system based on metasurface image encryption as described in any one of claims 1 to 9.

12. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processing module, the steps in the method for designing an optical system based on metasurface image encryption as described in any one of claims 1 to 9 can be implemented.

Images