Design method and apparatus for optical system based on metasurface image encryption

Abstract

Disclosed in the present invention are a design method and apparatus for an optical system based on metasurface image encryption. In the method, an encrypted image is directly obtained, and the encrypted image is stored in a disk, such that encryption is fast, and original unencrypted data does not need to be stored in the disk, thereby preventing information leakage from a source; and an optical neural network and an artificial neural network which are jointly optimized are generated by means of an end-to-end method, such that an image recognition or classification effect is better than that of merely an artificial neural network model.

Description

Design method and device of optical system based on metasurface image encryption

[0001] This application claims priority to a Chinese patent application filed on July 26, 2023, with application number 202310924213.2, entitled “Design method and device for optical system based on metasurface image encryption”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present invention relates to the fields of deep learning and optical technology, and in particular to a design method and device for an optical system based on metasurface image encryption. Background Art

[0003] Image data privacy protection is gaining increasing attention. For example, facial data collected in public places involves everyone's privacy. While facial data collection has many benefits, such as improving social security, the raw image data is already stored on disk. Leakage or improper use would pose significant security risks, raising concerns about image data privacy. Even with algorithmic encryption, the issue of raw data being stored on disk remains unavoidable, making it difficult to prevent third parties from decrypting and restoring the original image through technical means. Furthermore, algorithmic encryption poses speed and energy consumption challenges.

[0004] During the implementation of this invention, the inventors discovered that an optical system based on metasurface image encryption can help address this problem. By using a metasurface, this optical system creates a blurred image that is unreadable to the human eye upon acquisition, making it difficult to restore it to a clear image using software. This inherent advantage of image security is its high security. The inventors have optimized a customized algorithm model to utilize these metasurface-encrypted images for subsequent data recognition or classification. Summary of the Invention

[0005] In order to solve the privacy protection problem of image data in the above-mentioned prior art problems, the purpose of the present invention is to provide a design method and device for an optical system based on metasurface image encryption with good encryption effect and good image recognition or classification effect.

[0006] To achieve the above-mentioned object, one embodiment of the present invention provides a design method for an optical system based on metasurface image encryption, wherein the optical system includes an optical neural network and an artificial neural network, wherein the optical neural network includes several layers of the metasurface, and the design method includes the following steps:

[0007] Step S10: Initializing the hardware parameters of the optical neural network, wherein the hardware parameters include the hypersurface geometric parameters of several layers of primitives on the hypersurface;

[0008] Step S20: Acquire image data for training;

[0009] Step S30: Calculating imaging data obtained after the image data passes through the optical neural network;

[0010] Step S40: applying an artificial neural network to the imaging data to obtain a classification result;

[0011] Step S50: determining a loss value through a loss function according to the classification result and the actual value corresponding to the image data;

[0012] Step S60: looping the above steps S30 to S50, and simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value;

[0013] Step S70: confirming the model parameters of the artificial neural network that has completed training, and confirming the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the metasurface geometric parameters.

[0014] As a further improvement of the present invention, the optical neural network further includes an aperture, a lens group, and a COMS, wherein the aperture, the lens group, several layers of the metasurface, and the COMS are arranged in sequence; and step S30 further includes:

[0015] Calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group;

[0016] Performing FDTD calculations on optical signals passing through several layers of the metasurface;

[0017] The imaging data obtained when the light signal leaves the metasurface and enters the COMS is calculated.

[0018] As a further improvement of the present invention, the lens group includes several layers of metalenses, and the hardware parameters include metalens geometric parameters of the primitives on the several layers of the metalenses;

[0019] The step S10 further includes:

[0020] Initializing the metalens geometric parameters;

[0021] The step S70 further includes:

[0022] Based on the trained optical neural network, the metalens geometric parameters are confirmed.

[0023] As a further improvement of the present invention, the step of calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group includes:

[0024] Calculating the imaging of the light signal when it reaches the metalens through the aperture;

[0025] Performing FDTD calculation on the optical signal passing through the plurality of layers of the metalens;

[0026] The imaging of the light signal when it leaves the metalens and reaches the metasurface is calculated.

[0027] As a further improvement of the present invention, the hardware parameters include aperture parameters and COMS parameters;

[0028] The step S10 further includes:

[0029] Initialize the aperture parameters and the COMS parameters;

[0030] The step S70 further includes:

[0031] Based on the optical neural network that has completed training, at least one of the above-mentioned aperture parameters, COMS parameters, and the distances between adjacent devices such as the aperture, lens group, metasurface, and CMOS is confirmed.

[0032] As a further improvement of the present invention, the artificial neural network uses a deep residual network model as a keel model;

[0033] The step S40 includes:

[0034] Inputting the imaging data into the deep residual network model for feature extraction, and outputting a predicted probability value through a fully connected layer;

[0035] The highest probability is selected as the classification result.

[0036] As a further improvement of the present invention, step S50 includes:

[0037] The classification result is compared with the actual value corresponding to the image data, and the loss value after comparison is calculated using binary cross entropy or mean square error as the loss function.

[0038] As a further improvement of the present invention, the trainable parameters of the optical neural network include amplitude and phase determined according to the hardware parameters, the trainable parameters of the artificial neural network include weights and biases, the image data also includes a verification set for verification, and step S60 further includes:

[0039] Performing gradient updates on the amplitude and phase, the weight, and the bias through an optimization algorithm, and obtaining a new classification result through steps S30 and S40 based on the updated optical neural network and the artificial neural network, and then calculating a new loss value through step S50 until the loss value converges to a stable level, and recording the parameters of each network during this process;

[0040] Verifying the accuracy and loss rate of each network using the verification set, and confirming the optimal network as the optical neural network and the artificial neural network that have completed training;

[0041] The step S70 further includes:

[0042] Based on the amplitude and phase of the trained optical neural network, the metasurface geometric parameters are reversely deduced.

[0043] As a further improvement of the present invention, the hypersurface geometric parameters include the shape and period of the primitives, as well as the size parameters and rotation angle parameters corresponding to the shape;

[0044] The design method further includes the steps of: coordinating optical system design and process iteration on the metasurface geometric parameters to obtain structural parameters of the optical neural network.

[0045] To achieve one of the aforementioned objectives, an embodiment of the present invention provides a design device for an optical system based on metasurface image encryption, wherein the optical system includes an optical neural network and an artificial neural network, wherein the optical neural network includes several layers of the metasurface, and the design device includes:

[0046] An initialization module, configured to initialize hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of primitives on several layers of the hypersurface;

[0047] An acquisition module, used to acquire image data for training;

[0048] an optical neural network calculation module, configured to calculate imaging data obtained after the image data passes through the optical neural network;

[0049] An artificial neural network classification module, configured to obtain a classification result by applying an artificial neural network to the imaging data;

[0050] a loss function calculation module, configured to determine a loss value through a loss function according to the classification result and an actual value corresponding to the image data;

[0051] A model training module, configured to simultaneously train and update the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value;

[0052] A parameter generation module is used to confirm the model parameters of the artificial neural network that has completed training, and to confirm the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the metasurface geometric parameters.

[0053] To achieve one of the above-mentioned objectives, an embodiment of the present invention provides an electronic device, including:

[0054] a storage module storing a computer program;

[0055] The processing module can implement the steps in the above-mentioned method for designing an optical system based on metasurface image encryption when executing the computer program.

[0056] In order to achieve one of the above-mentioned purposes of the invention, one embodiment of the present invention provides a readable storage medium, which stores a computer program. When the computer program is executed by a processing module, it can implement the steps in the above-mentioned method for designing an optical system based on metasurface image encryption.

[0057] Compared with the existing technology, the present invention has the following beneficial effects: the design method proposes an end-to-end optimization implementation scheme. On the one hand, the encrypted image can be directly obtained by hardware. The image finally obtained after the optical signal passes through the metasurface is not a clear and directly recognizable image. The encrypted image can be directly stored on the disk. The encryption is undertaken by the hardware metasurface. The encryption process is an optical process with fast speed and low energy consumption, and there is no need to store the original unencrypted data on the disk, thus avoiding information leakage from the source. On the other hand, a jointly optimized optical neural network and artificial neural network are generated through an end-to-end method. The optical neural network can be better identified or classified by the artificial neural network, and the image recognition or classification effect is better than that of a simple artificial neural network model, which not only meets the public's demand for privacy protection of image data, but also meets the needs of image users for image acquisition purposes. In addition, the artificial neural network has a faster computing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] 1 is a flow chart of a method for designing an optical system according to an embodiment of the present invention;

[0059] FIG2 is a schematic diagram of the structure of an optical neural network according to an embodiment of the present invention;

[0060] FIG3 is a schematic structural diagram of a metasurface or a metalens according to an embodiment of the present invention;

[0061] FIG4 is a schematic diagram of a partial structure of one layer of a metasurface or a metalens according to an embodiment of the present invention;

[0062] FIG5 is a schematic diagram of modules of a design device for an optical system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0064] An embodiment of the present invention provides a method and device for designing an optical system based on metasurface image encryption, which has good encryption effect and good image recognition or classification effect.

[0065] The optical system of this embodiment includes an optical neural network and an artificial neural network, wherein the optical neural network includes several layers of the metasurface. The imaging of the light signal after passing through the metasurface cannot obtain a clear image, which is equivalent to encryption preprocessing before imaging. As a result, it is difficult to store a clear image when storing on a disk. Therefore, the optical system can achieve an encryption effect at the hardware level.

[0066] In addition, the programmable characteristics of several layers of the metasurface form a structure similar to a neural network, which can also undertake part of the image feature extraction task, further accelerating the calculation speed of subsequent artificial neural networks.

[0067] The design method for an optical system based on metasurface image encryption in this embodiment encrypts image information through a metasurface and simultaneously trains the trainable parameters of both the optical neural network and the artificial neural network through an end-to-end approach. By continuously updating and optimizing the models of the optical neural network and the artificial neural network, and controlling the hardware parameters of the optical neural network as well as the weights and biases of the artificial neural network, the method ultimately achieves the goal of judgment and classification, namely, better decrypting image information through the artificial neural network. Therefore, this design method meets the requirements of both image encryption and decryption, and the entire design method pursues global optimization.

[0068] The optical system can be optimized for specific goals. For example, it can be an optical system mainly used for face recognition, taking pictures of passers-by in public places and matching target objects in the crowd. In this process, images are acquired through an optical neural network and analyzed by an artificial neural network, i.e., face recognition. The optimization direction is that relatively blurry images for human eye recognition can be classified relatively accurately by the artificial neural network. Other scenarios can also be classification of men and women, human fatigue status, human and vehicle classification, license plate number recognition, etc. Different scenarios can have different more optimized parameters. The following steps are further explained using face recognition as an example.

[0069] The following three embodiments illustrate a design method for an optical system based on metasurface image encryption provided by one embodiment of the present invention. The main difference between the three embodiments is that in Example 1, the optical neural network is primarily designed and trained for the parameters of the metasurface. In Example 2, the lens of the optical neural network is configured as a metalens. In addition to training the parameters of the metasurface in the optical neural network in Example 1, the parameters of the metalens of the optical neural network are also trained. Example 3 can be based on either Example 1 or Example 2 and also trains the parameters of the aperture and / or CMOS.

[0070] Example 1

[0071] The following, in conjunction with Figures 1 to 4, illustrates a method for designing an optical system based on metasurface image encryption, provided by one embodiment of the present invention. Although this application provides the method operation steps shown in the following embodiments or flowcharts, based on conventional or no creative work, the execution order of the steps in the method that are not logically causally related is not limited to the execution order provided in the embodiments of this application. For example, the acquisition order of steps S10 and S20 below can be adjusted arbitrarily, without distinguishing the order in time.

[0072] A design method of an optical system based on metasurface image encryption in this embodiment, as shown in FIG1 , includes the following steps S10 to S70:

[0073] Step S10: Initialize the hardware parameters of the optical neural network, wherein the hardware parameters include the hypersurface geometric parameters of several layers of primitives on the hypersurface.

[0074] Figure 2 shows the schematic structure of an optical neural network, which includes an aperture, a lens assembly, a metasurface, and a complementary metal oxide semiconductor (CMOS). The aperture collects light field information, the lens assembly creates an image, the metasurface controls light propagation, and the CMOS performs photoelectric conversion and records the distribution of light intensity. The aperture, lens assembly, several layers of metasurface, and CMOS are arranged in this order.

[0075] The structure of the metasurface can be shown in Figure 3. The metasurface can have a multi-layer structure. The value of n in the figure can be 2, 3, 4, 5 or even more. There are many primitives to be trained on the metasurface of each layer, as shown in Figure 4. The multi-layer metasurface forms a structure similar to a neural network, which greatly improves its expression ability and enables the optical neural network to have classification function. In other words, the programmable metasurface structure plays an encryption effect on the one hand, and assists in data classification operations on the other hand.

[0076] Other apertures, lenses, and CMOS can be used off-the-shelf or can be deeply customized by participating in the training of the model through the following Examples 2 and 3, which will be explained in detail below.

[0077] The primitives can have shapes such as lines, cylinders, cuboids, elliptical cylinders, hollowed-out elliptical bodies, and hollowed-out cuboids. These different shapes can correspond to some identical or different dimensional parameters and rotation angles. Some of these parameters may be identical, such as the period and height, while others may differ, such as the radius of a cylinder, the length and width of a cuboid, or the major and minor diameters of an elliptical cylinder. Different primitives can also have different rotation angles. These primitive parameters collectively constitute the metasurface's geometric parameters. The effects of these different primitives on the light field can be precisely numerically calculated using algorithms such as FDTD.

[0078] The initialization process can be to generate a random matrix, which is the assignment of hardware parameters of the optical neural network. In this embodiment 1, if the trainable parameters in the optical neural network are mainly for the metasurface, the random matrix can be a parameter simulation of each primitive of the metasurface structure.

[0079] Step S20: Acquire image data for training.

[0080] In combination with the above, the design method of the optical system of this embodiment can optimize the model more specifically for some specific scenes to make the classification result more accurate. In this embodiment, face recognition is taken as an example, and the acquired image data is face data.

[0081] The step S20 may include steps S21 to S22:

[0082] Step S21: Unify the size of the facial data images, for example, upsample or downsample images of different sizes to a uniform size.

[0083] Step S22: Processing the color portrait into a gray image. This can make the image data more consistent and meet the classification requirements of different scenes in different weather conditions and different times of the day, such as day and night.

[0084] In subsequent steps, use this uniformly sized grayscale image for model training, verification, testing, etc.

[0085] Step S30: Calculate imaging data obtained after the image data passes through the optical neural network.

[0086] The step S30 may include steps S31 to S32:

[0087] Step S31: calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group;

[0088] Step S32: performing FDTD calculation on the optical signal passing through the plurality of layers of the metasurface;

[0089] Step S33: Calculate the imaging data obtained when the light signal leaves the metasurface and enters the COMS.

[0090] When calculating the propagation of light signals in the far field, that is, when the order of magnitude of the propagation distance is much greater than the order of magnitude of the wavelength of light, and when the light signal propagates between optical devices, the Rayleigh-Sommerfield formula or the Green's function vector diffraction formula can be used. When calculating the propagation of light signals in the near field, that is, when the order of magnitude of the propagation distance is roughly equivalent to the order of magnitude of the wavelength of light, and when the light signal propagates within an optical device, such as between several layers of a metasurface, the FDTD method can be used. This can simulate and calculate the imaging data when the image data reaches the COMS through the optical neural network.

[0091] Step S40: The imaging data is subjected to an artificial neural network to obtain a classification result.

[0092] The artificial neural network uses a deep residual network model (ResNet, Residual Neural Network) as the backbone model. ResNet constructs a deep network by introducing residual connections, which enables the network to learn and optimize more effectively. In this embodiment, more specifically, ResNet18 is used, a relatively shallow ResNet model, which consists of a convolutional layer, a pooling layer and a fully connected layer. Its main parts are the convolutional layer and the residual block, and the fully connected layer is located at the end of the network. The convolutional layer and the residual block are used to extract image features and gradually reduce the size of the feature map, and finally perform classification or regression tasks through the fully connected layer. The relatively shallow structure of ResNet18 makes it easier to train when computing resources are limited, and it performs well on the image classification task of the design method of this embodiment.

[0093] The step S40 further includes:

[0094] Inputting the imaging data into the deep residual network model for feature extraction, and outputting a predicted probability value through a fully connected layer;

[0095] The highest probability is selected as the classification result.

[0096] Taking the ResNet18 above as an example, the fully connected layer in the last few layers of ResNet18 plays the role of converting high-dimensional features into categories or regression outputs. It receives the feature vectors of the previous layers as input, performs linear transformation through a series of weight matrices and bias terms, and then applies the activation function to generate the final output. The activation function can be softmax, sigmoid or tanh, etc. The fully connected layer introduces nonlinearity in the convolutional neural network and maps high-dimensional features to specific task outputs, such as the face recognition task in this embodiment, to obtain multiple classification results and the corresponding probability of each classification. Finally, the highest probability is used as the output category.

[0097] Step S50: confirming a loss value through a loss function according to the classification result and the actual value corresponding to the image data.

[0098] Specifically, the classification result may be compared with the actual value corresponding to the image data, and the loss value after comparison may be calculated by using binary cross entropy or mean square error as a loss function.

[0099] The loss function can be reasonably weighed and adjusted based on specific task requirements, data characteristics, and model design to more effectively guide the trainable parameters during training. Here, the loss function can be a weighted sum of the losses in different models, such as optical neural networks and artificial neural networks, as the loss function for the entire network.

[0100] Step S60: looping the above steps S30 to S50, and simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value.

[0101] In this step, the loss value from step S50 can be back-propagated back to the network, and then the gradients of all parameters involved in the calculation in the optical neural network and the artificial neural network can be calculated using the chain rule. In this embodiment 1, the trainable parameters of the optical neural network include the amplitude and phase determined based on the hardware parameters, and the trainable parameters of the artificial neural network include weights and biases. The trainable parameters of the optical neural network and the artificial neural network can be gradient updated using a model training optimizer, which can use AdamW.

[0102] The step S60 specifically further includes:

[0103] Step S61: Through the optimization algorithm, the amplitude and phase, the weight and the deviation are gradient updated, and based on the updated optical neural network and the artificial neural network, a new classification result is obtained through the steps S30 and S40, and then a new loss value is calculated through step S50 until the loss value converges to a stable level, and the parameters of each network are recorded in the process.

[0104] Step S61 is used to update the model. The performance of the model is monitored and optimized using two important evaluation metrics: loss and accuracy. The model is updated until these two metrics converge to a stable level. During the training process, the model file after each round of optimization can be saved.

[0105] The step S60 further includes:

[0106] Step S62: Verify the accuracy and loss rate corresponding to each network through the verification set, and confirm the optimal network as the optical neural network and the artificial neural network that have completed training.

[0107] Step S62 is used to select the optimal model and verify the model through cross-validation. The validation set is used to select and tune the model's hyperparameters. By observing indicators such as loss and accuracy on the validation set, appropriate hyperparameters such as learning rate and regularization parameters can be selected, or the optimal model version can be determined. In addition, part of the image data can be set as a test set, for example, 70-80% of the samples are used for training, 10-15% for verification, and 10-15% for testing. The specific ratio can be adjusted according to the size of the face recognition and dataset.

[0108] Through the above steps S61 and S62, the model will update its parameters based on the data of the training set, and then evaluate the performance of the model on the validation set.

[0109] Step S70: confirming the model parameters of the artificial neural network that has completed training, and confirming the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the metasurface geometric parameters.

[0110] In this embodiment, the metasurface geometric parameters are reversely derived based on the amplitude and phase of the trained optical neural network. In the following embodiments 2 and 3, other hardware parameters can also be reversely derived.

[0111] The metasurface geometric parameters include the shape and period of the primitives, as well as the size parameters and rotation angle parameters corresponding to the shape. The design method further includes the step of: coordinating the metasurface geometric parameters with the optical system design and process iteration to obtain the structural parameters of the optical neural network.

[0112] Specifically, the structural parameters of the optical neural network can be a photomask data format file (GDS, Graphical Data System), which is a file format used to describe and record the layout of the metasurface, including information such as the physical layout and characteristic parameters of the metasurface, which serves as the basis and guidance for subsequent process manufacturing. Then, the corresponding optical neural network hardware structure can be produced. For example, the process department can create a metasurface lens that matches the parameters of the metasurface of Example 1.

[0113] After applying the above method to obtain the trained optical neural network and artificial neural network, the hardware structure of the optical neural network and the matching software are obtained. In actual use, by inputting new samples, the corresponding probability can be output to achieve the classification goal.

[0114] Example 2

[0115] The difference between Example 2 and Example 1 is that the lens group may include several layers of metalenses, and the hardware parameters include the metalens geometric parameters of the primitives on the several layers of the metalenses.

[0116] The metalens can achieve an even thinner effect. The structure of the metalens can be shown in FIG3 , which has a multi-layer structure. Each metalens layer is also provided with numerous primitives. The specific structure of the primitives can be found in the description of the primitives of the metasurface above. The metalens of this embodiment can process both monochromatic light, such as 940 nm wavelength, and multicolor light, such as the three central wavelengths of RGB, and can also optimize the dispersion effect.

[0117] Based on Example 1, the steps of the design method of Example 2 can be adaptively adjusted as follows:

[0118] The step S10 further includes:

[0119] Initialize the metalens geometric parameters.

[0120] The specific method of initialization can refer to the method in Example 1, which uses a random matrix to generate corresponding parameters. These parameters can be continuously updated and adjusted in subsequent model training and optimization.

[0121] Step S31 in Example 2 includes:

[0122] Step S311: calculating the imaging of the light signal when it reaches the metalens through the aperture;

[0123] Step S312: performing FDTD calculation on the optical signals passing through the plurality of layers of the metalens;

[0124] Step S313: Calculate the imaging of the light signal when it leaves the metalens and reaches the metasurface.

[0125] Since the optical signal propagates between several layers of metalenses, it belongs to the near-field propagation described above. Therefore, the Rayleigh-Sommerfield formula or the Green's function vector diffraction formula is also used for calculation in step S311, the FDTD calculation is used in step S312, and the Rayleigh-Sommerfield formula or the Green's function vector diffraction formula is continued to be used for calculation in step S313.

[0126] In step S60, the metalens geometric parameters are also trained as part of the optical neural network. Such a more comprehensive end-to-end design is more conducive to the subsequent decryption of the artificial neural network.

[0127] The step S70 further includes:

[0128] Based on the trained optical neural network, the metalens geometric parameters are confirmed.

[0129] This embodiment 2 not only obtains the geometric parameters of the metasurface in embodiment 1, but also obtains the geometric parameters of the metalens through an end-to-end design method.

[0130] Finally, the process department creates a metalens that matches the training parameters of Example 2.

[0131] Example 3

[0132] In Example 3, the hardware parameters include aperture parameters, COMS parameters, and at least one of the parameters such as the distance between adjacent devices such as the aperture, lens group, metasurface, and CMOS. Preferably, the aperture parameters, COMS parameters, the distance between adjacent devices such as the aperture, lens group, metasurface, and CMOS can all be used as part of the model training.

[0133] Example 3 can be implemented on the basis of Example 1 or on the basis of Example 2. Taking the implementation based on Example 2 as an example, not only the parameters of the super-lens and super-surface of the optical neural network are designed end-to-end, but also the aperture parameters, COMS parameters, and the distances between adjacent devices such as the aperture, lens group, super-surface, and CMOS are designed, further realizing the pursuit of global optimization of the entire design method.

[0134] Based on Example 2, the steps of the design method of Example 3 can be adaptively adjusted as follows: Step S10 further includes:

[0135] Initialize the aperture parameters and the COMS parameters.

[0136] The initialization here can still be obtained by assigning random values.

[0137] In step S60, the aperture parameters, COMS parameters, and the distances between the components of the optical neural network, that is, the parameters of the various optical components of the entire optical neural network are all part of the training.

[0138] The step S70 further includes:

[0139] Based on the optical neural network that has completed training, the aperture parameters and the COMS parameters are confirmed.

[0140] Finally, the process department creates or purchases the aperture and CMOS that match the aperture parameters and COMS parameters.

[0141] Compared with the prior art, this embodiment has the following beneficial effects:

[0142] On the one hand, the optical system designed by this design method can directly obtain an encrypted image. The image finally obtained after the light signal passes through the metasurface is not a clear and directly recognizable image. The encrypted image can be directly stored on the disk, with fast encryption and no need to store the original unencrypted data on the disk, thus avoiding information leakage from the source. On the other hand, a matching optical neural network and artificial neural network are generated through an end-to-end method. The artificial neural network can better identify or classify the optical neural network, and the image recognition or classification effect is better than that of a simple artificial neural network model, which not only meets the public's demand for privacy protection of image data, but also meets the needs of image users for the purpose of image acquisition.

[0143] In one embodiment, a design device for an optical system based on metasurface image encryption is provided, as shown in FIG5 . The design device for an optical system based on metasurface image encryption includes the following modules, and the specific functions of each module are as follows:

[0144] An initialization module, configured to initialize hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of primitives on several layers of the hypersurface;

[0145] An acquisition module, used to acquire image data for training;

[0146] an optical neural network calculation module, configured to calculate imaging data obtained after the image data passes through the optical neural network;

[0147] An artificial neural network classification module, configured to obtain a classification result by applying an artificial neural network to the imaging data;

[0148] a loss function calculation module, configured to determine a loss value through a loss function according to the classification result and an actual value corresponding to the image data;

[0149] A model training module, configured to simultaneously train and update the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value;

[0150] A parameter generation module is used to confirm the model parameters of the artificial neural network that has completed training, and to confirm the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the metasurface geometric parameters.

[0151] It should be noted that for details not disclosed in the design device of the optical system according to the embodiment of the present invention, please refer to the details disclosed in the design method of the optical system according to the embodiment of the present invention.

[0152] The optical system design apparatus may also include a computing device such as a computer, laptop, PDA, or cloud server, as well as a processing module, a storage module, and a computer program stored in the storage module and executable on the processing module, such as the aforementioned optical system design method program. When the processing module executes the computer program, the steps of each of the aforementioned optical system design method embodiments are implemented, such as the steps shown in FIG1 .

[0153] In addition, the present invention also proposes an electronic device, which includes a storage module and a processing module. When the processing module executes the computer program, it can implement the steps in the above-mentioned optical system design method, that is, implement the steps in any one of the technical solutions in the above-mentioned optical system design method.

[0154] The electronic device may be a part of a design device integrated into the optical system, or a local terminal device, or a part of a cloud server.

[0155] The processing module can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays (FPGAs), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processing module is the control center of the optical system design device, connecting the various components of the entire optical system design device using various interfaces and circuits.

[0156] The storage module can be used to store the computer programs and / or modules. The processing module implements the various functions of the optical system design device by running or executing the computer programs and / or modules stored in the storage module and accessing the data stored in the storage module. The storage module may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for the function, etc. In addition, the storage module may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0157] For example, the computer program can be divided into one or more modules / units, which are stored in a storage module and executed by a processing module to implement the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the optical system design device.

[0158] Furthermore, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by a processing module, can implement the steps in the above-mentioned method for designing an optical system, that is, implement the steps in any one of the technical solutions in the above-mentioned method for designing an optical system.

[0159] If the integrated module of the optical system design method is implemented as a software functional unit and sold or used as a standalone product, it can be stored on a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the process steps of the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored on a computer-readable storage medium. When executed by the processing module, the computer program can implement the steps of each of the above-mentioned method embodiments.

[0160] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium.

[0161] It should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation method can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0162] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A design method for an optical system based on metasurface image encryption, characterized in that: The optical system includes an optical neural network and an artificial neural network, the optical neural network includes several layers of the metasurface, and the design method includes the following steps: Step S10: Initializing the hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of several layers of primitives on the hypersurface; Step S20: Acquire image data for training; Step S30: calculating imaging data obtained after the image data passes through the optical neural network; Step S40: obtaining a classification result by applying an artificial neural network to the imaging data; Step S50: confirming a loss value through a loss function according to the classification result and the actual value corresponding to the image data; Step S60: looping the above steps S30 to S50, and simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value; Step S70: confirming the model parameters of the artificial neural network that has completed training, and confirming the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

2. The method for designing an optical system according to claim 1, characterized in that: The optical neural network further includes an aperture, a lens group and a COMS, wherein the aperture, the lens group, a plurality of layers of the metasurface and the COMS are arranged in sequence; the step S30 further includes: Calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group; Performing FDTD calculation on the optical signal passing through the plurality of layers of the metasurface; The imaging data obtained when the optical signal leaves the metasurface and enters the COMS is calculated.

3. The method for designing an optical system according to claim 2, characterized in that: The lens group includes several layers of metalenses, and the hardware parameters include metalens geometric parameters of primitives on several layers of the metalenses; The step S10 further includes: Initializing the metalens geometric parameters; The step S70 further includes: Based on the optical neural network that has completed the training, the geometric parameters of the metalens are confirmed.

4. The method for designing an optical system according to claim 3, characterized in that: The step of calculating the imaging of the light signal when it reaches the metasurface through the aperture and the lens group comprises: Calculating the imaging of the light signal when it reaches the superlens through the aperture; Performing FDTD calculation on the optical signal passing through several layers of the superlens; The imaging of the optical signal when it leaves the metalens and reaches the metasurface is calculated.

5. The method for designing an optical system according to claim 2, characterized in that: The hardware parameters include aperture parameters and COMS parameters; The step S10 further includes: Initialize at least one of the aperture parameters, the CMOS parameters, and the distances between adjacent devices such as the aperture, the lens group, the metasurface, and the CMOS; The step S70 further includes: Based on the optical neural network that has completed training, at least one of the above-mentioned aperture parameters, CMOS parameters, and the distances between adjacent devices such as the aperture, lens group, metasurface, and CMOS is confirmed.

6. The method for designing an optical system according to claim 1, characterized in that: The artificial neural network uses a deep residual network model as a keel model; The step S40 comprises: Inputting the imaging data into the deep residual network model for feature extraction, and outputting a predicted probability value through a fully connected layer; The highest probability is selected as the classification result.

7. The method for designing an optical system according to claim 1, characterized in that: The step S50 comprises: The classification result is compared with the actual value corresponding to the image data, and the loss value after comparison is calculated by using binary cross entropy or mean square error as the loss function.

8. The method for designing an optical system according to claim 1, characterized in that: The trainable parameters of the optical neural network include amplitude and phase determined according to the hardware parameters, the trainable parameters of the artificial neural network include weights and biases, the image data also includes a verification set for verification, and the step S60 further includes: By means of an optimization algorithm, the amplitude and phase, the weight and the deviation are gradient updated, and based on the updated optical neural network and the artificial neural network, a new classification result is obtained through steps S30 and S40, and a new loss value is calculated through step S50 until the loss value converges to a stable level, and the parameters of each network are recorded in the process; Verify the accuracy and loss rate of each network through the verification set, and confirm the optimal network as the optical neural network and the artificial neural network that have completed training; The step S70 further includes: Based on the amplitude and phase of the trained optical neural network, the metasurface geometric parameters are reversely deduced.

9. The method for designing an optical system according to claim 1, characterized in that: The hypersurface geometric parameters include the shape and period of the primitive, and the size parameters and rotation angle parameters corresponding to the shape; The design method also includes the steps of: coordinating optical system design and process iteration with the metasurface geometric parameters to obtain the structural parameters of the optical neural network.

10. A design device for an optical system based on metasurface image encryption, characterized in that: The optical system includes an optical neural network and an artificial neural network, the optical neural network includes several layers of the metasurface, and the design device includes: An initialization module, used to initialize the hardware parameters of the optical neural network, wherein the hardware parameters include hypersurface geometric parameters of primitives on several layers of the hypersurface; An acquisition module, used for acquiring image data for training; An optical neural network calculation module, used to calculate imaging data obtained after the image data passes through the optical neural network; An artificial neural network classification module, used for obtaining a classification result by applying an artificial neural network to the imaging data; A loss function calculation module, used to determine a loss value through a loss function according to the classification result and an actual value corresponding to the image data; A model training module, used for simultaneously training and updating the trainable parameters of the optical neural network and the artificial neural network by optimizing the loss value; A parameter generation module is used to confirm the model parameters of the artificial neural network that has completed training, and to confirm the hardware parameters based on the optical neural network that has completed training, wherein the hardware parameters include the hypersurface geometric parameters.

11. An electronic device, characterized in that: include: A storage module storing a computer program; The processing module can implement the steps in the method for designing an optical system based on metasurface image encryption as described in claim 1 when executing the computer program.

12. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processing module, the steps in the method for designing an optical system based on metasurface image encryption described in claim 1 can be implemented.

Images