Target model training and metasurface structure design method and device

Abstract

The invention provides a target model training and metasurface structure design method and device. The target model training method comprises the following steps: generating a metasurface structure, carrying out binarization coding on the metasurface structure to obtain a first structure distribution matrix, generating a spectral response corresponding to the first structure distribution matrix, and forming sample data by using the first structure distribution matrix and the spectral response; adding the sample data into the data set, and in response to determining that the number of the sample data included in the data set reaches a first threshold value, training the initial deep learning model by using the data set to obtain a target model, the target model being used for generating a fourth structure distribution matrix corresponding to the to-be-processed spectral response, the fourth structure distribution matrix is used for determining the metasurface structure of the metasurface element to be prepared, and the metasurface element is applied to metasurface color routing. By applying the scheme of the invention, the complexity and the like of the metasurface color routing design can be reduced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the fields of artificial intelligence and optical imaging, in particular to the technologies of deep learning, computer vision, sensor technology, nanotechnology and material science, and more particularly to a target model training and metasurface structure design method and device. BACKGROUND

[0002] With the wide application of complementary metal oxide semiconductor (CMOS) image sensors in different scenarios, accurate acquisition and processing of color information has become one of the key factors affecting the imaging quality. The traditional color routing method mainly relies on color filter array and microlens structure to realize, but the optical efficiency of this method is low, and there is a crosstalk problem, and it is limited by the material dispersion characteristics, and it is difficult to meet the imaging requirements of high resolution and high color fidelity. SUMMARY

[0003] The present disclosure provides a target model training and metasurface structure design method and device.

[0004] A target model training method comprises: generating a metasurface structure, and binarizing and encoding the metasurface structure to obtain a first structure distribution matrix, generating a spectral response corresponding to the first structure distribution matrix, and using the first structure distribution matrix and the spectral response to form sample data; adding the sample data to a data set, and in response to determining that the number of sample data included in the data set reaches a first threshold, training an initial deep learning model using the data set to obtain the target model, the target model being used to generate a fourth structure distribution matrix corresponding to a to-be-processed spectral response, the fourth structure distribution matrix being used to determine a metasurface structure of a to-be-prepared metasurface element, and the metasurface element being applied in metasurface color routing.

[0005] A metasurface structure design method comprises: obtaining a to-be-processed spectral response; inputting the spectral response into a target model to obtain an output fourth structure distribution matrix, the target model being trained according to the above target model training method; determining a metasurface structure of a to-be-prepared metasurface element according to the fourth structure distribution matrix, and the metasurface element being applied in metasurface color routing.

[0006] A target model training device comprises a sample construction unit and a model training unit. The sample construction unit is configured to generate a metasurface structure, perform binary coding on the metasurface structure, obtain a first structure distribution matrix, generate a spectral response corresponding to the first structure distribution matrix, and use the first structure distribution matrix and the spectral response to form sample data. The model training unit is configured to add the sample data to a data set, and in response to determining that the number of sample data included in the data set reaches a first threshold, train an initial deep learning model using the data set to obtain the target model, the target model being configured to generate a fourth structure distribution matrix corresponding to a to-be-processed spectral response, the fourth structure distribution matrix being configured to determine a metasurface structure of a to-be-prepared metasurface element, and the metasurface element being applied to metasurface color routing.

[0007] A metasurface structure design apparatus includes a first obtaining unit, a second obtaining unit, and a structure determining unit. The first obtaining unit is configured to obtain a to-be-processed spectral response. The second obtaining unit is configured to input the spectral response into a target model to obtain an output fourth structure distribution matrix, the target model being trained according to the target model training method. The structure determining unit is configured to determine a metasurface structure of a to-be-prepared metasurface element according to the fourth structure distribution matrix, and the metasurface element being applied to metasurface color routing.

[0008] A metasurface color routing includes: A metasurface element, a metasurface structure of the metasurface element being determined according to the metasurface structure design method.

[0009] An electronic device includes: At least one processor; and A memory connected to the at least one processor in communication; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0010] A non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to perform the method as described above.

[0011] A computer program product includes computer programs / instructions, the computer programs / instructions being executed by a processor to implement the method as described above.

[0012] It is to be understood that the details set forth herein do not limit the scope of the embodiments of the present disclosure but merely constitute illustrative of typical embodiments. Other embodiments can readily be devised in light of the disclosure herein without departing from the scope and spirit of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings are included to provide a further understanding of the present application, and are incorporated in and constitute a part of this specification. In the drawings: Figure 1 A flowchart of the target model training method embodiment of the present disclosure; Figure 2 A schematic diagram of the composition structure of the deep learning model of the present disclosure; Figure 3 A flowchart of the metasurface structure design method embodiment of the present disclosure; Figure 4 A schematic diagram of the color routing of the present disclosure; Figure 5 A schematic diagram of the form of the metasurface structure of the present disclosure; Figure 6 A schematic diagram of the overall implementation process of the metasurface structure design method of the present disclosure; Figure 7 A schematic diagram of the composition structure of the target model training device embodiment 700 of the present disclosure; Figure 8 A schematic diagram of the composition structure of the metasurface structure design device embodiment 800 of the present disclosure; Figure 9 A schematic block diagram of an electronic device 900 that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in the understanding thereof. These should be considered in a descriptive sense only and not limiting. Therefore, various changes and modifications can be made to the embodiments described herein by those of ordinary skill in the art without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted herein.

[0015] In addition, it should be understood that the term "and / or" herein merely describes an associated relationship between associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0016] Figure 1 A flowchart of the target model training method embodiment of the present disclosure. As Figure 1The embodiments are shown below.

[0017] In step 101, a metasurface structure is generated, and the metasurface structure is binarized to obtain a first structure distribution matrix. A spectral response corresponding to the first structure distribution matrix is generated, and the first structure distribution matrix and the spectral response are used to form sample data.

[0018] In step 102, the sample data is added to a data set. In response to determining that the number of sample data included in the data set reaches a first threshold, an initial deep learning model is trained using the data set to obtain a target model. The target model is used to generate a fourth structure distribution matrix corresponding to a to-be-processed spectral response. The fourth structure distribution matrix is used to determine a metasurface structure of a to-be-prepared metasurface element. The metasurface element is applied to metasurface color routing.

[0019] In recent years, metasurfaces, as a new type of two-dimensional artificial electromagnetic material, provide a new solution for color routing. Through the design of subwavelength artificial atomic structures, metasurfaces can flexibly control the phase, amplitude, and polarization of light wavefronts, thereby achieving efficient color splitting and routing at the nanoscale. Compared with traditional color routing methods, metasurface color routing has a more compact structure, higher optical efficiency, and lower crosstalk, and can effectively improve the color resolution and signal-to-noise ratio of image sensors.

[0020] Currently, the design methods of metasurface color routing mainly include analytical design methods based on forward phase (such as geometric phase, propagation phase, etc.) arrangement and inverse design methods based on optimization algorithms (such as topological optimization, genetic algorithm, etc.). However, these methods generally have high implementation complexity.

[0021] The above method embodiments can be used to train the target model using the constructed sample data. For a to-be-processed spectral response, the trained target model can be used to generate a corresponding structure distribution matrix, and the metasurface structure of the to-be-prepared metasurface element can be determined according to the generated structure distribution matrix. Accordingly, the corresponding metasurface element can be prepared. The metasurface element can be applied to metasurface color routing. That is, the method of designing metasurface color routing based on deep learning is proposed in the scheme of the present disclosure. Once the target model is trained, the target model can be used to generate a corresponding structure distribution matrix for any spectral response. Accordingly, the corresponding metasurface element can be prepared according to the structure distribution matrix. The entire process is simple and convenient to implement. The powerful reasoning ability of the target model improves the accuracy of the obtained structure distribution matrix, and thus improves the quality of the prepared metasurface element.

[0022] To train the target model, a sufficient number of sample data needs to be constructed first. For example, a metasurface structure can be generated, and the metasurface structure can be binarized to obtain a first structure distribution matrix. Then, a spectral response corresponding to the first structure distribution matrix can be generated, and the first structure distribution matrix and the spectral response can be used to form a sample data.

[0023] In some embodiments of the present disclosure, a first structure distribution matrix of N*N size can be generated, where N is a positive integer greater than 1. For any element in the first structure distribution matrix, if it is determined that a nano unit is arranged at the position corresponding to the element, the value of the element can be set to 1. If it is determined that no nano unit is arranged at the position corresponding to the element, the value of the element can be set to 0.

[0024] The specific value of N can be determined according to actual needs, such as 20. In addition, the nano unit can refer to a titanium dioxide matrix column (also known as a titanium dioxide nanocolumn) with a height of 600 nm and a size of 100 nm*100 nm. It is a dielectric material with high refractive index and low loss in the visible light band, and is an ideal choice for high-performance metasurfaces.

[0025] In actual applications, a metasurface structure can be randomly created, and the metasurface structure can be binarized to obtain a first structure distribution matrix corresponding to the metasurface structure. The metasurface structure can include N*N arrangement positions. If a nano unit is arranged at a position, the value of the element corresponding to the position in the first structure distribution matrix can be set to 1. Otherwise, the value of the element corresponding to the position in the first structure distribution matrix can be set to 0. In this way, a first structure distribution matrix composed of 0 and 1 can be obtained. In addition, to simplify processing, the arrangement of nano units in the metasurface structure can adopt a symmetrical manner.

[0026] Through the above processing, a complex nano structure distribution can be converted into a standard binary matrix, thereby simplifying the data representation form, reducing the complexity of data processing, and facilitating subsequent deep learning model processing.

[0027] For the generated first structure distribution matrix, its corresponding spectral response can also be generated. In some embodiments of the present disclosure, the spectral response corresponding to the first structure distribution matrix under normal incidence can be generated by a rigorous coupled wave analysis (RCWA, Rigorous Coupled-Wave Analysis) method, and a genetic algorithm can be used to constrain the distribution of the spectral response.

[0028] RCWA is a rigorous numerical method for calculating the electromagnetic scattering behavior of periodic structures (such as gratings, metasurfaces), which is based on the principle of expanding the electromagnetic field and the periodic dielectric constant in the Fourier space into a series of coupled waves, and accurately calculating the diffraction efficiency, transmission and reflection spectrum and other optical responses by solving the eigenvalue problem. In the scheme described in the disclosure, the RCWA method can be used to quickly and accurately generate the spectral response corresponding to the first structure distribution matrix under normal incidence. In addition, the genetic algorithm can refer to some relatively simple genetic algorithms to improve processing efficiency. The use of genetic algorithms to constrain the distribution of spectral responses can improve the quality of the obtained spectral responses and avoid introducing too many invalid samples for subsequent model training.

[0029] Using the first structure distribution matrix and the corresponding spectral response, a sample data can be formed, and the sample data can be added to the dataset. When the number of sample data included in the dataset reaches a first threshold, the initial deep learning model can be trained according to the dataset. The specific value of the first threshold can be determined according to actual needs.

[0030] In some embodiments of the disclosure, before training the initial deep learning model using the dataset, the dataset can be divided into M subsets, M is a positive integer greater than 1, and the M subsets include at least a training subset and a test subset. Accordingly, the training samples in the training subset can be used to train the deep learning model, and the test samples in the test subset can be used to verify the trained deep learning model. In response to determining that the verification is passed, the trained deep learning model can be determined as the required target model.

[0031] The specific value of M can be determined according to actual needs, such as 2 or 3. If M is 3, in addition to the training subset and the test subset, the M subsets can also include a validation subset. For example, the sample data in the dataset can be divided into the training subset, the validation subset and the test subset according to a ratio of 8:1:1. A sample data belongs to only one subset, such as a training sample belonging to the training subset, a validation sample belonging to the validation subset, and a test sample belonging to the test subset. The training subset is used to train the deep learning model, the validation subset is used to monitor the performance of the deep learning model during the training process to prevent overfitting and adjust hyperparameters such as learning rate, and the test subset is used to verify the deep learning model after the training is completed, that is, to evaluate the performance of the model to judge the generalization ability and effectiveness of the model.

[0032] In some embodiments of the present disclosure, when training the deep learning model using the training samples in the training subset, preset training parameters can be obtained, which can include epoch, batch size, initial learning rate, etc., and the deep learning model can be iteratively trained using an exponential decay learning rate method and an Adam (Adaptive Moment Estimation) optimizer according to the training parameters.

[0033] For example, the epoch can be 1000, the batch size can be 64, and the initial learning rate can be 0.0001. The exponential decay learning rate method refers to using a larger learning rate, such as 0.0001, in the early stage of training to quickly converge, and then gradually reducing the learning rate according to an exponential law as the epoch increases, which is equivalent to using a smaller "step" to perform fine search around the optimal solution in the later stage of training. The Adam optimizer is an advanced optimization algorithm that combines the momentum method and the adaptive learning rate mechanism. It calculates and adjusts the dedicated and dynamic learning rate for each parameter of the model, thereby achieving more efficient and stable navigation in a complex parameter space. Accordingly, the combination of the exponential decay learning rate method and the Adam optimizer can balance the training speed and convergence quality, thereby obtaining a stably converged deep learning model.

[0034] In some embodiments of the present disclosure, when training the deep learning model using the training samples in the training subset, the spectral response in the training samples can be input into the deep learning model to obtain an output second structure distribution matrix. Then, the first structure distribution matrix and the second structure distribution matrix in the training samples can be used to determine a loss function, and the deep learning model can be updated according to the loss function.

[0035] Specifically, in some embodiments of the present disclosure, the second structure distribution matrix can have a size of N*N, and each element in the matrix has a probability value between 0 and 1. Accordingly, the manner of determining the loss function according to the first structure distribution matrix and the second structure distribution matrix in the training sample can include: for an element i in the same position in the first structure distribution matrix and the second structure distribution matrix, 1≤i≤N*N, the following processing is performed respectively: obtaining the value of the element i in the first structure distribution matrix as a first value, and obtaining the value of the element i in the second structure distribution matrix as a second value; obtaining a first difference value between 1 and the first value and a second difference value between 1 and the second value; obtaining a first product of the logarithmic operation result of the first value and the second value, and obtaining a second product of the logarithmic operation result of the first difference value and the second difference value; obtaining a sum of the first product and the second product to obtain an evaluation result corresponding to the element i; obtaining a sum of the evaluation results corresponding to all elements, and obtaining a quotient of the sum and N*N, and determining the negative of the quotient as the required loss function.

[0036] That is, a cross-entropy loss function similar to binary classification can be used, and the loss function can be expressed as: ; (1) wherein, represents the value of the element i in the first structure distribution matrix, that is, the first value, represents the value of the element i in the second structure distribution matrix, that is, the second value, represents the obtained loss function.

[0037] Through the above loss function, the difference between the second structure distribution matrix output by the model and the first structure distribution matrix in the training sample can be accurately measured, thereby providing a clear gradient direction for model parameter updating and improving the training efficiency of the model.

[0038] In some embodiments of the present disclosure, the deep learning model can include a feature extraction module, a feature dimension reduction and enhancement module, and a feature fusion module. The feature extraction module can include L sub-modules connected in sequence, L being a positive integer greater than 1. Each sub-module can include a convolutional layer and a batch normalization layer. The feature extraction module can be used for feature extraction and feature stabilization processing of the input spectral response. The feature dimension reduction and enhancement module can be used for dimension reduction and key feature enhancement processing of the output result of the feature extraction module. The feature fusion module can be used for feature fusion of the output result of the feature dimension reduction and enhancement module, and can generate a second structure distribution matrix according to the feature fusion result.

[0039] Figure 2 The figure is a schematic diagram of the composition structure of the deep learning model of the present disclosure. As shown in the figure, Figure 2As shown, assuming that the value of L is 3, three sub-modules connected in sequence can be included in the feature extraction module, and each sub-module can include a convolutional layer and a batch normalization layer. The convolutional layer can use a 3*3 convolutional kernel to extract the features of the spectral response data and enhance the fitting. Each convolutional layer is equipped with a batch normalization layer to speed up the model convergence, reduce the internal covariate shift, enhance the stability, and the like. The feature dimension reduction and strengthening module can use a down-sampling layer to reduce the dimension of the extracted features and can strengthen the key features to improve the robustness. The feature fusion module can use a fully connected layer for feature fusion, and can adjust the dimension according to the feature fusion result and generate a second structure distribution matrix according to the probability value, and the like.

[0040] After the training of the deep learning model is completed, the test samples in the test subset can also be used to verify the trained deep learning model. In some embodiments of the present disclosure, the spectral response in the test sample can be input into the trained deep learning model to obtain an output third structure distribution matrix. Then, the third structure distribution matrix can be used to generate a spectral response by the RCWA method, and the generated spectral response can be compared with the spectral response input into the trained deep learning model. According to the comparison result, it can be determined whether the verification is passed.

[0041] For example, if it is determined that more than a predetermined percentage of test samples in the test subset meet the following requirements: after the spectral response in the test sample is input into the trained deep learning model, the difference between the generated spectral response and the input spectral response is less than a third threshold value, then it can be determined that the verification is passed. The specific values of the predetermined percentage and the third threshold value can be determined according to actual needs.

[0042] By using the above processing method, the generalization ability and design accuracy of the model can be objectively evaluated through the test subset independent of the training subset and the closed electromagnetic simulation verification loop, thereby providing a solid theoretical foundation for subsequent practical applications.

[0043] If the verification is passed, the target model can be applied to the actual metasurface structure design. If the verification is not passed, subsequent processing is not limited, for example, the target model can be further optimized and trained.

[0044] Correspondingly, Figure 3 The flowchart of the metasurface structure design method embodiment of the present disclosure is shown in FIG. 3. Figure 3 As shown, the following specific implementation modes are included.

[0045] In step 301, the spectral response to be processed is obtained.

[0046] In step 302, the spectral response is input into the target model to obtain an output fourth structure distribution matrix.

[0047] In step 303, the metasurface structure of the metasurface element to be prepared is determined according to the fourth structure distribution matrix, and the metasurface element is used in metasurface color routing.

[0048] The target model can be based on... Figure 1 The target model is trained using the method shown.

[0049] Metasurface color routing is mainly used to focus and separate wide-band incident light into different pixel units according to a specified wavelength. Figure 4 This is a schematic diagram of the color routing described in this disclosure, such as... Figure 4 As shown, within the visible light range, red light, green light, and blue light can be separated according to the pixel arrangement of red (R), green (G), and blue (B), ultimately forming individual pixel units to transmit the specified wavelength signal to the image sensor.

[0050] The spectral response to be processed can refer to a specified, desired target spectral response. This spectral response can be input into a pre-trained target model to obtain the fourth structure distribution matrix output by the target model. Furthermore, the metasurface structure of the metasurface element to be fabricated can be determined based on the fourth structure distribution matrix. For example... Figure 5 As shown, Figure 5 This is a schematic diagram of the metasurface structure described in this disclosure.

[0051] In addition, in some embodiments of this disclosure, when determining the metasurface structure of the metasurface element to be prepared based on the fourth structure distribution matrix, the spectral response corresponding to the fourth structure distribution matrix can be generated first using the RCWA method. The generated spectral response can be compared with the spectral response of the input target model. If the difference between the two is less than a second threshold based on the comparison result, the metasurface structure of the metasurface element can be determined based on the fourth structure distribution matrix. The specific value of the second threshold can be determined according to actual needs.

[0052] The feasibility of the metasurface structure designed in the target model can be verified through electromagnetic simulation. If so, metasurface components can be fabricated based on the metasurface structure. Furthermore, with... Figure 5 Taking the metasurface structure shown as an example, experiments show that within a bandwidth range of 400~700nm, the transmittance of red, green and blue light all exceeded a certain level, while the transmittance of other wavelengths of light was significantly reduced, achieving the expected color routing effect.

[0053] Based on the above introduction, Figure 6 This is a schematic diagram illustrating the overall implementation process of the metasurface structure design method described in this disclosure. Figure 6As shown, the spectral response to be processed can be obtained first, and then the spectral response can be input into the target model to obtain the output fourth structure distribution matrix. Through the fourth structure distribution matrix, the metasurface structure of the metasurface element to be prepared can be determined. Then, the electromagnetic simulation of the metasurface structure can be performed by the RCWA method, such as generating the spectral response corresponding to the fourth structure distribution matrix. The generated spectral response can be compared with the spectral response of the input target model. If the difference between the two is less than a second threshold, the corresponding metasurface element can be prepared according to the metasurface structure.

[0054] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure. Furthermore, for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0055] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0056] Figure 7 This is a schematic diagram of the structural composition of Embodiment 700 of the target model training device described in this disclosure. Figure 7 As shown, it includes: a sample construction unit 701 and a model training unit 702.

[0057] The sample construction unit 701 is used to generate a metasurface structure, perform binarization encoding on the metasurface structure to obtain a first structure distribution matrix, generate the spectral response corresponding to the first structure distribution matrix, and use the first structure distribution matrix and the spectral response to form sample data.

[0058] The model training unit 702 is used to add sample data to the dataset. In response to the determination that the number of sample data included in the dataset reaches a first threshold, the initial deep learning model is trained using the dataset to obtain the target model. The target model is used to generate the fourth structure distribution matrix corresponding to the spectral response to be processed. The fourth structure distribution matrix is ​​used to determine the metasurface structure of the metasurface element to be prepared. The metasurface element is applied to the metasurface color routing.

[0059] In some embodiments of this disclosure, the sample construction unit 701 can generate a first structure distribution matrix of size N*N, where N is a positive integer greater than 1. For any element in the first structure distribution matrix, in response to determining that nano-units are arranged at the position corresponding to the element, the value of the element can be set to 1, and in response to determining that no nano-units are arranged at the position corresponding to the element, the value of the element can be set to 0.

[0060] For the generated first structure distribution matrix, the sample construction unit 701 can also generate its corresponding spectral response. In some embodiments of this disclosure, the sample construction unit 701 can generate the spectral response corresponding to the first structure distribution matrix under normal incidence using the RCWA method, and can use a genetic algorithm to constrain the distribution of the spectral response.

[0061] Using the first structural distribution matrix and the corresponding spectral response, a sample data can be formed. The model training unit 702 can add the sample data to the dataset. When the number of sample data included in the dataset reaches the first threshold, the initial deep learning model can be trained based on the dataset.

[0062] In some embodiments of this disclosure, before training the initial deep learning model using the dataset, the model training unit 702 may first divide the dataset into M subsets, where M is a positive integer greater than 1. The M subsets include at least a training subset and a test subset. Accordingly, the deep learning model can be trained using the training samples in the training subset, and the trained deep learning model can be verified using the test samples in the test subset. In response to determining that the verification is successful, the trained deep learning model can be determined as the desired target model.

[0063] In some embodiments of this disclosure, when the model training unit 702 trains the deep learning model using training samples in the training subset, it can obtain pre-set training parameters, which may include: training period, batch sample size, and initial learning rate, etc., and can iteratively train the deep learning model using the exponential decay learning rate method and the Adam optimizer according to the training parameters.

[0064] In some embodiments of this disclosure, when the model training unit 702 trains the deep learning model using training samples in the training subset, it can also input the spectral response in the training samples into the deep learning model to obtain the output second structure distribution matrix. Then, the loss function can be determined based on the first structure distribution matrix and the second structure distribution matrix in the training samples, and the parameters of the deep learning model can be updated based on the loss function.

[0065] Specifically, in some embodiments of this disclosure, the second structure distribution matrix may be of size N*N, where each element takes a probability value between 0 and 1. Correspondingly, the model training unit 702 may determine the loss function based on the first and second structure distribution matrices in the training samples by performing the following processing on elements i at the same position in the first and second structure distribution matrices, where 1 ≤ i ≤ N*N: obtaining the value of element i in the first structure distribution matrix as the first value, obtaining the value of element i in the second structure distribution matrix as the second value; obtaining the first difference between 1 and the first value and the second difference between 1 and the second value; obtaining the first product of the logarithmic operation results of the first value and the second value, and obtaining the second product of the logarithmic operation results of the first difference and the second difference; obtaining the sum of the first product and the second product to obtain the evaluation result corresponding to element i; obtaining the sum of the evaluation results corresponding to each element, and obtaining the quotient of the sum and N*N, and determining the negative number of the quotient as the required loss function.

[0066] In some embodiments of this disclosure, the deep learning model may include: a feature extraction module, a feature dimensionality reduction and enhancement module, and a feature fusion module. The feature extraction module may include L sequentially connected sub-modules, where L is a positive integer greater than 1. Each sub-module may include: a convolutional layer and a batch normalization layer. The feature extraction module can be used to extract features and stabilize features from the input spectral response. The feature dimensionality reduction and enhancement module can be used to reduce the dimensionality of the output of the feature extraction module and enhance key features. The feature fusion module can be used to fuse features from the output of the feature dimensionality reduction and enhancement module and can generate a second structure distribution matrix based on the feature fusion result.

[0067] After training the deep learning model, the trained deep learning model can be validated using test samples from a test subset. In some embodiments of this disclosure, the model training unit 702 can input the spectral response from the test samples into the trained deep learning model to obtain the output third structure distribution matrix. Then, the spectral response corresponding to the third structure distribution matrix can be generated using the RCWA method. The generated spectral response can be compared with the spectral response input into the trained deep learning model, and the validation result can be used to determine whether the validation has passed.

[0068] Figure 8 This is a schematic diagram of the structural composition of Embodiment 800 of the metasurface structure design device described in this disclosure. Figure 8 As shown, it includes: a first acquisition unit 801, a second acquisition unit 802, and a structure determination unit 803.

[0069] The first acquisition unit 801 is used to acquire the spectral response to be processed.

[0070] The second acquisition unit 802 is used to input the spectral response into the target model to obtain the output fourth structure distribution matrix.

[0071] The structure determination unit 803 is used to determine the metasurface structure of the metasurface element to be prepared based on the fourth structure distribution matrix. The metasurface element is used in metasurface color routing.

[0072] In some embodiments of this disclosure, when the structure determination unit 803 determines the metasurface structure of the metasurface element to be prepared based on the fourth structure distribution matrix, it can first generate the spectral response corresponding to the fourth structure distribution matrix using the RCWA method. The generated spectral response can be compared with the spectral response of the input target model. If the difference between the two is less than a second threshold based on the comparison result, the metasurface structure of the metasurface element can be determined based on the fourth structure distribution matrix. The specific value of the second threshold can be determined according to actual needs.

[0073] The present disclosure also discloses a metasurface color routing scheme, which may include: a metasurface element, wherein the metasurface structure of the metasurface element may be according to... Figure 3 The method shown is used to determine this.

[0074] The specific workflow of each of the above device embodiments can be found in the relevant descriptions in the foregoing method embodiments, and will not be repeated here.

[0075] The solutions described in this disclosure can be applied to the field of artificial intelligence, particularly involving technologies such as deep learning, computer vision, sensor technology, nanotechnology, and materials science. Artificial intelligence is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. Artificial intelligence hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0076] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0077] Figure 9A schematic block diagram of an electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0078] like Figure 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. The RAM 903 may also store various programs and data required for the operation of the electronic device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0079] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0080] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as those described in this disclosure. For example, in some embodiments, the methods described in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the methods described in this disclosure can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the methods described herein by any other suitable means (e.g., by means of firmware).

[0081] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0082] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0083] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0084] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0085] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0086] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0087] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0088] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A target model training method, characterized in that, The method comprises: generating a metasurface structure, and performing binary coding on the metasurface structure to obtain a first structure distribution matrix, generating a spectral response corresponding to the first structure distribution matrix, and using the first structure distribution matrix and the spectral response to form sample data; adding the sample data to a data set, and in response to determining that the number of sample data included in the data set reaches a first threshold, training an initial deep learning model using the data set to obtain a target model, the target model being used to generate a fourth structure distribution matrix corresponding to a to-be-processed spectral response, the fourth structure distribution matrix being used to determine a metasurface structure of a to-be-prepared metasurface element, and the metasurface element being applied to a metasurface color routing.

2. The method of claim 1, wherein, The binary coding on the metasurface structure to obtain the first structure distribution matrix comprises: generating the first structure distribution matrix of N*N size, N being a positive integer greater than 1; wherein, for any element in the first structure distribution matrix, in response to determining that a nano unit is arranged at a position corresponding to the element, the value of the element is set to 1, and in response to determining that the nano unit is not arranged at the position corresponding to the element, the value of the element is set to 0.

3. The method of claim 1, wherein, The generation of the spectral response corresponding to the first structure distribution matrix comprises: generating the spectral response corresponding to the first structure distribution matrix under normal incidence by a rigorous coupled wave analysis method, and using a genetic algorithm to constrain the distribution of the spectral response.

4. The method of claim 2, wherein: before the training of the initial deep learning model using the data set, the method further comprises: dividing the data set into M subsets, M being a positive integer greater than 1, the M subsets at least including a training subset and a test subset; the training of the initial deep learning model using the data set to obtain a target model comprises: training the deep learning model using training samples in the training subset, and verifying the trained deep learning model using test samples in the test subset, and in response to determining that the verification is passed, the trained deep learning model is determined as the target model.

5. The method of claim 4, wherein, The training of the deep learning model using the training samples in the training subset comprises: inputting the spectral response in the training sample into the deep learning model to obtain an output second structure distribution matrix; determining a loss function according to the first structure distribution matrix in the training sample and the second structure distribution matrix; updating parameters of the deep learning model according to the loss function.

6. The method of claim 5, wherein: the second structure distribution matrix is of N*N size, and each element in the second structure distribution matrix has a probability value between 0 and 1; the determination of the loss function according to the first structure distribution matrix in the training sample and the second structure distribution matrix comprises: For elements i in the same position in the first and second structure distribution matrices, 1≤i≤N*N, the following processing is performed: obtaining the value of the element i in the first structure distribution matrix as a first value, obtaining the value of the element i in the second structure distribution matrix as a second value; obtaining a first difference between 1 and the first value and a second difference between 1 and the second value; obtaining a first product of the logarithmic operation result of the first value and the second value, and obtaining a second product of the logarithmic operation result of the first difference and the second difference; obtaining a sum of the first product and the second product to obtain an evaluation result corresponding to the element i; Obtaining the sum of the evaluation results of each element, and obtaining the quotient of the sum and N*N, and determining the negative of the quotient as the loss function.

7. The method of claim 4, wherein, The training of the deep learning model using the training samples in the training subset includes: Obtaining pre-set training parameters, the training parameters including: training period, batch sample number and initial learning rate; According to the training parameters, the exponential decay learning rate method and the adaptive moment estimation optimizer are used to iteratively train the deep learning model.

8. The method of claim 5, wherein the deep learning model comprises a feature extraction module, a feature dimension reduction and enhancement module, and a feature fusion module; the feature extraction module comprises L sub-modules connected in sequence, L being a positive integer greater than 1, and each sub-module comprising a convolutional layer and a batch normalization layer; The feature extraction module is configured to perform feature extraction and feature stabilization processing on the input spectral response, the feature dimension reduction and enhancement module is configured to perform dimension reduction and key feature enhancement processing on the output result of the feature extraction module, and the feature fusion module is configured to perform feature fusion on the output result of the feature dimension reduction and enhancement module and generate the second structure distribution matrix according to the feature fusion result. The verification of the trained deep learning model using the test samples in the test subset includes:

9. The method of claim 4, wherein, Inputting the spectral response in the test sample into the trained deep learning model to obtain a third structure distribution matrix as output; Generating a spectral response corresponding to the third structure distribution matrix by a rigorous coupled-wave analysis method, comparing the generated spectral response with the input spectral response, and determining whether the verification passes according to the comparison result. It includes:

10. A metasurface structure design method, characterized in that, Obtaining a spectral response to be processed; Inputting the spectral response into a target model to obtain a fourth structure distribution matrix as output, the target model being trained according to any one of the methods of claims 1-9; Determining a metasurface structure of a metasurface element to be prepared according to the fourth structure distribution matrix, the metasurface element being applied to a metasurface color routing. The determination of the metasurface structure of the metasurface element to be prepared according to the fourth structure distribution matrix includes:

11. The method of claim 10, wherein, ​ generate a spectral response corresponding to the fourth structure distribution matrix through a rigorous coupled-wave analysis method, and compare the generated spectral response with the input spectral response; in response to determining, according to a comparison result, that a difference between the two is less than a second threshold value, determine the metasurface structure according to the fourth structure distribution matrix. 12.A target model training apparatus, characterized by comprising: comprising: a sample construction unit and a model training unit; the sample construction unit is configured to generate a metasurface structure, binarize the metasurface structure to obtain a first structure distribution matrix, generate a spectral response corresponding to the first structure distribution matrix, and use the first structure distribution matrix and the spectral response to form sample data; the model training unit is configured to add the sample data to a data set, and in response to determining that a number of sample data included in the data set reaches a first threshold value, train an initial deep learning model using the data set to obtain the target model, the target model being configured to generate a fourth structure distribution matrix corresponding to a to-be-processed spectral response, the fourth structure distribution matrix being configured to determine a metasurface structure of a to-be-prepared metasurface element, the metasurface element being applied to metasurface color routing.

13. A metasurface structure design apparatus, characterized by, comprising: a first acquisition unit, a second acquisition unit, and a structure determination unit; the first acquisition unit is configured to acquire a to-be-processed spectral response; the second acquisition unit is configured to input the spectral response into a target model to obtain an output fourth structure distribution matrix, the target model being trained according to any one of the methods of claims 1-9; the structure determination unit is configured to determine a metasurface structure of a to-be-prepared metasurface element according to the fourth structure distribution matrix, the metasurface element being applied to metasurface color routing.

14. A metasurface color router, characterized in that, comprising: a metasurface element, a metasurface structure of the metasurface element being determined according to the method of claim 10.

15. An electronic device, comprising: comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

16. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method of any one of claims 1-11.

17. A computer program product, characterised in that, comprising computer programs / instructions, which, when executed by a processor, implement the method of any one of claims 1-11.

Images