Metasurface reverse design method based on CNN-Transform hybrid architecture
Through the hybrid architecture of CNN and Transformer, combined with local feature extraction and global dependency modeling, the problems of low efficiency and high computational cost in traditional methods are solved, and efficient and accurate prediction of metasurface structure parameters is achieved.
Patent Information
- Application Number
- CN202510776848.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-11
- Publication Date
- 2025-09-19
AI Technical Summary
Traditional metasurface inverse design methods based on optimization algorithms are inefficient, prone to falling into local optimal traps, and have high computational costs, making it difficult to achieve efficient and accurate parameter prediction from target electromagnetic response to physical structure.
By adopting a hybrid architecture of convolutional neural network (CNN) and Transformer, combining the local feature extraction capability of CNN with the global dependency modeling capability of Transformer, and performing reverse design through data preparation and hybrid architecture model, efficient and accurate prediction from electromagnetic response to metasurface structure parameters can be achieved.
It achieves more accurate and efficient prediction of metasurface structure parameters, improves design efficiency and model generalization ability, avoids local optimal traps, and reduces computational costs.
Smart Images
Figure CN120671531A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a method for reverse designing a hypersurface by combining a convolutional neural network (CNN) with a Transformer. Background Art
[0002] As an artificial electromagnetic material, metasurfaces are composed of subwavelength-sized unit structures arranged periodically or non-periodically on a two-dimensional plane. Due to their unprecedented flexibility in controlling the amplitude, phase, polarization and other characteristics of electromagnetic waves, they have shown great application potential in imaging, sensing, communications and other fields in recent years. To overcome the bottleneck of forward design, metasurface inverse design has emerged. Its core goal is to directly deduce the physical structure that can achieve this function based on the preset target electromagnetic response. However, this inverse problem is often unstable, that is, a small change in the structural parameters may cause a drastic change in the electromagnetic response, which makes the search for the global optimal solution extremely difficult.
[0003] Traditional inverse design methods based on optimization algorithms generally suffer from significant drawbacks, such as low efficiency, susceptibility to local optimality, and high computational cost. Therefore, the research community urgently needs a more powerful and efficient design paradigm. In this context, deep learning demonstrates tremendous potential. Unlike traditional methods that rely on iterative search and explicit optimization objectives, deep learning models can automatically construct complex, highly nonlinear implicit mappings from target responses to physical structures by learning from large amounts of data. Once the model is trained, it can achieve extremely fast inference and prediction.
[0004] In the early exploration of deep learning applications for metasurface reverse design, models such as multi-layer perceptrons (MLPs) and recurrent neural networks (RNNs / LSTMs) achieved some progress. However, these models have inherent limitations in fully exploiting the information in input-response sequences. MLPs typically treat each data point in the input-response sequence as an independent feature, and their fully connected structures lack mechanisms for effectively capturing local features in the sequence. While RNNs / LSTMs can handle sequential dependencies, they may struggle to identify complex, long-spanning feature patterns in the response spectrum. In contrast, the core advantage of convolutional neural networks (CNNs), particularly one-dimensional CNNs, lies in their ability to efficiently extract local features and patterns in the input-response sequence using sliding convolution kernels. This makes CNNs well-suited for identifying and understanding fine structures and detailed variations in the target response. Conversely, the Transformer architecture, with its self-attention mechanism, demonstrates superior ability to capture global features. This self-attention mechanism allows the model to directly compute the interdependencies between any two positions in the sequence, effectively capturing long-range dependencies and the overall response spectrum. This is crucial for understanding target functions that require global coordination. Summary of the Invention
[0005] The purpose of the present invention is to address the shortcomings of the existing technology and provide a metasurface inverse design method with a hybrid architecture of convolutional neural network (CNN) and Transformer, aiming to combine the powerful local feature extraction capability of CNN and the excellent global dependency modeling capability of Transformer model to achieve more accurate and efficient prediction of metasurface structure parameters.
[0006] To achieve the above objectives, the present invention adopts the following technical solution, which includes two main stages: data preparation and reverse engineering using a hybrid architecture model:
[0007] Phase 1: Data Preparation: This phase aims to build a high-quality dataset for training and evaluating the subsequent CNN-Transformer hybrid model. Specifically, it includes the following steps:
[0008] Step A1: Define the structural parameter space. Clearly describe the key adjustable parameters (such as geometric dimensions and material properties) required to describe the metasurface unit structure and their reasonable physical value ranges. This defines the design possibility boundary.
[0009] Step A2: Generate data through electromagnetic simulation. Within the defined parameter space, systematically sample multiple combinations of structural parameters. Using electromagnetic simulation software (e.g., CST), calculate the electromagnetic response characteristics (e.g., S-parameters, absorption spectra, etc.) corresponding to each parameter set, generating a large number of paired "structural parameter-electromagnetic response" data sets.
[0010] Step A3: Data preprocessing: Standardize the simulation data, including data cleaning (removing outliers) and numerical normalization / standardization (to facilitate model training). The processed dataset is then divided into training, validation, and test sets.
[0011] Phase 2: After preparing the dataset, reverse engineering is performed using the hybrid architecture model. The specific process is as follows:
[0012] Step B1: The electromagnetic response curve data is used as the input of the model, and the input data is upsampled using a transposed convolutional network (TransposedCNN). The purpose of this step can be to enhance the resolution of subsequent feature extraction and adjust the data dimension to match the requirements of the subsequent network structure.
[0013] Step B2: Input the data processed in step B1 into the CNN module and use the CNN convolution kernel sliding operation to perform downsampling to efficiently extract local features, fine structures and detail change information in the target response curve.
[0014] Step B3: Divide the feature map extracted by CNN into a series of sequence units, namely Tokens, in the spatial dimension and add auxiliary information.
[0015] Step B4: Input the token sequence with auxiliary information into the Transformer encoder, and use its core self-attention mechanism to calculate the mutual dependencies between tokens in the sequence, thereby effectively capturing the long-range dependencies and global context information in the response spectrum.
[0016] Step B5: Input the data processed by the Transformer encoder into an MLP to predict and output the hypersurface unit structure parameters. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 This is a flow chart for implementing the data preparation method proposed in the present invention.
[0018] Figure 2 This is a flowchart for the reverse design implementation of the CNN-Transformer hybrid architecture model proposed in this invention.
[0019] Figure 3 This is the network structure diagram of the CNN-Transformer hybrid architecture model proposed in this invention. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to specific examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not intended to limit the present invention.
[0021] This paper proposes a metasurface inverse design method based on a hybrid architecture of convolutional neural networks (CNNs) and transformers. The core idea of this method is to combine the advantages of CNNs in extracting local features of electromagnetic response curves with the ability of transformers to capture long-range dependencies and global contextual information in sequential data, thereby achieving efficient and accurate prediction from the target electromagnetic response to the metasurface structural parameters.
[0022] The implementation process of the present invention mainly includes two stages: stage one: data preparation and stage two: reverse design based on the CNN-Transformer hybrid architecture model.
[0023] Phase 1: Data Preparation: This phase aims to build a high-quality, representative dataset to lay a solid foundation for the effective training and reliable evaluation of the subsequent CNN-Transformer hybrid architecture model.
[0024] Step A1: Define the structural parameter space. First, it is necessary to clarify the basic structure of the designed metasurface unit, such as an open resonant ring. Secondly, determine the key adjustable geometric parameters (such as length L, width W, height H, inner diameter R_in, outer diameter R_out, etc.) and material properties (such as dielectric constant ε) that affect its electromagnetic response. Then, set a reasonable physical value range for each adjustable parameter (for example, the range of parameter H can be set to [1mm, 1.5mm], and the range of W can be set to [1.6mm, 2.4mm]). This step defines an N-dimensional structural parameter space, where N is the number of adjustable parameters.
[0025] Step A2: Generate data through electromagnetic simulation. Systematic sampling is performed within the N-dimensional parameter space defined in step A1 to generate multiple sets of structural parameter combinations. The sampling strategy uses a fixed step size plus permutations to generate all parameter combinations to ensure good coverage of the parameter space. For each set of sampled structural parameter combinations, full-wave simulation calculations are performed using the professional electromagnetic simulation software CST Microwave Studio. The simulation settings require specifying the incident wave conditions and boundary conditions, and recording the electromagnetic response characteristic curves corresponding to each set of parameters. Ultimately, a data set containing K sets of paired data is formed. The size of K usually needs to be in the thousands to hundreds of thousands to ensure the generalization ability of the model.
[0026] Step A3: Data preprocessing. Data cleaning involves checking simulation results and removing data points due to simulation errors, non-convergence, or other anomalies. Data standardization involves processing electromagnetic response curves and linearly scaling them to the range [0, 1] or [-1, 1]. This improves model training stability and convergence speed. Dataset partitioning involves randomly dividing the preprocessed dataset D into training, validation, and test sets. Typical partitioning ratios are 70% / 15% / 15% or 80% / 10% / 10%.
[0027] Phase 2: Reverse design based on the CNN-Transformer hybrid architecture model. In this phase, the CNN-Transformer hybrid architecture model is trained using the prepared dataset to predict the corresponding structural parameters based on the input target electromagnetic response.
[0028] Step B1: Input processing and upsampling. The model receives a vector of length M of target electromagnetic response curve data as input and uses a transposed convolutional network (CNN) to upsample the input response data. This increases the feature dimension of the data and adjusts the data dimension to match the input requirements of the subsequent CNN module.
[0029] Step B2: Local feature extraction. The data processed in step B1 is fed into a CNN module, which consists of two convolutional layers. The stride of the convolution operation is set to half the kernel size (stride = kernel_size / 2), achieving a 50% overlap between adjacent convolution operations. This overlapping convolution helps capture features more meticulously and ensures continuity in feature extraction.
[0030] Step B3: Tokenization, introduction of prediction tags and position encoding. In this step, the feature map output by CNN is first divided into M' feature tokens along the sequence dimension. They serve as carriers of local features of the response curve and provide rich contextual information for the model, but do not directly participate in the final parameter output. Subsequently, N learnable exclusive prediction tags are introduced that correspond one-to-one to the N hypersurface structure parameters to be predicted. These prediction tags are spliced with feature tokens to form a unified input sequence, and precise position encoding is added to each token in the sequence (including all feature tokens and all prediction tags) to give the model the ability to perceive the sequence order. Finally, a structured sequence with complete position awareness that integrates contextual feature information and N specified parameter decoding entries is constructed.
[0031] Step B4: Sequence Relationship Modeling (Transformer Encoder). Receive the complete sequence from step B3, which contains N prediction tokens and M' feature tokens, and has been injected with positional encoding, and input it into the Transformer encoder module for processing. The core function of this encoder is to accurately model the complex interdependencies between all tokens in the sequence, especially to effectively capture long-range dependency characteristics and global context information. The Transformer encoder consists of L encoder layers (Encoder Layer) with exactly the same structure stacked vertically. Each layer performs a complete representation refinement and transformation on the input sequence and passes the result to the next layer. Each encoder layer consists of two core components: the first is the Multi-Head Self-Attention (MHSA) mechanism, which dynamically calculates the strength of the association (attention weight) between any two tokens in the sequence and uses this as a basis for weighted aggregation of information to update the representation of each token; the second is the Feed-Forward Network (FFN), which is typically composed of two linear transformation layers and a nonlinear activation function in between. It performs independent, position-by-position deep nonlinear transformations on each token representation output by the MHSA, further enhancing the model's feature expression capabilities. To improve training stability and support deeper network structures, this architecture adopts a pre-layer normalization strategy. That is, at the input of each core component (MHSA and FFN), a layer normalization (LayerNorm) operation is first applied to normalize the input data, and then the normalized result is fed into the core component for calculation. Furthermore, the output of each core component is added to the component's original input (pre-normalization) via a residual connection. This design greatly promotes the efficient propagation of gradients in deep networks, contributing to the stability of the training process and improved model performance. After iterative processing by the L-layer encoder, the representation of each token in the model's output sequence is deeply integrated with global contextual information, paving the way for subsequent parameter prediction.
[0032] Step B5: Output prediction, model training and evaluation. First, obtain the complete sequence of the final output of the Transformer encoder (step B4), which contains the final hidden states of N prediction tags and M' feature Tokens. Next, input this complete sequence data containing all Tokens information into the prediction layer MLP. Inside the prediction layer, the system will accurately select the N final hidden state vectors corresponding to the N introduced prediction tags as the basis for subsequent calculations, while the representation of the feature Tokens is effectively ignored at this stage and does not directly participate in the prediction of the final parameter values. The MLP prediction head consists of a linear layer. Taking into account that the structural parameters of the hypersurface in this embodiment are designed to be non-negative values, the ReLU activation function is applied immediately after the linear layer to ensure that the output prediction value meets the physical constraints. Finally, these N scalar outputs processed by the ReLU activation function are combined to form the N-dimensional structural parameter vector p_pred predicted by the model. During model training, the mean squared error (MSE) is used as the loss function to quantify the difference between the predicted value p_pred and the true value p_true. The AdamW optimizer is used, combined with advanced training strategies such as exponential moving average (EMA), learning rate warmup, and cosine annealing, to achieve stable and efficient model parameter updates and improve generalization. After training, the model is evaluated on an independent test set, using metrics such as the MSE and MAE to assess the accuracy of parameter predictions.
Claims
1. A metasurface inverse design method based on a CNN-Transformer hybrid architecture, characterized by: The following steps are involved: Step A1: Define the structural parameter space. Clearly describe the key adjustable parameters required to describe the metasurface unit structure and their reasonable physical value ranges. Step A2: Generate data through electromagnetic simulation. Multiple combinations of structural parameters are sampled within the defined parameter space. Using electromagnetic simulation software, the corresponding electromagnetic response characteristics for each parameter set are calculated, generating a large number of paired "structural parameter-electromagnetic response" data sets. Step A3: Data preprocessing: Standardize the simulation data, including data cleaning and numerical normalization / standardization, and divide the processed data set into training set, validation set, and test set. Step B1: The electromagnetic response curve data is used as the input of the model, and the input data is upsampled using a transposed convolutional network. Step B2: Input the data processed in step B1 into the CNN module and perform downsampling using the CNN convolution kernel sliding operation. Step B3: Divide the feature map extracted by CNN into a series of sequence units, namely Tokens, in the spatial dimension and add auxiliary information. Step B4: Input the token sequence with auxiliary information into the Transformer encoder, and use its core self-attention mechanism to calculate the mutual dependencies between tokens in the sequence. Step B5: Input the data processed by the Transformer encoder into an MLP to predict and output the hypersurface unit structure parameters.