Wafer image super-resolution method and related apparatus

Abstract

The application discloses a wafer image super-resolution method and related equipment, and relates to the technical field of image processing. The wafer image super-resolution method comprises the following steps: obtaining an initial wafer image with a first resolution, inputting the initial wafer image into a preset super-resolution model, and obtaining a target wafer image with a second resolution, wherein the second resolution is higher than the first resolution, the super-resolution model is obtained by training a preset to-be-trained model based on extracted spatial features and frequency features, the spatial features comprise repeated structure features of a wafer image across regions, the frequency features comprise high-frequency defect features of the wafer image, and the extraction of the spatial features is based on a preset two-dimensional selection scanning strategy. The application accurately depicts long-range dependence of repeated circuits across regions of a wafer and focuses on high-frequency defect features of the wafer through feature extraction, and combines Hilbert curve scanning and linear scanning, so that the wafer detection resolution is improved, and the wafer detection efficiency is ensured.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to wafer image super-resolution methods and related equipment. Background Technology

[0002] As chip feature sizes continue to shrink, traditional bright-field microscopes are limited by the optical diffraction limit, making it difficult to accurately identify nanoscale defects, leading to an increased risk of missed detections and misjudgments. Therefore, in order to observe even smaller defects, it is often necessary to rely on high-magnification objectives.

[0003] However, using high-magnification objectives in wafer inspection significantly reduces the field of view, prolongs the scanning time of the entire wafer, and reduces inspection efficiency, making it difficult to improve wafer inspection resolution while ensuring wafer inspection efficiency. Summary of the Invention

[0004] The main objective of this application is to provide a wafer image super-resolution method and related equipment, which aims to solve the technical problem of simultaneously ensuring wafer inspection efficiency and inspection resolution.

[0005] To achieve the above objectives, this application proposes a wafer image super-resolution method, the method comprising:

[0006] Obtain an initial wafer image with a resolution of the first resolution;

[0007] The initial wafer image is input into a preset super-resolution model to obtain a target wafer image with a second resolution, wherein the second resolution is higher than the first resolution. The super-resolution model is obtained by training a preset training model based on spatial features and frequency features extracted from preset wafer sample images. The spatial features include repeating structural features across regions of the wafer image, and the frequency features include high-frequency defect features of the wafer image. The extraction of spatial features is based on a preset two-dimensional selective scanning strategy, which is obtained by fusing a preset Hilbert curve scan and a preset linear scan. The Hilbert scan is based on the spatial position of the wafer image.

[0008] In one embodiment, the model to be trained includes a shallow feature extraction module, a dual-domain transformation module, and a high-resolution reconstruction module. Before the step of inputting the initial wafer image into a preset super-resolution model to obtain a target wafer image of the second resolution, the method further includes:

[0009] A preset high-resolution wafer sample image is obtained, and the wafer sample image is downsampled a preset number of times to obtain a first resolution sample image corresponding to the wafer sample image.

[0010] The first resolution sample image is input into the shallow feature extraction module to obtain the shallow features of the first resolution sample image;

[0011] The shallow features are input into a preset dual-domain transformation module to obtain target deep features that fuse the spatial features and the frequency features;

[0012] The deep features are input into the high-resolution reconstruction module to obtain a wafer reconstruction image with improved resolution;

[0013] Based on the reconstructed wafer image and the wafer sample image, the parameters of the model to be trained are adjusted to obtain the super-resolution model.

[0014] In one embodiment, the dual-domain transformation module includes multiple cascaded spatial frequency groups, each spatial frequency group includes multiple spatial frequency sub-modules, and each spatial frequency sub-module includes a spatial feature extraction unit and a frequency feature extraction unit. The step of inputting the shallow features into the preset dual-domain transformation module to obtain target deep features that fuse the spatial features and the frequency features includes:

[0015] The shallow features are input into the spatial feature extraction unit, and the features to be processed input into the spatial feature extraction unit are further extracted to obtain the spatial features;

[0016] The spatial features are input into the frequency feature extraction unit of the same spatial frequency submodule to extract the frequency features from the spatial features, thereby obtaining the spatial frequency features.

[0017] Determine whether there are any unused spatial frequency submodules in the future;

[0018] If it exists, the frequency feature is input into the spatial feature extraction unit of the next spatial frequency submodule until there is no unused spatial frequency submodule, thus obtaining the target deep feature.

[0019] In one embodiment, the spatial feature extraction unit includes a visual state spatial subunit and a multi-scale feature extraction unit, and the step of further extracting features from the features to be processed input into the spatial feature extraction unit includes:

[0020] The features to be processed are normalized, and the normalized features are input into the visual state space sub-unit to extract the global features of the wafer image.

[0021] The global features are input into the multi-scale feature extraction unit, and the local features of the wafer image are supplemented into the global features to obtain the spatial features that fuse the global features and the local features.

[0022] In one embodiment, the step of extracting global features from the wafer image to obtain the global features includes:

[0023] The normalized features to be processed are processed successively based on the linear mapping layer and activation function layer of the state space group unit to obtain the first nonlinear feature;

[0024] The normalized feature to be processed is processed sequentially based on the linear mapping layer, the depthwise separable convolutional layer and the activation function layer of the state space group unit to obtain a second nonlinear feature. The first nonlinear feature and the second nonlinear feature are features obtained by processing the same feature to be processed through different processing methods.

[0025] The second nonlinear feature is input into the two-dimensional selection scan layer of the state space group unit. The nonlinear feature is sequentially scanned, state feature extracted, and two-dimensional mapped to obtain a serialized feature. The state feature extraction is based on a preset state space equation, and the computational complexity of the state space equation is linear.

[0026] The normalization operation is performed on the serialized features, and the normalized serialized features are multiplied element-wise with the first nonlinear feature to obtain the fused features;

[0027] The global feature is obtained by adding the feature to be processed and the fused feature element by element.

[0028] In one embodiment, the step of supplementing the global features with local features of the wafer image to obtain the spatial features that fuse the global features and the local features includes:

[0029] The global features are normalized, and then convolved with multiple convolutional layers of different kernel sizes to obtain multiple local features.

[0030] In the channel dimension of the local features, multiple local features are concatenated to obtain concatenated local features;

[0031] The spliced ​​local features are nonlinearly transformed by the activation function layer, and the nonlinearly transformed spliced ​​local features are convolved based on the preset fusion convolution layer to obtain multi-scale enhanced features;

[0032] The spatial features are obtained by adding the multi-scale enhanced features and the global features element by element.

[0033] In one embodiment, the nonlinear feature is a two-dimensional feature, and the step of sequentially performing serialization scanning, state feature extraction, and two-dimensional mapping on the nonlinear feature to obtain the serialized feature includes:

[0034] Based on the scanning order corresponding to the Hilbert scan, the nonlinear feature is scanned to obtain a one-dimensional Hilbert sequence, wherein the scanning order corresponding to the Hilbert scan is determined based on the spatial position of the nonlinear feature;

[0035] Based on the scanning order corresponding to the linear scan, the nonlinear feature is scanned to obtain a one-dimensional linear scan sequence, wherein the scanning order corresponding to the linear scan is a predetermined row order or column order;

[0036] Based on the state-space equation, features of the Hilbert sequence and the linear sequence are extracted respectively to obtain the Hilbert features corresponding to the Hilbert sequence and the linear features corresponding to the linear sequence;

[0037] Based on the spatial location, the element positions of the Hilbert feature are adjusted to obtain a Hilbert feature with the same spatial location as the linear feature;

[0038] The Hilbert feature and the linear feature with the adjusted element positions are added element by element to obtain the fused scan feature;

[0039] The fused scanning features are mapped in two dimensions to obtain the two-dimensional serialized features.

[0040] In one embodiment, the frequency features include low-frequency features and high-frequency features, and the step of extracting the frequency features from the spatial features to obtain spatial frequency features includes:

[0041] The normalization operation is performed on the spatial features, and the channel dimension of the normalized spatial features is reduced based on a preset channel transformation convolutional layer to obtain reduced spatial features.

[0042] The reduced spatial features are nonlinearly transformed by the activation function layer, and the nonlinearly transformed reduced spatial features are decomposed into low-frequency features and high-frequency features based on the preset discrete wavelet transform layer.

[0043] The low-frequency features and the high-frequency features are input into the visual state space sub-unit to obtain the low-frequency spatial features corresponding to the low-frequency features and the high-frequency spatial features corresponding to the high-frequency features.

[0044] Based on the inverse wavelet transform layer corresponding to the discrete wavelet transform layer, the low-frequency spatial features and the high-frequency spatial features are fused to obtain spatial frequency fusion features;

[0045] Based on the channel transformation convolutional layer and the activation function layer, the channel dimension of the spatial frequency fusion feature is expanded to obtain the spatial frequency feature.

[0046] In one embodiment, the reduced spatial features are two-dimensional features, and the high-frequency features include horizontal high-frequency features, vertical high-frequency features, and diagonal high-frequency features. The step of decomposing the nonlinearly transformed reduced spatial features into the low-frequency features and the high-frequency features based on a preset discrete wavelet transform layer includes:

[0047] Based on the discrete wavelet transform layer, in the horizontal direction, the reduced spatial features are low-pass filtered and downsampled to obtain the initial low-frequency features and the initial high-frequency features;

[0048] Based on the discrete wavelet transform layer, in the vertical direction, the initial low-frequency features are subjected to low-pass filtering and downsampling to obtain the low-frequency features and the vertical high-frequency features;

[0049] Based on the discrete wavelet transform layer, in the vertical direction, the initial high-frequency features are subjected to low-pass filtering and downsampling to obtain the horizontal high-frequency features and the diagonal high-frequency features.

[0050] In addition, to achieve the above objectives, this application also proposes a wafer image super-resolution device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the wafer image super-resolution method as described above.

[0051] One or more technical solutions proposed in this application have at least the following technical effects:

[0052] An initial wafer image with a first resolution is acquired. This initial wafer image is then input into a preset super-resolution model to obtain a target wafer image with a second resolution, where the second resolution is higher than the first resolution. The super-resolution model is trained on a preset training model based on spatial and frequency features extracted from preset wafer sample images. The spatial features include repeating structural features across regions of the wafer image, and the frequency features include high-frequency defect features of the wafer image. The extraction of spatial features is based on a preset two-dimensional selective scanning strategy, which is obtained by fusing a preset Hilbert curve scan and a preset linear scan. The Hilbert scan is based on the spatial position of the wafer image.

[0053] To address the issue that using high-magnification objectives in wafer inspection significantly reduces the field of view, prolongs the scanning time of the entire wafer, and decreases inspection efficiency, making it difficult to improve wafer inspection resolution while maintaining efficiency, this application improves the resolution of the initial wafer image through a pre-defined super-resolution model, obtaining a target wafer image with a second-resolution image. Because this application accurately characterizes the long-range dependence of repetitive circuits across wafer regions through spatial features during model training and focuses on high-frequency defect features of the wafer through frequency features, the wafer resolution can be effectively improved based on these features. Furthermore, during spatial feature extraction, this application uses Hilbert curve scanning, scanning based on the spatial position of the wafer image, combined with linear scanning. This ensures spatial continuity while avoiding redundant calculations in multi-directional, block-based scanning, reducing wafer scanning time and thus improving inspection efficiency. Therefore, this application can improve wafer inspection resolution while maintaining wafer inspection efficiency. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating an embodiment of the wafer image super-resolution method of this application.

[0057] Figure 2 This is a technical roadmap provided for Embodiment 1 of the wafer image super-resolution method of this application;

[0058] Figure 3 This is a schematic diagram of the network structure provided in Embodiment 1 of the wafer image super-resolution method of this application;

[0059] Figure 4 This is a flowchart illustrating Embodiment 2 of the wafer image super-resolution method of this application;

[0060] Figure 5 This is a comparative schematic diagram of different scanning methods provided in Embodiment 2 of the wafer image super-resolution method of this application;

[0061] Figure 6 This is a schematic diagram of the selective scanning mechanism provided in Embodiment 2 of the wafer image super-resolution method of this application;

[0062] Figure 7 This is a flowchart illustrating Embodiment 3 of the wafer image super-resolution method of this application;

[0063] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the wafer image super-resolution method in the embodiments of this application;

[0064] Figure 9 This is a schematic diagram illustrating the data acquisition consent process involved in the wafer image super-resolution method in this application embodiment.

[0065] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0066] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0067] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0068] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or wafer image super-resolution device capable of performing the above functions. The following description uses a wafer image super-resolution device as an example to illustrate this embodiment and the subsequent embodiments.

[0069] As chip feature sizes continue to shrink, traditional bright-field microscopes, limited by the optical diffraction limit, struggle to accurately identify nanoscale defects, increasing the risk of missed detections and misjudgments. Therefore, high-magnification objectives are often required to observe even smaller defects. However, using high-magnification objectives in wafer inspection significantly reduces the field of view, prolongs the scanning time of the entire wafer, and decreases inspection efficiency, making it difficult to improve wafer inspection resolution while maintaining efficiency.

[0070] Current traditional super-resolution methods include interpolation, reconstruction-based methods, and dictionary learning methods. Interpolation methods generate high-resolution images by interpolating adjacent pixels, which cannot recover high-frequency information such as scratches and bridging defects on the wafer surface, resulting in blurred image edges and texture distortion. Reconstruction-based methods are sensitive to noise and model mismatch and have high computational overhead, making it difficult to adapt to the global regularity of wafer images. Dictionary learning methods rely on manually designed features and fixed dictionaries, have poor generalization ability, and cannot cope with complex circuit patterns.

[0071] In deep learning-based super-resolution, convolutional neural network-based methods are limited by the local receptive field of the convolutional kernel, making it impossible to model long-range dependencies across regions of wafer images. This results in poor global structural consistency, and small defects are easily masked. Transformer-based methods have computational complexity that increases quadratically with the input resolution, leading to high memory consumption and slow inference in high-resolution wafer image processing, making it difficult to meet high-throughput detection requirements. Generative models, on the other hand, prioritize visual perception fidelity, often sacrificing physical texture fidelity, which may generate false defects or structural deviations, failing to meet the accuracy requirements of industrial inspection.

[0072] In state-space model-based methods, the original state-space model is a one-dimensional sequence design. Traditional linear scanning destroys the local continuity of pixels in two-dimensional wafer images, resulting in the loss of local defect details. Existing methods rely only on spatial domain feature modeling, which cannot take into account both global structure and local details. Furthermore, they are not designed for lightweight industrial inspection equipment, resulting in a large number of parameters and computational loads, making it difficult to integrate into existing systems to achieve real-time inference.

[0073] Based on this, embodiments of this application provide a wafer image super-resolution method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the wafer image super-resolution method of this application.

[0074] In this embodiment, the wafer image super-resolution method includes steps S10~S20:

[0075] Step S10: Obtain an initial wafer image with a first resolution.

[0076] It should be noted that the initial wafer image refers to the raw wafer microscopic image directly acquired by the bright-field microscopy system without super-resolution processing, used as input data for subsequent super-resolution reconstruction. The first resolution refers to the actual spatial resolution of the initial wafer image, which is determined by the optical parameters of the imaging system, and in this embodiment, it is a lower resolution.

[0077] Step S20: Input the initial wafer image into a preset super-resolution model to obtain a target wafer image with a second resolution, wherein the second resolution is higher than the first resolution. The super-resolution model is obtained by training a preset training model based on spatial features and frequency features extracted from preset wafer sample images. The spatial features include the repeating structure features across regions of the wafer image, and the frequency features include the high-frequency defect features of the wafer image. The extraction of spatial features is based on a preset two-dimensional selective scanning strategy, which is obtained by fusing a preset Hilbert curve scan and a preset linear scan. The Hilbert scan is based on the spatial position of the wafer image.

[0078] It should be noted that the super-resolution model in this embodiment is a deep learning model based on a state-space model, fusing spatial and frequency domain features, used to reconstruct high-resolution images from low-resolution wafer images. The second resolution refers to the resolution of the target wafer image output after processing by the super-resolution model; its value is higher than the first resolution, representing an image with higher detail density. Wafer sample images refer to the paired high- and low-resolution wafer image datasets used to train the super-resolution model. The high-resolution images are acquired by a bright-field microscopy system and flat-field corrected, while the low-resolution images are obtained by downsampling the high-resolution images.

[0079] Spatial features refer to the global regularity information embodied by the repetitive circuit structures existing across regions in a wafer image, reflecting the periodic layout in wafer manufacturing. Frequency features refer to the high-frequency components characterized by a wafer image in the frequency domain, mainly corresponding to the detailed information of local nanoscale defects such as scratches and bridging. The two-dimensional selective scanning strategy refers to a state-space model input serialization method for two-dimensional images proposed in this embodiment. Hilbert curve scanning is a fractal scanning method that can map a two-dimensional image into a one-dimensional sequence according to spatial proximity, keeping adjacent pixels close to each other in the sequence, thereby preserving local structural information.

[0080] It is understood that in this embodiment, the acquired initial wafer image is used as input and fed into a preset super-resolution model for processing, outputting a target wafer image with a second resolution. This super-resolution model is obtained by training on preset wafer sample images. During training, the model jointly extracts spatial and frequency features from the wafer sample images: spatial features characterize the repetitive structural features across regions of the wafer image, while frequency features focus on high-frequency defect features such as scratches and bridging. The extraction of spatial features relies on a preset two-dimensional selective scanning strategy, which integrates Hilbert curve scanning and linear scanning. Hilbert curve scanning traverses the wafer image according to its spatial position to maintain local continuity between pixels, thereby achieving accurate recovery of nanoscale defect details and maintaining global structural consistency while improving image resolution.

[0081] Because the super-resolution model simultaneously models spatial and frequency features using wafer sample images during the training phase, the spatial features are extracted through a two-dimensional selective scanning strategy that integrates Hilbert curve scanning and linear scanning. This effectively preserves the long-range dependencies of repeating structures across regions and the spatial continuity of local pixels in the wafer image. Meanwhile, the frequency features are focused on high-frequency defect details through frequency domain transformation. As a result, the model can accurately reconstruct a second-resolution target wafer image containing real nanoscale defect information from the initial wafer image during the inference phase. This avoids the global structural distortion caused by the limited receptive field of traditional convolutional networks and overcomes the problem of high computational complexity and difficulty in deployment of Transformer-type methods.

[0082] Furthermore, since the two-dimensional selective scanning strategy combines the ability of Hilbert curve scanning to preserve spatial proximity with the supplementary role of linear scanning in sequence integrity, the state-space model can efficiently model long-range dependencies when processing two-dimensional wafer images, without destroying the detailed representation of local defects due to the serialization process. This significantly improves the physical fidelity and reliability of the super-resolution reconstruction results in industrial inspection scenarios, meeting the actual needs of advanced processes for high-precision and high-efficiency wafer defect detection.

[0083] In one feasible implementation, the model to be trained includes a shallow feature extraction module, a dual-domain transformation module, and a high-resolution reconstruction module. The specific implementation before inputting the initial wafer image into the preset super-resolution model to obtain the target wafer image at a second resolution can also be:

[0084] A preset high-resolution wafer sample image is acquired, and the wafer sample image is downsampled a preset number of times to obtain a first-resolution sample image corresponding to the wafer sample image. The first-resolution sample image is input into the shallow feature extraction module to obtain shallow features of the first-resolution sample image. The shallow features are input into a preset dual-domain transformation module to obtain target deep features that fuse the spatial features and the frequency features. The deep features are input into the high-resolution reconstruction module to obtain a wafer reconstruction image with improved resolution. Based on the wafer reconstruction image and the wafer sample image, the parameters of the model to be trained are adjusted to obtain the super-resolution model.

[0085] It should be noted that the model to be trained refers to the super-resolution neural network that has not yet converged during the training phase. It consists of a shallow feature extraction module, a dual-domain transformation module, and a high-resolution reconstruction module connected sequentially, used to learn the mapping relationship from low-resolution wafer images to high-resolution images. The shallow feature extraction module is the initial processing unit in the model to be trained, used to perform preliminary feature encoding on the input first-resolution sample image, extracting basic texture and edge information to generate shallow features. The dual-domain transformation module is the core processing unit in the model to be trained, used to jointly model the shallow features in the spatial and frequency domains, fusing spatial and frequency features to output the target deep features.

[0086] The high-resolution reconstruction module is the final processing unit in the model to be trained. It is used to upsample the deep features of the target and reconstruct them into a second-resolution wafer reconstruction image. In this embodiment, the first-resolution sample image is a low-resolution training sample with the same resolution as the initial wafer image, obtained by downsampling a preset high-resolution wafer sample image a preset number of times. In this embodiment, bicubic interpolation is used to downsample the high-resolution image to obtain paired high- and low-resolution data.

[0087] The reconstructed wafer image is an output image with a second resolution, generated by the model to be trained based on the first-resolution sample image. It is used to perform loss calculations with the original high-resolution wafer sample image.

[0088] It should also be noted that this embodiment uses images of 12-inch patterned wafers acquired by a bright-field illumination imaging system to construct the experimental dataset. This imaging system mainly consists of a light source, camera, objective lens, and microcomputer. The light source is a single-band deep ultraviolet laser with a wavelength of 266 nm and a power of 100 mW; the camera has a pixel size of 11 μm, a resolution of 2048 × 2048 pixels, and an exposure time set to 30 ms; the objective lens is a 134x high-magnification objective lens with a numerical aperture (NA) of 0.95. With this configuration, this embodiment acquired a total of 800 high-resolution wafer microscopic images with a resolution of 2048 × 2048. Subsequently, flat-field correction was performed on the 800 images to address the problem of uneven light source and remove fixed noise from the images. Finally, 40 images were randomly selected from the corrected images as the validation set, 40 as the test set, and the remaining 720 were used for training.

[0089] It is understandable that, since the model to be trained is clearly divided into three functional stages—shallow feature extraction module, dual-domain transformation module, and high-resolution reconstruction module—and the first-resolution sample image obtained by downsampling high-resolution wafer sample images is used as input during the training process, the model can perform end-to-end optimization with the real physical imaging process as the supervision signal, thereby ensuring the consistency between the training objective and the actual application scenario and improving the model's generalization ability.

[0090] Furthermore, because the dual-domain transformation module integrates spatial features extracted based on a two-dimensional selective scanning strategy and frequency features representing high-frequency defects during the training phase, and transforms these fused features into a second-resolution wafer reconstruction image through the high-resolution reconstruction module, while the model parameters are adjusted based on the error between the wafer reconstruction image and the original high-resolution wafer sample image, the final super-resolution model not only effectively enhances image resolution but also simultaneously preserves or strengthens key repeating structures and minute defect information in the wafer during the reconstruction process. The overall technical approach of this embodiment can be referred to... Figure 2 .

[0091] In one feasible implementation, the dual-domain transformation module includes multiple cascaded spatial frequency groups, each spatial frequency group includes multiple spatial frequency sub-modules, and each spatial frequency sub-module includes a spatial feature extraction unit and a frequency feature extraction unit. The specific implementation of inputting the shallow features into the preset dual-domain transformation module to obtain target deep features that fuse the spatial features and the frequency features can also be:

[0092] The shallow features are input into the spatial feature extraction unit, and the features to be processed in the spatial feature extraction unit are further extracted to obtain the spatial features. The spatial features are then input into the frequency feature extraction unit of the same spatial frequency submodule, and the frequency features in the spatial features are extracted to obtain the spatial frequency features. It is then determined whether there are any unused spatial frequency submodules. If there are, the frequency features are input into the spatial feature extraction unit of the next spatial frequency submodule until there are no unused spatial frequency submodules, thus obtaining the target deep features.

[0093] It should be noted that the Spatial Frequency Group (SFG) refers to the structural unit in the dual-domain transform module, which is composed of multiple cascaded spatial frequency sub-modules and is used to deepen the extraction and fusion of spatial and frequency features in stages. The Spatial Frequency Sub-module (SFB) is the basic building block of the Spatial Frequency Group, containing a Spatial Feature Extraction Unit (SDT) and a Frequency Feature Extraction Unit (FDT), used to process input features sequentially and output spatial frequency features. The Spatial Feature Extraction Unit is the first processing unit in the Spatial Frequency Sub-module, used to perform spatial domain modeling on the input features to be processed, extracting spatial features reflecting the repeating structure across regions of the wafer.

[0094] The frequency feature extraction unit is the second processing unit in the spatial frequency submodule. It is used to further perform frequency domain analysis on the spatial features output by the spatial feature extraction unit in the same submodule, extract high-frequency defect information, and generate spatial frequency features. The feature to be processed refers to the intermediate feature representation currently being processed by a certain spatial frequency submodule during the internal transmission process of the dual-domain transform module. Spatial frequency features refer to the fused features obtained after sequential processing by the spatial feature extraction unit and the frequency feature extraction unit, simultaneously containing spatial structure information and frequency detail information. The target deep feature refers to the deep fused feature finally output by the dual-domain transform module after processing by all spatial frequency submodules, used for subsequent high-resolution reconstruction. The overall network structure described above in this embodiment can be referred to... Figure 3 .

[0095] Understandably, wafer microscopic images possess significant unique characteristics compared to super-resolution tasks involving natural images. First, the circuit patterns on the wafer surface follow strict physical manufacturing rules. An entire wafer is typically composed of repeating units with consistent orientation, spacing, and shape, exhibiting a clear global layout logic. Second, local details are core factors determining wafer quality, such as the gate edge morphology of individual transistors, the corner curvature of wires, and the circular boundary precision of contact holes. The fineness of these local features directly affects the actual performance of the circuit. Finally, some manufacturing defects (such as scratches, bridging, and contaminants) exist only in small local areas (potentially occupying only a few pixels), yet they play a crucial role in wafer yield. In summary, wafer images exhibit a significant characteristic of global constraints on local areas and local support for global processes, which is fundamentally different from the random structure of natural images.

[0096] Therefore, in this embodiment, shallow features are first used as initial features to be processed and input into the spatial feature extraction unit of the first spatial frequency submodule. Spatial domain modeling is performed on these features to extract spatial features representing the repeating structure across regions of the wafer image. Subsequently, these spatial features are sent to the frequency feature extraction unit within the same spatial frequency submodule to perform frequency domain decomposition, extracting high-frequency components containing details such as scratches and bridging, thus obtaining spatial frequency features. It is then determined whether there are any unused spatial frequency submodules in the dual-domain transformation module. If so, the currently obtained spatial frequency features are used as the features to be processed in the next spatial frequency submodule and input into its spatial feature extraction unit, repeating the above spatial-frequency joint extraction process. This process is executed sequentially until all spatial frequency submodules have been traversed. The final output features are the target deep features, thereby achieving multi-level, progressive deep fusion of wafer image features in the spatial and frequency domains.

[0097] Because the dual-domain transformation module uses multiple cascaded spatial frequency groups, each group contains multiple sequentially connected spatial frequency sub-modules, and each sub-module processes the features in the order of first spatial feature extraction and then frequency feature extraction, the spatial structure information and high-frequency defect information can be repeatedly refined and interacted at different levels. Thus, the model's ability to jointly represent complex repeating patterns and weak defects in wafer images is enhanced through the above-mentioned progressive deepening method.

[0098] Furthermore, since each spatial frequency submodule uses the spatial frequency features output from the previous stage as input to the next stage spatial feature extraction unit, forming a feature transfer chain that alternately enhances spatial and frequency, it avoids the defects caused by single-domain modeling. Without significantly increasing the computational burden, it effectively improves the discriminability and completeness of the target's deep features, providing effective features for the subsequent high-resolution reconstruction module to generate physically accurate and detailed second-resolution target wafer images.

[0099] In one feasible implementation, the spatial feature extraction unit includes a visual state spatial subunit and a multi-scale feature extraction unit. The specific implementation of further feature extraction of the features to be processed input into the spatial feature extraction unit can also be:

[0100] The features to be processed are normalized, and the normalized features are input into the visual state space subunit. Global features of the wafer image are extracted to obtain global features. The global features are input into the multi-scale feature extraction unit, and local features of the wafer image are supplemented into the global features to obtain the spatial features that fuse the global features and the local features.

[0101] It should be noted that the Visual State Space Subunit (VSSM) is the core component of the spatial feature extraction unit. Based on a state space model, it models the input features to capture long-range dependencies in wafer images and extract global features. The Multi-Scale Feature Extraction (MSFE) unit is a supplementary component of the spatial feature extraction unit. Through parallel or sequential multi-scale convolution operations, it captures local detail information under different receptive fields, enhancing local feature representation.

[0102] Normalization refers to performing layer normalization or similar standardization on the input features to stabilize the training process and improve the consistency of feature distribution. Global features refer to high-level semantic features extracted from visual state space sub-units that reflect the overall structural layout and cross-regional repetition patterns of the wafer image. Local features refer to the fine textures and geometric details of specific regions in the wafer image (such as individual circuit units or defect neighborhoods), supplemented by multi-scale feature extraction units.

[0103] Understandably, in this embodiment, the features to be processed are first normalized to stabilize their numerical distribution. Then, the normalized features are input into the visual state space sub-unit, which utilizes its state space modeling capabilities to model the long-range dependencies of the wafer image, extracting global features representing the overall repeating structure. These global features are then fed into a multi-scale feature extraction unit. Through convolutional kernels or receptive field mechanisms of different scales, local detail information of key regions in the wafer image is supplemented based on the global features. Finally, a spatial feature integrating global and local features is output, thereby achieving collaborative modeling of the overall structure and local details of the wafer image in the spatial domain.

[0104] Because the visual state space sub-unit effectively captures the periodic repetitive structures across regions of the wafer image, while the multi-scale feature extraction unit specifically enhances the detailed representation of local circuit patterns or potential defect areas, this embodiment, through the above steps, simultaneously preserves global consistency and local geometric accuracy within a single spatial feature, avoiding the problems of neglecting details in global modeling or lacking context in local modeling in traditional methods.

[0105] In one feasible implementation, the specific implementation of extracting global features from the wafer image to obtain the global features can also be:

[0106] The normalized feature to be processed is processed sequentially based on the linear mapping layer and activation function layer of the state space unit to obtain a first nonlinear feature. Then, the normalized feature to be processed is processed sequentially based on the linear mapping layer, depthwise separable convolutional layer, and activation function layer of the state space unit to obtain a second nonlinear feature. The first and second nonlinear features are obtained by processing the same feature to be processed using different methods. The second nonlinear feature is input into the two-dimensional selection scan layer of the state space unit, where sequential scanning, state feature extraction, and two-dimensional mapping are performed to obtain a serialized feature. The state feature extraction is based on a preset state space equation with linear computational complexity. The serialized feature is normalized, and the normalized serialized feature is multiplied element-wise by the first nonlinear feature to obtain a fused feature. Finally, the feature to be processed and the fused feature are added element-wise to obtain the global feature.

[0107] It should be noted that the state space group unit refers to the internal structural module of the visual state space subunit, which consists of linear mapping layers, activation function layers, depthwise separable convolutional layers, and two-dimensional selective scan layers, etc., and is used to achieve efficient state space modeling. The activation function layer enhances the model's expressive power and generates nonlinear features by introducing nonlinear transformations. In this embodiment, the activation function can be SiLU (Sigmoid Linear Unit). The depthwise separable convolutional layer is a lightweight convolution operation that first performs spatial convolution on each channel independently, and then fuses the channel information through 1×1 convolution, reducing computational cost while preserving local spatial details.

[0108] The 2D selective scan layer is a module used to execute the 2D selective scan strategy, converting 2D feature maps into 1D sequences for processing by the state-space equations. Serialized features refer to intermediate feature representations that are serialized by the 2D selective scan layer, modeled by the state-space equations, and then mapped back to 2D form. Fusion features are the result of element-wise multiplication of the normalized serialized features with the first nonlinear feature.

[0109] It should also be noted that the state-space model is a mathematical framework for describing the evolution of dynamic systems, and its theoretical foundation originates from classical control theory. The state-space model was initially used to characterize the dynamic behavior of linear time-invariant systems in continuous time, expressing the changes in the system state over time through differential equations. For a one-dimensional continuous input sequence x(t)∈R, its corresponding hidden state h(t)∈R N The mapping relationship between the output sequence y(t)∈R can be expressed as:

[0110]

[0111]

[0112] in, Let h(t) be the time derivative of the hidden state vector h(t); A∈R N×N It is a matrix used to control the dynamic evolution of the hidden states; matrix B∈R N×1 Used to model the effect of input x(t) on the system; matrix C∈R 1×N The current hidden state is mapped to the output y(t). The above formula constitutes a standard linear time-invariant state-space system.

[0113] However, the state-space equation expressed by the above formula is designed for continuous input sequences and cannot handle discrete data such as text and digital images. Therefore, this embodiment uses the zero-order hold technique to discretize the continuous system described by the formula. The discretized formula is as follows:

[0114]

[0115]

[0116] Here, Δ represents the discretization step size. Through this transformation, continuous parameters A and B can be mapped to discrete parameters. , Through mathematical derivation, the recurrence relation for an RNN (Recurrent Neural Network) can be obtained as follows:

[0117]

[0118]

[0119] in, It is the input data. It is the output data. It is the hidden state from the previous step. It is the hidden state of the current step.

[0120] Furthermore, equation (3) can also be equivalently represented as a convolution form:

[0121]

[0122]

[0123] Where L represents the length of the input sequence, Represents a structured convolution kernel. This represents the convolution operation. A major advantage of representing the state-space equation in convolutional form is that it can be trained in parallel, just like a convolutional neural network (CNN), providing us with a simple way to compute y quickly using convolution.

[0124] It is understood that the global feature extraction in this embodiment adopts a dual-path parallel structure. The first path preserves the original high-level semantics, while the second path introduces local context and feeds it into the state space equation with linear complexity through a two-dimensional selection scanning layer for long-range modeling. Furthermore, the two paths achieve gated fusion through element-wise multiplication, thereby enabling the model to perceive the repetitive structure across regions in the wafer image and dynamically adjust the attention weights for different regions without significantly increasing the computational burden.

[0125] Furthermore, since the computational complexity of the state-space equation is linear and the two-dimensional selection scanning layer effectively maintains the spatial proximity between pixels, long sequence modeling is both efficient and faithful. Finally, by adding the fused features to the residuals of the original features to be processed, not only is the gradient vanishing problem alleviated, but the global features are also preserved for the input details, thereby significantly improving the modeling efficiency and reconstruction accuracy of the super-resolution model when processing large-scale wafer images.

[0126] In one feasible implementation, the specific implementation of supplementing the global features with the local features of the wafer image to obtain the spatial features that fuse the global features and the local features can also be:

[0127] The global features are normalized, and then convolved with multiple convolutional layers of different kernel sizes to obtain multiple local features. These local features are then concatenated along their channel dimensions to obtain concatenated local features. The concatenated local features are then subjected to a non-linear transformation through an activation function layer, and convolved with a pre-defined fusion convolutional layer to obtain multi-scale enhanced features. Finally, the multi-scale enhanced features and the global features are added element-wise to obtain the spatial features.

[0128] It should be noted that, in this embodiment, convolutional layers with different kernel sizes refer to multiple parallel convolutional operation units, each employing a different kernel size (e.g., 3×3, 5×5, 7×7) to capture local structural information within different receptive fields. Local features are feature representations reflecting local details of the wafer image at different scales, obtained by convolving the global features with the aforementioned different kernels. Stitched local features are combined features formed by stitching multiple local features along the channel dimension, integrating multi-scale local information. A fusion convolutional layer refers to a 1×1 convolutional layer or similar structure used to perform channel compression and information fusion on the stitched local features after nonlinear transformation, outputting multi-scale enhanced features with a unified dimension. Multi-scale enhanced features refer to enhanced features obtained after processing by the fusion convolutional layer, incorporating multi-scale local details, used to supplement missing local information in the global features.

[0129] It is understood that this embodiment extracts local features in parallel through multiple convolutional layers with different kernel sizes, and then integrates them uniformly through a fusion convolutional layer after being stitched together in the channel dimension. This enables the model to simultaneously perceive multi-scale local structures such as tiny defects, medium-sized circuit units, and large repeating patterns in the wafer image, thereby significantly enhancing the ability of spatial features to cover diverse local details under complex manufacturing processes.

[0130] Furthermore, since the multi-scale enhancement features are ultimately fused with the global features through residual connections, it avoids local information from overriding global semantics and ensures the effective injection of high-frequency details. Thus, without disrupting the overall periodic layout of the wafer image, it accurately supplements the local geometric and texture information of key areas, providing a more comprehensive and robust feature foundation for subsequent frequency feature extraction and high-fidelity super-resolution reconstruction.

[0131] In summary, this embodiment acquires an initial wafer image with a first resolution, inputs the initial wafer image into a preset super-resolution model, and obtains a target wafer image with a second resolution, wherein the second resolution is higher than the first resolution. The super-resolution model is obtained by training a preset training model based on spatial features and frequency features extracted from preset wafer sample images. The spatial features include the repeating structure features across regions of the wafer image, and the frequency features include the high-frequency defect features of the wafer image. The extraction of spatial features is based on a preset two-dimensional selective scanning strategy, which is obtained by fusing a preset Hilbert curve scan and a preset linear scan. The Hilbert scan is based on the spatial position of the wafer image.

[0132] To address the issue that using high-magnification objectives in wafer inspection significantly reduces the field of view, prolongs the scanning time of the entire wafer, and decreases inspection efficiency, making it difficult to improve wafer inspection resolution while maintaining efficiency, this embodiment improves the resolution of the initial wafer image through a preset super-resolution model, obtaining a target wafer image with a second resolution. Because this embodiment accurately characterizes the long-range dependence of repetitive circuits across wafer regions through spatial features during model training and focuses on high-frequency defect features of the wafer through frequency features, the wafer resolution can be effectively improved based on these features. Furthermore, in spatial feature extraction, this embodiment uses Hilbert curve scanning, scanning based on the spatial position of the wafer image, combined with linear scanning. This ensures spatial continuity while avoiding redundant calculations in multi-directional, block-based scanning, reducing wafer scanning time and thus improving inspection efficiency. Therefore, this embodiment can improve wafer inspection resolution while maintaining wafer inspection efficiency.

[0133] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 The nonlinear feature is a two-dimensional feature. The step of sequentially performing serialization scanning, state feature extraction, and two-dimensional mapping on the nonlinear feature to obtain the serialized feature includes steps A10 to A60:

[0134] Step A10: Based on the scanning order corresponding to the Hilbert scan, the nonlinear feature is scanned to obtain a one-dimensional Hilbert sequence, wherein the scanning order corresponding to the Hilbert scan is determined based on the spatial position of the nonlinear feature;

[0135] It should be noted that nonlinear features refer to the two-dimensional feature maps obtained after processing by the linear mapping layer and activation function layer in the state-space group unit, possessing nonlinear expressive capabilities. A Hilbert sequence refers to a one-dimensional feature sequence formed by expanding the two-dimensional nonlinear features according to the traversal order defined by the Hilbert curve, where each element corresponds to a spatial location in the original image. The scanning order refers to the sequential order in which pixels or feature points are read during a Hilbert scan. This order is uniquely determined by the recursive path of the Hilbert curve at a given image size and strictly depends on the spatial coordinates of the nonlinear features. A comparison of different scanning methods in this embodiment can be found in [reference needed]. Figure 5 .

[0136] It is understandable that, since the traversal order of Hilbert scan is strictly generated based on the spatial position of nonlinear features, its fractal properties ensure that when two-dimensional features are flattened into a one-dimensional sequence, spatially adjacent pixels also remain approximately close in the sequence. Therefore, this embodiment can effectively alleviate the problem of local dependency breakage caused by vertical jumps in traditional linear scans through the above scanning method.

[0137] Furthermore, since the Hilbert sequence is used as the input for state feature extraction, its good spatial continuity enables the state space equation to more accurately capture the inherent regularity of periodic repeating structures in wafer images when modeling long-distance dynamic evolution. This significantly improves the physical rationality and detail fidelity of global feature modeling without increasing computational complexity, providing a foundation for the subsequent generation of high-precision super-resolution images.

[0138] Step A20: Based on the scanning order corresponding to the linear scan, the nonlinear feature is scanned to obtain a one-dimensional linear scan sequence, wherein the scanning order corresponding to the linear scan is a predetermined row order or column order;

[0139] It should be noted that linear scanning refers to a serialization method that traverses two-dimensional features row by row or column by column in a fixed direction, with the scanning order being a predetermined row or column order. A linear scanning sequence is a one-dimensional feature sequence obtained by expanding two-dimensional nonlinear features according to a linear scanning order, but adjacent elements at positions such as row transitions may not have a proximity relationship in the original space.

[0140] It is understood that, in this embodiment, while performing a Hilbert scan on the nonlinear feature, a separate processing path is established to traverse the nonlinear feature according to a preset scanning order corresponding to linear scans (e.g., row-wise from left to right, top to bottom row priority order), transforming it from a two-dimensional form into a one-dimensional linear scan sequence. This scanning order is fixed and regular, independent of image content, and determined solely by the spatial dimension of the image. This provides a simple and computationally efficient supplementary sequence for subsequent feature fusion, compensating for potential continuity deficiencies in certain local regions caused by Hilbert scans.

[0141] Since linear scanning uses a predetermined row or column order to serialize nonlinear features, it is simple to implement and has extremely low computational overhead, and can quickly generate a complete one-dimensional sequence representation covering the entire graph; thus providing a globally comprehensive basic path for state space modeling.

[0142] Step A30: Based on the state-space equation, extract the features of the Hilbert sequence and the linear sequence respectively to obtain the Hilbert features corresponding to the Hilbert sequence and the linear features corresponding to the linear sequence;

[0143] It should be noted that Hilbert features refer to the one-dimensional feature sequence obtained by inputting the Hilbert sequence into the state-space equation and mapping it to its internal hidden states, preserving long-range dependency information based on spatial proximity. Linear features refer to the one-dimensional feature sequence output after inputting a linear scan sequence into the same state-space equation, reflecting the global context modeling results based on the rule traversal order.

[0144] Understandably, because the state-space equations extract features independently but structurally consistent for both the Hilbert sequence and the linear scan sequence, the model can simultaneously capture the different spatial association patterns emphasized by the two scanning strategies. Hilbert features focus on long-range modeling that maintains the continuity of local neighborhoods, while linear features ensure global context awareness that covers the entire graph and has a regular structure. By extracting these features, complementary enhancements in multi-view sequence modeling can be achieved without significantly increasing model complexity.

[0145] Step A40: Based on the spatial location, adjust the element positions of the Hilbert feature to obtain a Hilbert feature whose spatial location is the same as the linear feature;

[0146] It should be noted that spatial location refers to the position of each pixel or feature point in the nonlinear feature within the original two-dimensional image coordinate system, serving as the geometric reference for both Hilbert and linear scans. Adjusting element positions refers to reordering the elements in the one-dimensional sequence according to the mapping relationship between the original spatial locations and sequence indices, aligning their order with other sequences. A Hilbert feature with the same spatial location as the linear feature refers to a new sequence obtained by rearranging the original Hilbert features according to the sequence order corresponding to the linear scan, where each element at its position corresponds to the same spatial location in the image at the same index as the linear feature.

[0147] Understandably, since Hilbert features are one-dimensional sequences generated based on the Hilbert scan order, their element order differs from that of linear features. However, both originate from the same non-linear feature, and each element corresponds to a unique original spatial position. Therefore, this embodiment reorders the elements in the Hilbert feature based on a predefined mapping relationship between this spatial position and the linear scan index, thereby obtaining a Hilbert feature that is strictly aligned with the linear feature in terms of sequence index. This provides a positional consistency prerequisite for subsequent element-by-element fusion operations, ensuring that the two features represent information from the same spatial region at the same sequence position.

[0148] Step A50: The Hilbert feature and the linear feature with the adjusted element positions are added element by element to obtain the fused scan feature;

[0149] It should be noted that the Hilbert feature with adjusted element positions refers to the Hilbert feature whose sequence index is strictly aligned with the linear feature in spatial position after reordering. The fused scan feature refers to the one-dimensional fused sequence representation obtained by adding the adjusted Hilbert feature and the linear feature element by element at the same sequence position, which combines the modeling results of the two scanning strategies.

[0150] Since the adjusted Hilbert features and linear features have achieved spatial semantic alignment in the sequence dimension, element-wise addition can ensure that information from the two scanning perspectives is effectively superimposed in the same spatial position, thereby avoiding semantic confusion or detail blurring caused by misaligned fusion.

[0151] Furthermore, since Hilbert scanning excels at preserving the continuity of local structures while linear scanning ensures complete global traversal, the fused scan feature formed by the combination of the two possesses both sensitivity to local details and integrity of global structures. This significantly enhances the state-space model's ability to jointly represent periodic repeating units and small defect regions in wafer images, providing a more reliable intermediate feature representation for super-resolution reconstruction.

[0152] Step A60: Perform the two-dimensional mapping on the fused scanning features to obtain the two-dimensional serialized features.

[0153] It should be noted that two-dimensional mapping refers to the operation of rearranging one-dimensional fused scan features back into the spatial dimension of the original image according to a preset linear scan order, restoring it to a two-dimensional feature map. Serialized features refer to the two-dimensional feature representation obtained after two-dimensional mapping.

[0154] It is understood that after obtaining the one-dimensional fused scanning features in this embodiment, a two-dimensional mapping operation is performed according to the same inverse mapping rule as linear scanning, reconstructing it into a two-dimensional feature map with the same spatial resolution as the original nonlinear features. This feature map inherits the fusion information of the state-space equation modeling results of the two scanning strategies and restores the standard spatial layout, facilitating processing by subsequent modules. The selective scanning mechanism in this embodiment can be referred to... Figure 6 .

[0155] Since the fused scan features are obtained by fusing Hilbert and linear scan modeling results under the premise of spatial semantic alignment, two-dimensional mapping of them can ensure that the reconstructed serialized features contain complementary long-range dependency information at each spatial location, thereby injecting richer and more robust contextual representations while preserving the original image geometry.

[0156] Furthermore, since this two-dimensional mapping strictly follows the inverse process of linear scanning, it is fully compatible with the input format of subsequent modules and requires no additional adaptation. Thus, without increasing the complexity of the model, it provides effective intermediate features for the output of high-quality global features of the visual state space sub-unit, thereby improving the reconstruction accuracy of the entire super-resolution model for wafer nanoscale defects.

[0157] In summary, this embodiment employs a dual-path parallel strategy of Hilbert scanning and linear scanning. Hilbert scanning maximizes the preservation of the continuity of spatially adjacent pixels in the sequence through fractal curves, while linear scanning ensures regular and complete full-image traversal. After being modeled separately by state-space equations, the two are fused at a unified spatial location, effectively complementing the limitations of a single scanning method. Thus, without significantly increasing the computational burden, the state-space model's ability to jointly perceive periodic structures and minute defects in wafer images is significantly improved.

[0158] Furthermore, this embodiment ensures that the fused sequence representation can be accurately restored to spatially consistent two-dimensional features through spatial alignment and two-dimensional mapping, avoiding semantic misalignment or detail distortion. The linear complexity of the state-space equation itself guarantees the real-time performance of high-resolution image processing, enabling this enhanced two-dimensional selective scanning mechanism to accurately preserve local details of nanoscale defects and efficiently model long-range dependencies across regions, ultimately providing rich and computationally efficient high-quality intermediate features for super-resolution reconstruction.

[0159] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 7 The frequency features include low-frequency features and high-frequency features. The step of extracting the frequency features from the spatial features to obtain spatial frequency features includes steps H10~H50:

[0160] Step H10: Perform the normalization operation on the spatial features, and reduce the channel dimension of the normalized spatial features based on a preset channel transformation convolutional layer to obtain reduced spatial features.

[0161] It should be noted that the channel transformation convolutional layer refers to a 1×1 convolutional layer used to adjust the number of feature channels. In this embodiment, it is used to reduce the channel dimension of the normalized spatial features from C (channels) to C / 4 to reduce the computational overhead of subsequent frequency domain processing. Reduced spatial features refer to the intermediate feature representation after processing by the channel transformation convolutional layer, where the number of channels is reduced but the spatial resolution remains unchanged, with dimensions of H (height) × W (width) × (C / 4).

[0162] Understandably, since the computational cost of frequency domain processing is positively correlated with the number of channels in the input features, directly performing frequency domain decomposition on high-dimensional spatial features would lead to significant resource consumption. Therefore, this embodiment pre-compresses the channel dimension through a channel transformation convolutional layer, significantly reducing the data scale of subsequent operations, thereby improving the overall efficiency of the frequency feature extraction module with almost no loss of key information.

[0163] Step H20: The reduced spatial features are nonlinearly transformed through the activation function layer, and the reduced spatial features after nonlinear transformation are decomposed into low-frequency features and high-frequency features based on a preset discrete wavelet transform layer;

[0164] It should be noted that the Discrete Wavelet Transform layer refers to the differentiable module that implements the Discrete Wavelet Transform (DWT), which is used to decompose the input features in the frequency domain into a low-frequency approximate subband (LL) and three high-frequency detail subbands (LH, HL, HH), thereby separating the main structure of the image from detailed information such as edges / texture.

[0165] Low-frequency features refer to the low-frequency sub-bands (LL) obtained after discrete wavelet transform, which retain the main structure and slowly changing global information in the reduced spatial features, with a spatial resolution of half that of the original image (H / 2×W / 2).

[0166] The high-frequency features refer to the three high-frequency sub-bands (LH, HL, HH) obtained after discrete wavelet transform, which correspond to detailed information (such as edges, scratches, bridging and other defects) in the horizontal, vertical and diagonal directions, respectively, and also have a resolution of H / 2×W / 2.

[0167] Understandably, since discrete wavelet transform inherently possesses multi-resolution and direction-selective analysis capabilities, it can effectively separate reduced spatial features into two parts: the main structure (low frequency) and details (high frequency). This allows the model to explicitly focus on high-frequency components closely related to defect detection, avoiding the neglect of weak defects in the unified spatial domain processing of traditional methods.

[0168] Furthermore, since high-frequency features clearly correspond to the directional details of key industrial defects such as scratches and bridging, while low-frequency features retain the consistency of the overall circuit layout of the wafer, the two together constitute an accurate frequency domain characterization of the physical properties of the wafer image, thus providing a foundation for the subsequent fusion and reconstruction stages, and significantly improving the reliability and practicality of super-resolution results in quantitative defect analysis.

[0169] In one feasible implementation, the reduced spatial features are two-dimensional features, and the high-frequency features include horizontal high-frequency features, vertical high-frequency features, and diagonal high-frequency features. A further specific implementation of decomposing the nonlinearly transformed reduced spatial features into low-frequency features and high-frequency features based on a preset discrete wavelet transform layer can also be:

[0170] Based on the discrete wavelet transform layer, in the horizontal direction, the reduced spatial features are low-pass filtered and downsampled to obtain initial low-frequency features and initial high-frequency features. Based on the discrete wavelet transform layer, in the vertical direction, the initial low-frequency features are low-pass filtered and downsampled to obtain low-frequency features and vertical high-frequency features. Based on the discrete wavelet transform layer, in the vertical direction, the initial high-frequency features are low-pass filtered and downsampled to obtain horizontal high-frequency features and diagonal high-frequency features.

[0171] It should be noted that low-pass filtering refers to smoothing the signal using a low-pass filter, preserving the slowly changing core information. The initial low-frequency feature refers to the intermediate low-frequency sub-band obtained after low-pass filtering and downsampling the reduced spatial features in the horizontal direction, with dimensions of H×(W / 2)×(C / 4). The initial high-frequency feature refers to the intermediate high-frequency sub-band obtained after high-pass filtering and downsampling the reduced spatial features in the horizontal direction, corresponding to horizontal details, with the same dimensions. The horizontal high-frequency feature ultimately represents the high-frequency sub-band (LH) of the image's horizontal edge or texture variations, obtained by low-pass filtering and downsampling the initial high-frequency feature in the vertical direction.

[0172] The vertical high-frequency feature, representing the high-frequency subband (HL) detailing in the vertical direction, is obtained by high-pass filtering and downsampling the initial low-frequency feature in the vertical direction. The diagonal high-frequency feature, representing the high-frequency subband (HH) detailing in the diagonal direction, is obtained by high-pass filtering and downsampling the initial high-frequency feature in the vertical direction.

[0173] It is understandable that, since the horizontal high-frequency features, vertical high-frequency features, and diagonal high-frequency features are obtained by further decomposing the initial high-frequency features and initial low-frequency features in the vertical direction, this embodiment preserves the hierarchical relationship and spatial alignment between frequency bands through the above processing. Thus, when the visual state space sub-units are input for modeling in the subsequent process, cross-frequency band semantic misalignment can be avoided, and the geometric accuracy and coherence of high-frequency detail reconstruction can be improved.

[0174] Step H30: Input the low-frequency features and the high-frequency features into the visual state space sub-unit to obtain the low-frequency spatial features corresponding to the low-frequency features and the high-frequency spatial features corresponding to the high-frequency features.

[0175] It should be noted that low-frequency spatial features refer to the enhanced low-frequency representation output by state space modeling after inputting low-frequency features into the visual state space sub-unit, which incorporates long-range dependency information, preserving and enhancing the global consistency of periodic repeating structures in the wafer image. High-frequency spatial features refer to the enhanced high-frequency representation obtained by processing high-frequency features into the visual state space sub-unit through the same mechanism, further modeling cross-regional contextual relationships based on frequency domain details, used to improve the structural coherence and positioning accuracy of defects such as scratches and bridging.

[0176] Understandably, since low-frequency and high-frequency features respectively characterize the global structure and local defects in a wafer image, their inherent spatial relationships can easily be overlooked if processed in isolation only in the frequency domain. Therefore, this embodiment inputs both into visual state space sub-units and utilizes their efficient long-range modeling capabilities to explicitly model cross-regional dependencies within their respective frequency bands. This allows low-frequency spatial features to more accurately reflect the consistency of repeating units across the entire wafer, while high-frequency spatial features can effectively connect discretely distributed similar defects or edge structures, avoiding fragmentation of details.

[0177] Step H40: Based on the inverse wavelet transform layer corresponding to the discrete wavelet transform layer, the low-frequency spatial features and the high-frequency spatial features are fused to obtain spatial frequency fusion features;

[0178] It should be noted that the inverse wavelet transform layer refers to the differentiable reconstruction module corresponding to the discrete wavelet transform layer, used to losslessly or nearly losslessly synthesize a low-frequency subband and three high-frequency subbands into a two-dimensional feature map with the original resolution. Spatial-frequency fusion features refer to the two-dimensional feature representation obtained by recombining low-frequency and high-frequency spatial features through the inverse wavelet transform layer. This representation includes both enhanced global structural information through state-space modeling and enhanced local defect details.

[0179] It is understandable that since the low-frequency and high-frequency spatial features are enhanced through state-space modeling, which strengthens the spatial context within their respective frequency bands, direct splicing or simple addition would disrupt the frequency domain structure. Therefore, this embodiment employs an inverse wavelet transform layer that is strictly dual to the decomposition process for fusion, ensuring the frequency domain structure of the information during reconstruction.

[0180] Step H50: Based on the channel transformation convolutional layer and the activation function layer, the channel dimension of the spatial frequency fusion feature is expanded to obtain the spatial frequency feature.

[0181] It should be noted that expanding the channel dimension refers to restoring the number of channels of the spatial frequency fusion feature from the reduced C / 4 to the original number of channels C through 1×1 convolution, in order to match the feature dimension of the network backbone.

[0182] Understandably, since the channel dimension has been compressed before frequency domain processing to reduce computational overhead, failure to restore the channel dimension after processing would result in insufficient feature representation and disrupt the structural consistency between model layers. Therefore, this embodiment accurately restores the number of channels through learnable channel transformation convolutional layers and enhances nonlinear representation capabilities by combining activation function layers, thereby ensuring the information integrity and modeling capacity of spatial frequency features without increasing redundant parameters.

[0183] In summary, this embodiment employs discrete wavelet transform to perform multi-directional and multi-scale frequency domain decomposition of spatial features, clearly separating low-frequency features characterizing global periodic structures and three types of high-frequency features characterizing nanoscale defects such as scratches and bridging. Furthermore, it independently models each frequency band through visual state space sub-units, enabling the model to apply differentiated contextual enhancements to components with different physical meanings. This significantly improves the perception and reconstruction capabilities of weak and local defects while preserving the prior knowledge of wafer image manufacturing.

[0184] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the wafer image super-resolution method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.

[0185] This application provides a wafer image super-resolution device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the wafer image super-resolution method in the first embodiment described above.

[0186] The following is for reference. Figure 8 The diagram illustrates a structural schematic suitable for implementing a wafer image super-resolution device according to embodiments of this application. The wafer image super-resolution device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The wafer image super-resolution device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0187] like Figure 8 As shown, the wafer image super-resolution device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the wafer image super-resolution device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the wafer image super-resolution device to communicate wirelessly or wiredly with other devices to exchange data. Although wafer image super-resolution devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0188] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0189] The wafer image super-resolution device provided in this application, employing the wafer image super-resolution method in the above embodiments, can solve the technical problem of simultaneously ensuring wafer inspection efficiency and inspection resolution. Compared with the prior art, the beneficial effects of the wafer image super-resolution device provided in this application are the same as those of the wafer image super-resolution method provided in the above embodiments, and other technical features in this wafer image super-resolution device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0190] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0191] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0192] All user-related data involved in this application was obtained with the user's permission or consent, as per [reference]. Figure 9 In other words, when this application is applied to a specific product or technology, user permission is required to acquire and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations and regulatory standards of the relevant countries and regions.

[0193] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A wafer image super-resolution method, characterized in that, The method includes: Obtain an initial wafer image with a resolution of the first resolution; The initial wafer image is input into a preset super-resolution model to obtain a target wafer image with a second resolution, wherein the second resolution is higher than the first resolution. The super-resolution model is obtained by training a preset training model based on spatial features and frequency features extracted from preset wafer sample images. The spatial features include repeating structural features across regions of the wafer image, and the frequency features include high-frequency defect features of the wafer image. The extraction of spatial features is based on a preset two-dimensional selective scanning strategy, which is obtained by fusing a preset Hilbert curve scan and a preset linear scan. The Hilbert curve scan is based on the spatial position of the wafer image. The two-dimensional selection scanning strategy includes extracting Hilbert features of the one-dimensional Hilbert sequence and linear features of the one-dimensional linear scan sequence obtained by scanning, respectively; adjusting the element positions of the Hilbert features based on the spatial position to obtain Hilbert features with the same spatial position as the linear features; and adding the adjusted Hilbert features and the linear features element by element to obtain fused scan features.

2. The method as described in claim 1, characterized in that, The model to be trained includes a shallow feature extraction module, a dual-domain transformation module, and a high-resolution reconstruction module. Before the step of inputting the initial wafer image into the preset super-resolution model to obtain a target wafer image of the second resolution, the method further includes: A preset high-resolution wafer sample image is obtained, and the wafer sample image is downsampled a preset number of times to obtain a first resolution sample image corresponding to the wafer sample image. The first resolution sample image is input into the shallow feature extraction module to obtain the shallow features of the first resolution sample image; The shallow features are input into a preset dual-domain transformation module to obtain target deep features that fuse the spatial features and the frequency features; The deep features are input into the high-resolution reconstruction module to obtain a wafer reconstruction image with improved resolution; Based on the reconstructed wafer image and the wafer sample image, the parameters of the model to be trained are adjusted to obtain the super-resolution model.

3. The method as described in claim 2, characterized in that, The dual-domain transformation module includes multiple cascaded spatial frequency groups, each spatial frequency group includes multiple spatial frequency sub-modules, and each spatial frequency sub-module includes a spatial feature extraction unit and a frequency feature extraction unit. The step of inputting the shallow features into the preset dual-domain transformation module to obtain the target deep features that fuse the spatial features and the frequency features includes: The shallow features are input into the spatial feature extraction unit, and the features to be processed input into the spatial feature extraction unit are further extracted to obtain the spatial features; The spatial features are input into the frequency feature extraction unit of the same spatial frequency submodule to extract the frequency features from the spatial features, thereby obtaining the spatial frequency features. Determine whether there are any unused spatial frequency submodules in the future; If it exists, the frequency feature is input into the spatial feature extraction unit of the next spatial frequency submodule until there is no unused spatial frequency submodule, thus obtaining the target deep feature.

4. The method as described in claim 3, characterized in that, The spatial feature extraction unit includes a visual state spatial sub-unit and a multi-scale feature extraction unit. The step of further extracting features from the features to be processed input into the spatial feature extraction unit includes: The features to be processed are normalized, and the normalized features are input into the visual state space sub-unit to extract the global features of the wafer image. The global features are input into the multi-scale feature extraction unit, and the local features of the wafer image are supplemented into the global features to obtain the spatial features that fuse the global features and the local features.

5. The method as described in claim 4, characterized in that, The step of extracting global features from the wafer image to obtain the global features includes: The normalized features to be processed are processed successively based on the linear mapping layer and the activation function layer of the state space group unit to obtain the first nonlinear feature; The normalized feature to be processed is processed sequentially based on the linear mapping layer, the depthwise separable convolutional layer and the activation function layer of the state space group unit to obtain a second nonlinear feature. The first nonlinear feature and the second nonlinear feature are features obtained by processing the same feature to be processed through different processing methods. The second nonlinear feature is input into the two-dimensional selection scan layer of the state space group unit. The nonlinear feature is sequentially scanned, state feature extracted, and two-dimensional mapped to obtain a serialized feature. The state feature extraction is based on a preset state space equation, and the computational complexity of the state space equation is linear. The normalization operation is performed on the serialized features, and the normalized serialized features are multiplied element-wise with the first nonlinear feature to obtain the fused features; The global feature is obtained by adding the feature to be processed and the fused feature element by element.

6. The method as described in claim 5, characterized in that, The step of supplementing the global features with local features of the wafer image to obtain the spatial features that fuse the global features and the local features includes: The global features are normalized, and then convolved with multiple convolutional layers of different kernel sizes to obtain multiple local features. In the channel dimension of the local features, multiple local features are concatenated to obtain concatenated local features; The spliced ​​local features are nonlinearly transformed by the activation function layer, and the nonlinearly transformed spliced ​​local features are convolved based on the preset fusion convolution layer to obtain multi-scale enhanced features; The spatial features are obtained by adding the multi-scale enhanced features and the global features element by element.

7. The method as described in claim 5, characterized in that, The nonlinear feature is a two-dimensional feature. The steps of sequentially performing serialization scanning, state feature extraction, and two-dimensional mapping on the nonlinear feature to obtain the serialized feature include: Based on the scanning order corresponding to the Hilbert scan, the nonlinear feature is scanned to obtain a one-dimensional Hilbert sequence, wherein the scanning order corresponding to the Hilbert scan is determined based on the spatial position of the nonlinear feature; Based on the scanning order corresponding to the linear scan, the nonlinear feature is scanned to obtain a one-dimensional linear scan sequence, wherein the scanning order corresponding to the linear scan is a predetermined row order or column order; Based on the state-space equation, features of the Hilbert sequence and the linear sequence are extracted respectively to obtain the Hilbert features corresponding to the Hilbert sequence and the linear features corresponding to the linear sequence; Based on the spatial location, the element positions of the Hilbert feature are adjusted to obtain a Hilbert feature with the same spatial location as the linear feature; The Hilbert feature and the linear feature with the adjusted element positions are added element by element to obtain the fused scan feature; The fused scanning features are mapped in two dimensions to obtain the two-dimensional serialized features.

8. The method as described in claim 5, characterized in that, The frequency features include low-frequency features and high-frequency features. The step of extracting the frequency features from the spatial features to obtain spatial frequency features includes: The normalization operation is performed on the spatial features, and the channel dimension of the normalized spatial features is reduced based on a preset channel transformation convolutional layer to obtain reduced spatial features. The reduced spatial features are nonlinearly transformed by the activation function layer, and the nonlinearly transformed reduced spatial features are decomposed into low-frequency features and high-frequency features based on the preset discrete wavelet transform layer. The low-frequency features and the high-frequency features are input into the visual state space sub-unit to obtain the low-frequency spatial features corresponding to the low-frequency features and the high-frequency spatial features corresponding to the high-frequency features. Based on the inverse wavelet transform layer corresponding to the discrete wavelet transform layer, the low-frequency spatial features and the high-frequency spatial features are fused to obtain spatial frequency fusion features; Based on the channel transformation convolutional layer and the activation function layer, the channel dimension of the spatial frequency fusion feature is expanded to obtain the spatial frequency feature.

9. The method as described in claim 8, characterized in that, The reduced spatial features are two-dimensional features, and the high-frequency features include horizontal high-frequency features, vertical high-frequency features, and diagonal high-frequency features. The step of decomposing the nonlinearly transformed reduced spatial features into the low-frequency features and the high-frequency features based on a preset discrete wavelet transform layer includes: Based on the discrete wavelet transform layer, in the horizontal direction, the reduced spatial features are low-pass filtered and downsampled to obtain initial low-frequency features and initial high-frequency features; Based on the discrete wavelet transform layer, in the vertical direction, the initial low-frequency features are subjected to low-pass filtering and downsampling to obtain the low-frequency features and the vertical high-frequency features; Based on the discrete wavelet transform layer, in the vertical direction, the initial high-frequency features are subjected to low-pass filtering and downsampling to obtain the horizontal high-frequency features and the diagonal high-frequency features.

10. A wafer image super-resolution device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the wafer image super-resolution method as described in any one of claims 1 to 9.

Images