Wafer pattern defect intelligent detection method and system based on multi-modal data fusion

Abstract

The invention discloses a multi-modal data fusion wafer pattern defect intelligent detection method and system, and relates to the industrial vision related field, and the method comprises the steps: configuring an industrial vision imaging system; synchronously acquiring a first light path image and a second light path image of a wafer pattern area of the optical substrate; performing spatial registration and image enhancement processing to obtain a multi-modal registration image; extracting the first feature map and the second feature map, and generating a pattern multi-feature map after fusion; performing comparative analysis with a preset gold template, and positioning a pattern anomaly candidate area; and inputting the pattern multi-feature map and the pattern anomaly candidate region into a pre-trained AI visual recognition model, performing depth detection on the pattern multi-feature map under the guidance of the pattern anomaly candidate region, and outputting a defect recognition result. The technical problems that existing wafer pattern defect detection is low in recognition precision and high in missing detection and misjudgment rate are solved, and the technical effects of improving the defect recognition precision and reducing the missing detection and misjudgment rate are achieved.

Description

Technical Field

[0001] This application relates to the field of industrial vision, and in particular to a method and system for intelligent detection of wafer pattern defects using multimodal data fusion. Background Technology

[0002] Wafer pattern defect detection is a core step in ensuring the yield and reliability of optical device production, directly impacting the performance stability and lifespan of downstream optical modules. Current mainstream detection technologies rely on single-dimensional imaging, capturing wafer surface pattern information using optical imaging equipment and combining it with traditional image comparison or basic algorithms to complete defect identification and judgment. However, these technologies are limited by their single-dimensional information acquisition capabilities, making it difficult to cover the characteristic differences of defects from different causes. Furthermore, their weak anti-interference capabilities result in insufficient ability to distinguish low-contrast defects and latent defects, easily leading to missed detections and false judgments, thus failing to meet the demands of high-precision production quality control.

[0003] Currently, wafer pattern defect detection suffers from technical problems such as low recognition accuracy and high rate of missed detection and false judgment. Summary of the Invention

[0004] This application provides a method and system for intelligent detection of wafer pattern defects through multimodal data fusion. It employs an industrial vision imaging system that integrates geometric topography imaging and material / film / subsurface information imaging. This system simultaneously acquires multimodal images of the wafer pattern on an optical substrate, performs spatial registration and enhancement processing on the images to generate a multimodal registered image, extracts and fuses multimodal features to obtain a pattern multi-feature map, compares this map with a defect-free gold template to locate candidate regions for pattern anomalies, and inputs the multi-feature map and candidate regions into a pre-trained AI visual recognition model. Guided by the candidate regions, the model completes depth detection of defects and outputs the recognition results. These techniques address the technical problems of low recognition accuracy and high false negative rates in existing wafer pattern defect detection methods, achieving the technical effect of improving defect recognition accuracy and reducing false negative rates.

[0005] This application provides a method for intelligent detection of wafer pattern defects using multimodal data fusion, comprising: configuring an industrial vision imaging system, the industrial vision imaging system including at least a first imaging optical path and a second imaging optical path; synchronously acquiring a first optical path image and a second optical path image of a wafer pattern region of an optical substrate through the industrial vision imaging system, wherein a preset optical functional pattern is formed on the surface of the optical substrate wafer; performing spatial registration and image enhancement processing on the first optical path image and the second optical path image to obtain a multimodal registration image; extracting a first feature map and a second feature map based on the multimodal registration image, fusing them to generate a pattern multi-feature map; comparing and analyzing the pattern multi-feature map with a preset gold template to locate candidate regions of pattern anomalies; inputting the pattern multi-feature map and the candidate regions of pattern anomalies into a pre-trained AI visual recognition model, the AI ​​visual recognition model performing depth detection on the pattern multi-feature map under the guidance of the candidate regions of pattern anomalies, and outputting defect recognition results.

[0006] In a possible implementation, the pattern multi-feature map is compared and analyzed with a preset gold template to locate the pattern anomaly candidate region, and the following processing is performed: the difference between the pattern multi-feature map and the gold template is calculated pixel by pixel to generate a pixel difference map; the pixel difference map is decomposed into a multi-scale Gaussian pyramid to obtain multiple difference sub-maps; adaptive threshold segmentation is performed on the multiple difference sub-maps respectively, and the segmented abnormal regions are mapped back to the original image scale for merging and morphological optimization, and isolated regions with an area smaller than a preset value are removed to obtain the pattern anomaly candidate region.

[0007] In a possible implementation, the following processing is performed: the gold template is a gold template feature map, generated by inputting standard reference data into the same feature extraction and fusion process as that used to extract the pattern multi-feature map; wherein, the standard reference data is: a multimodal registration image generated by acquiring image data from a confirmed defect-free similar optical substrate wafer sample using the industrial vision imaging system; or a multimodal registration simulation image generated by calculating and rendering using an optical imaging simulation model based on the design layout data of the optical functional pattern.

[0008] In a possible implementation, the pattern multi-feature map and the pattern anomaly candidate region are input into a pre-trained AI visual recognition model. Guided by the pattern anomaly candidate region, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map, outputs a defect recognition result, and performs the following processing: The AI ​​visual recognition model is a Siamese network based on an encoder-decoder architecture and integrates a spatial attention guidance module; the spatial attention guidance module uses the pattern anomaly candidate region as a spatial attention mask to modulate the input pattern multi-feature map, enhancing the feature response within the candidate region and suppressing the feature response within the non-candidate region, generating an attention-modulated feature map; the first branch of the Siamese network uses the local feature block corresponding to the candidate region in the attention-modulated feature map as input to extract the local detail features of the defect; the second branch of the Siamese network uses the entire attention-modulated feature map as input to extract global context features containing the relationship between the candidate region and the surrounding pattern; the feature fusion layer of the Siamese network fuses the local detail features and the global context features, performs defect semantic segmentation and classification based on the fused features, and outputs the defect recognition result.

[0009] In a possible implementation, the following processing is performed: the first imaging optical path is a bright-field or dark-field microscopic imaging optical path for acquiring high-contrast two-dimensional geometric morphology of the pattern; the second imaging optical path is a spectral imaging optical path for acquiring the material composition of the pattern region, a polarization imaging optical path for acquiring the film properties of the pattern region, or a confocal imaging optical path for acquiring subsurface information of the pattern region.

[0010] In a possible implementation, spatial registration and image enhancement processing are performed on the first and second optical path images to obtain a multimodal registered image. The following processing is then performed: a first feature point set is extracted from the first optical path image, and a second feature point set is extracted from the second optical path image. The first and second feature point sets are matched using feature descriptors to obtain multiple pairs of matching feature point pairs. Correct matching pairs are selected from the multiple pairs of matching feature point pairs, and a spatial geometric transformation matrix from the second optical path image to the first optical path image is calculated using the correct matching pairs. The spatial geometric transformation matrix is ​​applied to register the second optical path image to generate a second optical path registered image. Denoising and adaptive contrast enhancement processing are performed on the first and second optical path registered images respectively to obtain a first enhanced optical path image and a second enhanced optical path image. The first enhanced optical path image and the second enhanced optical path image are stitched together along the channel dimension, and intermodal normalization processing is performed on the stitched multi-channel image to generate the multimodal registered image.

[0011] In a possible implementation, the following processing is performed: the intermodal normalization processing is Z-score normalization processing.

[0012] In a possible implementation, based on the multimodal registration image, a first feature map and a second feature map are extracted, fused, and a patterned multi-feature map is generated. The following processing is performed: the first modality image data and the second modality image data in the multimodal registration image are processed respectively through a feature extraction network to obtain the first feature map and the second feature map; the first feature map and the second feature map are concatenated along the channel dimension and then subjected to global average pooling to obtain multiple global description vectors, wherein each channel corresponds to one global description vector; the multiple global description vectors are input into a multilayer perceptron to learn and output multiple weight coefficients; the multiple weight coefficients are normalized using the Softmax function to obtain multiple channel attention weights; the corresponding channels of the first feature map and the second feature map are weighted using the multiple channel attention weights, and then the channels are merged to generate the patterned multi-feature map.

[0013] In a possible implementation, the following processing is also performed: statistical analysis of the defect identification results for the same batch to generate a defect distribution map; based on the defect distribution map, locating the source of deviation in the patterning process; and generating parameter control suggestions based on the source of deviation.

[0014] This application also provides a multimodal data fusion-based intelligent detection system for wafer pattern defects, comprising: an industrial vision imaging system configuration module for configuring an industrial vision imaging system, wherein the industrial vision imaging system includes at least a first imaging optical path and a second imaging optical path; an optical path image acquisition module for simultaneously acquiring a first optical path image and a second optical path image of an optical substrate wafer pattern region through the industrial vision imaging system, wherein a preset optical functional pattern is formed on the surface of the optical substrate wafer; an image registration module for performing spatial registration and image enhancement processing on the first optical path image and the second optical path image to obtain a multimodal registration image; a feature extraction and fusion module for extracting a first feature map and a second feature map based on the multimodal registration image, and fusing them to generate a pattern multi-feature map; a pattern anomaly candidate region localization module for comparing and analyzing the pattern multi-feature map with a preset gold template to locate pattern anomaly candidate regions; and a defect recognition module for inputting the pattern multi-feature map and the pattern anomaly candidate regions into a pre-trained AI visual recognition model, wherein the AI ​​visual recognition model, guided by the pattern anomaly candidate regions, performs depth detection on the pattern multi-feature map and outputs a defect recognition result.

[0015] The proposed intelligent detection method and system for wafer pattern defects using multimodal data fusion, as described in this application, first configures an industrial vision imaging system, which includes at least a first imaging optical path and a second imaging optical path. Then, the industrial vision imaging system simultaneously acquires first and second optical path images of the wafer pattern region on an optical substrate. The optical substrate wafer surface has a preset optical functional pattern. Next, spatial registration and image enhancement processing are performed on the first and second optical path images to obtain a multimodal registration image. Based on the multimodal registration image, a first feature map and a second feature map are extracted and fused to generate a pattern multi-feature map. This pattern multi-feature map is then compared and analyzed with a preset gold template to locate candidate regions for pattern anomalies. Finally, the pattern multi-feature map and the candidate regions for pattern anomalies are input into a pre-trained AI visual recognition model. Guided by the candidate regions for pattern anomalies, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map and outputs the defect identification result. Through the above process, the method and system proposed in this application achieve the technical effect of improving defect identification accuracy and reducing the rate of missed detections and false positives. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a flowchart illustrating the intelligent detection method for wafer pattern defects using multimodal data fusion provided in this application embodiment.

[0018] Figure 2 This is a schematic diagram of the structure of the intelligent detection system for wafer pattern defects based on multimodal data fusion provided in an embodiment of this application.

[0019] Figure labeling: Industrial vision imaging system configuration module 10, optical path image acquisition module 20, image registration module 30, feature extraction and fusion module 40, pattern anomaly candidate region localization module 50, defect recognition module 60. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0021] This application provides an intelligent detection method for wafer pattern defects based on multimodal data fusion, such as... Figure 1 As shown, the method includes:

[0022] Step S100: Configure an industrial vision imaging system, which includes at least a first imaging optical path and a second imaging optical path. The first imaging optical path is a bright-field or dark-field microscopic imaging optical path for acquiring high-contrast two-dimensional geometric morphology of a pattern; the second imaging optical path is a spectral imaging optical path for acquiring the material composition of the pattern region, a polarization imaging optical path for acquiring film properties of the pattern region, or a confocal imaging optical path for acquiring subsurface information of the pattern region.

[0023] Specifically, an industrial vision imaging system is constructed, comprising at least two imaging optical paths, each undertaking a different imaging task. The first optical path, the first imaging optical path, is used to acquire high-contrast two-dimensional geometric morphology of the wafer pattern. It employs either bright-field or dark-field microscopy imaging. In bright-field microscopy, light travels directly along the optical axis onto the wafer, and imaging through reflected light clearly reveals the two-dimensional geometric contours of the pattern with high contrast. In dark-field microscopy, light travels obliquely onto the wafer, with only scattered light participating in imaging, highlighting pattern edges and subtle protrusions / depressions. The second optical path, the second imaging optical path, is used to acquire supplementary information about the pattern region beyond its geometric morphology. It can select one or more of three types: spectral imaging for acquiring material composition, polarization imaging for acquiring film properties, and confocal imaging for acquiring subsurface information. Spectral imaging determines material composition by analyzing the wavelength distribution of light; polarization imaging uses the polarization characteristics of light to reflect film thickness, refractive index, etc.; and confocal imaging acquires structural information below the sample surface by focusing at different depths.

[0024] Based on the detection requirements, the corresponding optical path combination is selected, and the core parameters of the optical path are determined to ensure the stability and consistency of synchronous imaging of the two optical paths. The core components used include a microscope objective, a light source, a detector, and an optical path switching module. For example, a bright-field optical path is selected as the first imaging optical path, equipped with a 20x infinity-corrected microscope objective. The light source is a white LED surface light source with a brightness adjustment range of 500-2000 lumens. The detector is a CMOS camera with a resolution of 2048×2048 and a frame rate of 30 frames per second. The second imaging optical path is a spectral imaging optical path, equipped with a hyperspectral camera with a spectral range of 400-1000 nanometers and a spectral resolution of 5 nanometers. It is connected to the first optical path via a synchronous trigger line to ensure that the imaging trigger time difference is less than 1 millisecond. Simultaneously, through optical platform calibration, the overlap of the imaging areas of the two optical paths is ≥99%.

[0025] Step S200: The industrial vision imaging system synchronously acquires a first optical path image and a second optical path image of the optical substrate wafer pattern area, wherein a preset optical functional pattern is formed on the surface of the optical substrate wafer.

[0026] Specifically, optical substrate wafers are base materials used to manufacture optical devices. They can be silicon wafers, glass substrates, etc. The surface needs to be processed with preset patterns to achieve specific optical functions. Preset optical function patterns are specific structures designed according to device requirements, such as photoresist patterns used in photolithography processes and pixel array patterns used in display devices.

[0027] The wafer position is fixed, and a dual-optical-path synchronous triggering mechanism is activated to acquire images and perform preliminary storage and format standardization. For example, the optical substrate wafer is fixed on a high-precision motorized platform using vacuum adsorption. The platform's positioning accuracy is ±0.1 micrometers. A motion control card drives the platform's movement, achieving full-area wafer scanning imaging. The image acquisition card uses a PCIe interface, supporting dual-channel synchronous acquisition, and converts the acquired image data into 16-bit TIFF format to avoid data loss. The storage server uses a RAID5 array with read / write speeds ≥500MB / s, ensuring real-time storage of massive amounts of image data. A unique identifier is added to each pair of images, including the wafer number, acquisition location, and timestamp, for easy association processing.

[0028] Step S300: Spatial registration and image enhancement processing are performed on the first optical path image and the second optical path image to obtain a multimodal registered image.

[0029] Specifically, due to the different imaging principles of dual-path imaging, the two sets of images acquired may have differences such as spatial offset and scaling. Therefore, spatial registration must first be performed through feature point matching to align corresponding regions. This involves adjusting the spatial relationship between the two sets of images so that the pixel positions of the same physical region correspond consistently in both sets. Then, denoising and contrast enhancement processes are used to improve image clarity and contrast, suppress noise, and highlight effective features, ultimately generating a multimodal registered image to prepare for feature extraction.

[0030] In one possible implementation, spatial registration and image enhancement processing are performed on the first and second optical path images to obtain a multimodal registered image. Step S300 further includes step S310, extracting a first feature point set from the first optical path image and a second feature point set from the second optical path image, and matching the first and second feature point sets using feature descriptors to obtain multiple pairs of matching feature points. Specifically, a robust feature extraction algorithm is selected to generate feature points and descriptors, including algorithms such as SIFT, SURF, and ORB. For example, the ORB algorithm is used to extract feature points, setting the maximum number of feature points to 5000, the number of pyramid layers to 8, and the scale factor of each layer to 1.2. The generated ORB descriptor is a 256-bit binary vector. The feature point set is a collection of key points in the image that possess uniqueness and stability. Each feature point contains information such as position and grayscale changes. The feature descriptor is a vector formed by encoding the grayscale and texture information around the feature point, used to characterize the uniqueness of the feature point and achieve cross-image matching. Feature point matching is performed using a distance metric. For example, the Hamming distance between two descriptors is calculated during matching. A Hamming distance threshold of 30 is set. When the Hamming distance between two descriptors is less than the threshold, they are determined to be a matching feature point pair, that is, a combination of feature points corresponding to the same physical location in two sets of images.

[0031] Step S320: Select correct matching pairs from the multiple pairs of matching feature point pairs, and calculate the spatial geometric transformation matrix from the second optical path image to the first optical path image using the correct matching pairs. Specifically, since there may be mismatched pairs during feature point matching, a random sampling consensus algorithm is used to select correct matching pairs, that is, feature point pairs that accurately correspond to the same physical location, eliminating mismatches caused by noise or texture similarity. Based on the coordinate relationship of the correct matching pairs, the spatial geometric transformation matrix from the second optical path image to the first optical path image, that is, the affine transformation matrix, is calculated using the least squares method. This matrix can be used to convert the pixel coordinates of the second optical path image to the corresponding coordinates of the first optical path image, thereby achieving position correction of the second optical path image.

[0032] Step S330: Apply the spatial geometric transformation matrix to register the second optical path image, generating a second optical path registered image. Specifically, the spatial geometric transformation matrix calculated in step S320 is applied to the second optical path image, and an interpolation algorithm is used to resample the pixels of the second optical path image, so that the spatial positions of the second optical path image and the first optical path image are completely aligned, generating a second optical path registered image. For example, a bicubic interpolation algorithm is used to register the second optical path image. The kernel function of the bicubic interpolation is a cubic B-spline function, and the interpolation window size is 4×4. During the processing, for each pixel coordinate of the second optical path image, its corresponding coordinate in the first optical path image is calculated through an affine transformation matrix. If the corresponding coordinate is a non-integer coordinate, the pixel value at that position is calculated through bicubic interpolation, finally generating a second optical path registered image with the same resolution as the first optical path image.

[0033] Step S340: Denoising and adaptive contrast enhancement processing are performed on the first optical path image and the second optical path registration image, respectively, to obtain a first enhanced optical path image and a second enhanced optical path image. Specifically, a corresponding denoising algorithm is selected according to the type of image noise, and denoising processing is performed on the first optical path image and the second optical path registration image to eliminate noise generated during the imaging process, such as electronic noise and ambient light noise. For example, for Gaussian noise in the first optical path image, bilateral filtering is used for denoising, with a spatial domain standard deviation of 2 and a gray-level domain standard deviation of 50, which can remove noise while preserving edge details; for salt-and-pepper noise in the second optical path image, median filtering is used for denoising, with a window size of 3×3. Then, an adaptive histogram equalization algorithm is used to adjust the contrast according to the gray-level distribution characteristics of different regions of the image, improving the clarity of details in the image and making defective areas easier to identify, ultimately obtaining two sets of enhanced images. For example, the contrast enhancement uses the CLAHE algorithm, which divides the image into 8×8 sub-blocks, calculates the histogram for each sub-block independently, and sets the contrast limit threshold to 2.0 to avoid artifacts caused by over-enhancement.

[0034] Step S350: The first enhanced optical path image and the second enhanced optical path image are stitched together along the channel dimension, and inter-modal normalization processing is performed on the stitched multi-channel image to generate the multi-modal registration image. The inter-modal normalization processing is Z-score normalization. Specifically, the two sets of enhanced images are stitched together along the channel dimension to form a multi-channel image, such as the first enhanced optical path image being channel 1 and the second enhanced optical path image being channel 2, with each channel corresponding to one modal information. Then, Z-score normalization is used to eliminate the magnitude differences between different modal data, ensuring that the data in each channel are under the same standard, thus generating the multi-modal registration image. Z-score normalization is a data normalization method that converts the original data into standardized data with a mean of 0 and a standard deviation of 1 by calculating the mean and standard deviation of the data. The formula is: Standardized pixel value = (Original pixel value - Channel mean) / Channel standard deviation.

[0035] Step S400: Based on the multimodal registration image, extract the first feature map and the second feature map, and fuse them to generate a patterned multi-feature map.

[0036] Specifically, the multimodal registration image contains two channels of information after being stitched together from two single-channel images. A feature extraction network is used to mine the deep semantic information of each channel, representing features such as pattern edges, textures, and material distribution in the image. The feature extraction network is built on a convolutional neural network, extracting shallow and deep features layer by layer through operations such as convolution and pooling. For example, ResNet18 is used as the feature extraction network to process the two-channel data separately, obtaining the first and second feature maps corresponding to the two modalities. A channel attention mechanism is then introduced, using global average pooling and a multilayer perceptron to learn the importance of each channel's features, assigning weight coefficients to enhance the response of important feature channels and suppress interference in redundant channels. Finally, the weighted feature maps are fused to generate a pattern multi-feature map containing more comprehensive defect recognition information, improving the accuracy of subsequent detection.

[0037] In one possible implementation, based on the multimodal registration image, a first feature map and a second feature map are extracted and fused to generate a patterned multi-feature map. Step S400 further includes step S410, where a feature extraction network processes the first modality image data and the second modality image data in the multimodal registration image respectively to obtain the first feature map and the second feature map. Specifically, the feature extraction network independently extracts deep features based on the characteristics of the two modalities, ensuring that key information of each modality is not lost. The two channels of the multimodal registration image carry different information: channel 1 focuses on geometric shape, and channel 2 focuses on supplementary features. Through a feature extraction network with the same structure but independently trained parameters, the two data streams are forward-propagated separately. Local features are extracted through convolution operations, and key information is retained through pooling operations to reduce dimensionality. Features are obtained layer by layer from shallow to deep, ultimately generating a first feature map and a second feature map with consistent dimensions and size. For example, ResNet18 is selected as the feature extraction network, the last fully connected layer of the network is removed, and five convolutional blocks and pooling layers are retained. Two network branches with independent parameters are constructed for the two modalities. Channel 1 data from the multimodal registration image is input into the first branch. After four 2×2 max pooling operations, and with ReLU activation applied to each convolutional layer, the first feature map is extracted. Similarly, channel 2 data is input into the second branch and processed using the same network structure to extract the second feature map of the same size. The network is initialized using a He normal distribution, with the standard deviation of the convolutional layer weights initialized to 0.01 and the biases initialized to 0, ensuring stable convergence of parameters during training.

[0038] Step S420: After concatenating the first feature map and the second feature map along the channel dimension, global average pooling is performed to obtain multiple global description vectors, where each channel corresponds to one global description vector. Specifically, the first feature map and the second feature map are concatenated along the channel dimension to form a higher-dimensional feature map, integrating all deep features of the two modalities. The global average pooling operation calculates the average value of all pixel values ​​in each channel of the concatenated feature map, compressing the two-dimensional feature map into a one-dimensional vector. This retains the global feature information of each channel, reduces the number of parameters, avoids overfitting, and simultaneously achieves global information fusion, providing a feature basis for attention weight learning. For example, concatenating the 256×256×64 first feature map and the 256×256×64 second feature map along the channel dimension yields a concatenated feature map of size 256×256×128, where channels 1-64 correspond to the original first feature map, and channels 65-128 correspond to the original second feature map. Global average pooling is performed on each of the 128 channels of the stitched feature map to calculate the average value of 256×256 pixels for each channel. A one-dimensional vector value is generated for each channel, and finally a set of 128-dimensional global description vectors is formed to fully represent the global features of the stitched feature map.

[0039] Step S430: Input the multiple global description vectors into a multilayer perceptron (MLP) to learn and output multiple weight coefficients. Specifically, the global description vector set is input into the MLP, which consists of an input layer, hidden layers, and an output layer. A fully connected layer performs a nonlinear transformation of the global description vectors, mining the correlation between the features of each channel and the defect detection task, and adaptively allocating weight coefficients. The magnitude of the weight coefficient represents the contribution of the corresponding channel features to defect identification. Channels with high contribution, such as those containing defect edges or material anomalies, receive larger weights, while redundant channels with low contribution receive smaller weights. For example, a two-layer MLP can be constructed: the input layer has 128 neurons, corresponding to a 128-dimensional global description vector; the hidden layer has 32 neurons, using ReLU activation to achieve nonlinear feature transformation; and the output layer has 128 neurons, corresponding to the weight coefficients of the 128 channels, without an activation function to ensure that the value range of the weight coefficients is suitable for subsequent normalization operations. The 128-dimensional global description vector is input into the MLP, and after processing by the hidden and output layers, 128 weight coefficients are output. During training, the defect recognition accuracy is used as the loss function. The parameters of the multilayer perceptron are updated iteratively through the Adam optimizer, with 500 iterations and a batch size of 32, so that the weight coefficients can accurately reflect the importance of the features of each channel.

[0040] Step S440: The multiple weight coefficients are normalized using the Softmax function to obtain multiple channel attention weights. Specifically, the weight coefficients output by the multilayer perceptron have a variable value range and cannot be directly used for feature weighting. The Softmax function converts the weight coefficients into a probability distribution form, with each weight coefficient ranging from 0 to 1, and the sum of all coefficients being 1, thus standardizing the weights. The standardized channel attention weights can intuitively reflect the relative importance of each channel, ensuring a reasonable proportion of features in each channel during the weighting process and avoiding excessive dominance of a single channel feature in the fusion result.

[0041] Step S450: Multiple channel attention weights are used to weight corresponding channels of the first feature map and the second feature map, and then the channels are merged to generate the patterned multi-feature map. Specifically, the channel attention weights are multiplied pixel-by-pixel with the corresponding channels of the first and second feature maps to enhance the response of important feature channels and suppress interference from redundant feature channels. The weighted feature maps are then merged to generate a patterned multi-feature map that integrates the core features of both modalities. For example, the 64 channels of the first feature map are multiplied pixel-by-pixel with the first 64 attention weights, and the 64 channels of the second feature map are multiplied pixel-by-pixel with the last 64 attention weights; that is, the pixel value of a certain channel in the feature map = the original pixel value × the corresponding attention weight. After weighting, the two feature maps are concatenated along the channel dimension to obtain a 256×256×128 feature map. The number of channels is compressed to 64 through a 1×1 convolutional layer, which retains the core features and reduces the data dimension. Finally, a 256×256×64 patterned multi-feature map is generated, ensuring the non-linear expressive power of the features and the efficiency of subsequent processing.

[0042] Step S500: Compare and analyze the multi-feature map of the pattern with the preset gold template to locate the candidate region of pattern abnormality.

[0043] Specifically, the gold template is a standard template for characterizing the pattern features of defect-free wafers. Its generation process is completely consistent with that of the pattern multi-feature map. Both have the same dimensions and scale, making them comparable, and can be generated from actual defect-free samples or simulation data. The difference between the pattern multi-feature map and the gold template is calculated pixel-by-pixel. The larger the difference value, the higher the probability of a defect in the corresponding area, thus generating a pixel difference map. Then, multi-scale Gaussian pyramid decomposition is used to enhance the defect difference signals of different sizes. Anomaly regions are marked by adaptive threshold segmentation. Finally, morphological operations are used to optimize the region morphology, eliminating false anomaly regions to obtain candidate pattern anomaly regions, which are the focus range for depth detection.

[0044] In one possible implementation, the pattern multi-feature map is compared and analyzed with a preset gold template to locate candidate regions for pattern anomalies. Step S500 further includes step S510, calculating the difference between the pattern multi-feature map and the gold template pixel by pixel to generate a pixel difference map. The gold template is a gold template feature map, generated by inputting standard reference data into the same feature extraction and fusion process as that used to extract the pattern multi-feature map. The standard reference data can be: a multimodal registration image generated from image data acquired using the industrial vision imaging system from a confirmed defect-free optical substrate wafer sample; or a multimodal registration simulation image generated by calculating and rendering using an optical imaging simulation model based on the design layout data of the optical functional pattern. Specifically, to first generate the gold template feature map, which is a standardized defect-free feature map, its generation process is completely consistent with that of the pattern multi-feature map, ensuring the same feature dimensions and scale. The standard reference data is defect-free data used to generate the gold template feature map, including both actual defect-free sample images and simulation-generated images. For example, the gold template generation uses actual defect-free sample data. Ten identical wafers, confirmed to be defect-free by manual inspection, are selected. Images are acquired and processed according to steps S100-S400 to generate multimodal registration images. Feature maps are then extracted and fused to generate feature maps. The average of the feature maps from the ten wafers is taken to obtain the gold template. A suitable difference measurement method is then used to calculate pixel-by-pixel differences, such as Euclidean distance, absolute difference, or cosine similarity. The difference values ​​are used as the grayscale values ​​of the corresponding pixels to generate a pixel difference map.

[0045] Step S520: Perform multi-scale Gaussian pyramid decomposition on the pixel difference map to obtain multiple difference sub-images. Specifically, a Gaussian pyramid is constructed, and the pixel difference map is decomposed into difference sub-images of different scales through multiple Gaussian filtering and downsampling. Each sub-image corresponds to defect features of different sizes, such as small-scale sub-images corresponding to tiny defects and large-scale sub-images corresponding to larger defects, achieving full coverage detection of defects of different sizes. For example, a 4-layer Gaussian pyramid decomposition is constructed, with the original pixel difference map being layer 0 of the pyramid, with a resolution of 256×256. Layer 1: Gaussian filtering (standard deviation 1) is applied to layer 0, followed by a 2x downsampling to obtain a difference sub-image with a resolution of 128×128. Layer 2: Gaussian filtering (standard deviation 2) is applied to layer 1, followed by a 2x downsampling to obtain a 64×64 difference sub-image. Layer 3: Gaussian filtering (standard deviation 4) is applied to layer 2, followed by a 2x downsampling to obtain a 32×32 difference sub-image. Layer 4: Apply Gaussian filtering (standard deviation 8) to layer 3, and downsample by 2 times to obtain a 16×16 difference submap.

[0046] Step S530: Adaptive threshold segmentation is performed on the multiple difference sub-images respectively, and the segmented abnormal regions are mapped back to the original image scale for merging and morphological optimization. Isolated regions with areas smaller than a preset value are removed to obtain the pattern anomaly candidate regions. Specifically, adaptive threshold segmentation is performed on each difference sub-image, that is, the segmentation threshold is automatically determined according to the local gray-level distribution of the image to adapt to the situation of uneven gray-level in the image, and regions with difference values ​​higher than the threshold are marked as abnormal regions. Then, the segmented binary images are upsampled and mapped back to the original image scale, merged, and morphological opening and closing operations are performed to adjust the region shape, eliminate noise interference and holes in the region, make the outline of the abnormal region more regular, and remove small-area regions, that is, false abnormal regions with too small an area caused by noise, to obtain the final pattern anomaly candidate regions. For example, Otsu adaptive threshold segmentation is used on the four difference sub-images respectively to automatically determine the optimal segmentation threshold for each sub-image. Regions with difference values ​​higher than the threshold are marked as white to indicate anomalies, and regions with difference values ​​lower than the threshold are marked as black to indicate normal, resulting in a binary anomaly image. By using bilinear interpolation upsampling, the binarized anomaly maps of each sub-image are mapped back to the original 256×256 image scale. The anomaly regions of all sub-images are then merged using a logical OR operation. Morphological optimization is performed on the merged binary image: first, a 5×5 structuring element is used for closing operations to fill holes within the anomaly regions; then, a 3×3 structuring element is used for opening operations to remove edge burrs. Finally, an area threshold of 10 pixels is set to remove isolated regions with an area smaller than 10 pixels, resulting in candidate anomaly regions. These candidate regions are marked with rectangular bounding boxes containing region coordinates and area information.

[0047] Step S600: Input the pattern multi-feature map and the pattern anomaly candidate region into the pre-trained AI visual recognition model. Guided by the pattern anomaly candidate region, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map and outputs the defect recognition result.

[0048] Specifically, the multi-feature map of the pattern and candidate regions for pattern anomalies are input into a pre-trained AI visual recognition model. This pre-trained AI visual recognition model is trained on a large amount of labeled data and has defect recognition capabilities. It can be adapted to specific wafer defect detection scenarios through transfer learning. Guided by the candidate regions, the model focuses on areas where defects may exist and, combined with global feature information, performs deep defect detection, ultimately outputting defect recognition results, including defect location, type, and level.

[0049] In one possible implementation, the pattern multi-feature map and the pattern anomaly candidate region are input into a pre-trained AI visual recognition model. Guided by the pattern anomaly candidate region, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map and outputs a defect recognition result. Step S600 further includes step S610, where the AI ​​visual recognition model is a Siamese network based on an encoder-decoder architecture and integrates a spatial attention guidance module. Specifically, the AI ​​visual recognition model adopts a Siamese network based on an encoder-decoder architecture. The encoder-decoder architecture is a type of neural network structure. The encoder extracts deep features through downsampling and is responsible for feature extraction. The decoder restores the feature map to the original image scale through upsampling and is responsible for feature reconstruction and defect localization, achieving pixel-level prediction. The Siamese network contains two branch networks with identical structures and shared parameters, which can process two inputs simultaneously, extracting local and global features respectively. Simultaneously, the AI ​​visual recognition model integrates a spatial attention guidance module, which modulates the feature map through a spatial mask to enhance the feature response of the candidate region and suppress interference from irrelevant regions. For example, the Siamese network contains two structurally identical encoder branches. The encoder uses ResNet34 and contains 5 convolutional blocks, each followed by a 2×2 max pooling operation, outputting a global feature map of size 8×8×512. The decoder uses a U-Net architecture and contains 4 upsampling modules. Each upsampling module enlarges the feature map size by a factor of 2 through transposed convolutions and concatenates it with the feature map of the corresponding layer in the encoder. A spatial attention guidance module is embedded in the first layer of the decoder. It takes an anomaly candidate region mask as input, maps the mask to an attention weight map with the same size as the feature map through a 1×1 convolution, and then multiplies it pixel-by-pixel with the decoder's input feature map to achieve feature modulation. The network is initialized using a He normal distribution, with the standard deviation of the convolutional layer weights initialized to 0.01 and the biases initialized to 0.

[0050] In step S620, the spatial attention guidance module uses the pattern anomaly candidate region as a spatial attention mask to modulate the input pattern multi-feature map, enhancing the feature response within the candidate region and suppressing the feature response of non-candidate regions, thereby generating an attention-modulated feature map. Specifically, the spatial attention guidance module uses the pattern anomaly candidate region as a spatial attention mask, which is a binary map or a weight map, used to mark the spatial regions that need to be focused on. The mask size is adjusted to match the feature map through convolution, and then multiplied pixel-by-pixel with the feature map, causing the model to focus on the anomaly candidate region, enhancing the feature response intensity of that region, and simultaneously suppressing the feature response of non-candidate regions, generating an attention-modulated feature map and improving the targeting of defect detection. For example, the rectangular bounding box of the pattern anomaly candidate region is converted into a binary mask, with pixel values ​​of 1 within the candidate region and 0 outside the region. A 1×1 convolutional layer maps the binary mask to a weight map with the same number of channels as the pattern multi-feature map. The convolutional layer has no activation function, ensuring that the mask information is not distorted. The weight map is multiplied pixel by pixel with the pattern multi-feature map, that is, the pixel value at a certain position of the feature map = the original pixel value × the corresponding weight map pixel value. This makes the features within the candidate region preserved or enhanced, while the features outside the region are suppressed, generating an attention-modulated feature map that effectively focuses on the features of the candidate region.

[0051] In step S630, the first branch of the Siamese network takes the local feature block corresponding to the candidate region in the attention modulation feature map as input to extract the local detail features of the defect. The second branch of the Siamese network takes the entire attention modulation feature map as input to extract the global context features containing the relationship between the candidate region and the surrounding pattern. Specifically, the first branch of the Siamese network, i.e., the local branch, takes the local feature block corresponding to the candidate region in the attention modulation feature map as input, that is, it performs candidate region cropping on the attention modulation feature map to obtain the local feature block input to the first branch, focusing on extracting the local detail features of the defect, such as defect edges, shape, and texture. The second branch, i.e., the global branch, takes the entire attention modulation feature map as input to extract the global context features containing the relationship between the candidate region and the surrounding pattern, such as the position of the candidate region in the overall pattern and its correlation with the surrounding structure, achieving complementarity between local and global features. Among them, the local detail features are features that characterize the local structure and morphology of the defect itself, used to accurately identify the defect type; the global context features are features that characterize the relationship between the defect and the surrounding environment, used to eliminate false defects and accurately locate the defect position.

[0052] In step S640, the feature fusion layer of the Siamese network fuses the local detail features and the global context features, performs defect semantic segmentation and classification based on the fused features, and outputs the defect recognition result. Specifically, the feature fusion layer of the Siamese network performs size alignment and concatenation fusion of local and global features, compresses the number of channels through convolutional layers, and fuses local detail features and global context features, combining the advantages of both types of features. Semantic segmentation branches and classification branches are constructed respectively. The location and range of the defect are determined through defect semantic segmentation, the defect type is determined through a classifier, and finally a complete defect recognition result is output. For example, the local feature map output by the first branch (32×32×64) is adjusted to 64×64×64 through transpose convolution, and concatenated with the global feature map output by the second branch (64×64×64) along the channel dimension to obtain a 64×64×128 fused feature map. The number of channels of the fused feature map is compressed to 64 through a 1×1 convolutional layer to obtain the final 64×64×64 fused feature map. The semantic segmentation branch uses a U-Net decoder to upsample the feature map to 256×256 and outputs a binary segmentation map through a softmax function, with the defect region represented as 1 and the background as 0, accurately locating the defect's range. The classification branch uses two fully connected layers and outputs the defect type probability distribution through a softmax function, selecting the type with the highest probability as the final classification result. It also outputs the defect's bounding box coordinates, area, and confidence score, forming a complete defect recognition result.

[0053] In one possible implementation, the method further includes step S700, which involves statistically analyzing the defect identification results of the same batch to generate a defect distribution map. Specifically, defect data from the same batch of wafers is collected, classified and statistically analyzed according to defect type and location, and the quantity, type, location, and size of defects are statistically analyzed. Based on the spatial coordinates of the wafers, a defect distribution map is generated to visually present the distribution pattern of defects within the batch. For example, defect identification results from 50 wafers in the same batch are collected, and a two-dimensional coordinate system is established for each wafer to record the coordinates, type, area, and confidence level of each defect. Defects are classified into three types: scratches, corner defects, and material inhomogeneity. The total number, average area, and distribution density of each type of defect are statistically analyzed. A defect distribution heatmap is plotted using Matplotlib, with the wafer coordinates as the horizontal and vertical axes. The color intensity of the heatmap represents the defect density, and a scatter plot is superimposed, with different colored scatter points representing different defect types and the scatter point size corresponding to the defect area. The generated defect distribution map includes batch information, statistical data, and visualization graphics, presenting defect concentration areas and dominant defect types.

[0054] Step S800: Based on the defect distribution map, locate the source of deviation in the patterning process. Specifically, combining the patterns in the defect distribution map, correlate each stage of the wafer patterning process, such as photolithography, development, etching, coating, and stripping, and analyze the correlation between defect distribution and process parameters using the controlled variable method to locate the source of process deviations causing defects, such as photolithography exposure dose deviation, insufficient etching time, and uneven coating thickness. For example, based on the defect distribution map, it was found that scratch defects are concentrated in the wafer edge area and distributed along the wafer rotation direction. Corresponding to the wafer patterning process stages, edge defects are mostly related to wafer clamping during the development and etching stages after photolithography. Tracing the etching equipment parameters of this batch of wafers, it was found that the clamping device pressure was uneven. Through comparative experiments, 10 wafers were selected for production using the adjusted clamping pressure, while other parameters remained unchanged. As a result, edge scratch defects were reduced by more than 90%, confirming that the source of process deviation was the uneven clamping device pressure during the etching stage.

[0055] Step S900: Generate parameter control recommendations based on the sources of deviation. Specifically, based on the identified sources of process deviation, combined with process principles and actual production conditions, determine the direction and range of parameter adjustment, formulate targeted parameter control recommendations, adjust parameters of relevant process steps, optimize equipment status or operating procedures, reduce defects in subsequent production, and improve wafer production quality.

[0056] This application employs an industrial vision imaging system that includes a geometric topography imaging optical path and a material / film / subsurface information imaging optical path. It simultaneously acquires multimodal images of optical substrate wafer patterns, performs spatial registration and enhancement processing on the images to generate multimodal registered images, extracts and fuses multimodal features to obtain a pattern multi-feature map, compares it with a defect-free gold template to locate candidate regions for pattern anomalies, and inputs the multi-feature map and candidate regions into a pre-trained AI visual recognition model. Guided by the candidate regions, it completes defect depth detection and outputs recognition results. These techniques solve the technical problems of low recognition accuracy and high false negative rate in existing wafer pattern defect detection, achieving the technical effect of improving defect recognition accuracy and reducing false negative rate.

[0057] In the above text, refer to Figure 1 A method for intelligent detection of wafer pattern defects based on multimodal data fusion according to embodiments of the present invention is described in detail. Next, reference will be made to... Figure 2 A wafer pattern defect intelligent detection system based on multimodal data fusion according to an embodiment of the present invention is described.

[0058] The intelligent wafer pattern defect detection system based on multimodal data fusion according to embodiments of the present invention addresses the technical problems of low recognition accuracy and high false negative rate in existing wafer pattern defect detection methods, achieving the technical effect of improving defect recognition accuracy and reducing false negative rate. The intelligent wafer pattern defect detection system based on multimodal data fusion includes: an industrial vision imaging system configuration module 10, an optical path image acquisition module 20, an image registration module 30, a feature extraction and fusion module 40, a pattern anomaly candidate region localization module 50, and a defect recognition module 60.

[0059] An industrial vision imaging system configuration module 10 is used to configure an industrial vision imaging system, which includes at least a first imaging optical path and a second imaging optical path; an optical path image acquisition module 20 is used to simultaneously acquire a first optical path image and a second optical path image of an optical substrate wafer pattern area through the industrial vision imaging system, wherein a preset optical functional pattern is formed on the surface of the optical substrate wafer; an image registration module 30 is used to perform spatial registration and image enhancement processing on the first optical path image and the second optical path image to obtain a multimodal registration image; a feature extraction and fusion module 40 is used to extract a first feature map and a second feature map based on the multimodal registration image, and fuse them to generate a pattern multi-feature map; a pattern anomaly candidate region localization module 50 is used to compare and analyze the pattern multi-feature map with a preset gold template to locate the pattern anomaly candidate region; and a defect recognition module 60 is used to input the pattern multi-feature map and the pattern anomaly candidate region into a pre-trained AI vision recognition model, wherein the AI ​​vision recognition model, guided by the pattern anomaly candidate region, performs depth detection on the pattern multi-feature map and outputs a defect recognition result.

[0060] The detailed description of the specific configuration of the pattern anomaly candidate region localization module 50 is explained as follows: As mentioned above, the pattern multi-feature map is compared and analyzed with a preset golden template to locate the pattern anomaly candidate region. The pattern anomaly candidate region localization module 50 may further include: a difference calculation unit for calculating the difference between the pattern multi-feature map and the golden template pixel by pixel to generate a pixel difference map; a multi-scale Gaussian pyramid decomposition unit for performing multi-scale Gaussian pyramid decomposition on the pixel difference map to obtain multiple difference sub-maps; and an adaptive threshold segmentation unit for performing adaptive threshold segmentation on the multiple difference sub-maps respectively, and mapping the segmented anomaly regions back to the original image scale for merging and morphological optimization, removing isolated regions with an area smaller than a preset value to obtain the pattern anomaly candidate region.

[0061] The difference calculation unit may further include: the gold template is a gold template feature map, generated by inputting standard reference data into the same feature extraction and fusion process as that used to extract the pattern multi-feature map; wherein the standard reference data is: a multimodal registration image generated by acquiring image data from a confirmed defect-free similar optical substrate wafer sample using the industrial vision imaging system; or a multimodal registration simulation image generated by calculating and rendering using an optical imaging simulation model based on the design layout data of the optical functional pattern.

[0062] The specific configuration of the defect identification module 60 is described in detail below: As mentioned above, the pattern multi-feature map and the pattern anomaly candidate region are input into a pre-trained AI visual recognition model. Guided by the pattern anomaly candidate region, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map and outputs a defect identification result. The defect identification module 60 may further include: an AI visual recognition model building unit, where the AI ​​visual recognition model is a Siamese network based on an encoder-decoder architecture and integrates a spatial attention guidance module; and a feature modulation unit, whereby the spatial attention guidance module uses the pattern anomaly candidate region as a spatial attention mask to modulate the input pattern... Multi-feature map feature modulation is performed to enhance feature responses within candidate regions and suppress feature responses within non-candidate regions, generating an attention-modulated feature map. A feature extraction unit is used in the first branch of the Siamese network to extract local detail features of the defect by taking local feature blocks corresponding to the candidate regions in the attention-modulated feature map as input. The second branch of the Siamese network takes the entire attention-modulated feature map as input to extract global contextual features containing the relationship between the candidate region and surrounding patterns. A defect semantic segmentation unit is used in the feature fusion layer of the Siamese network to fuse the local detail features and the global contextual features, performing defect semantic segmentation and classification based on the fused features, and outputting the defect identification result.

[0063] The detailed description of the specific configuration of the industrial vision imaging system configuration module 10 is explained as follows: As mentioned above, the industrial vision imaging system configuration module 10 may further include: the first imaging optical path is a bright field or dark field microscopic imaging optical path for acquiring high-contrast two-dimensional geometric morphology of the pattern; the second imaging optical path is a spectral imaging optical path for acquiring the material composition of the pattern region, or a polarization imaging optical path for acquiring the film properties of the pattern region, or a confocal imaging optical path for acquiring subsurface information of the pattern region.

[0064] The image registration module 30 is described in detail below: As mentioned above, spatial registration and image enhancement processing are performed on the first optical path image and the second optical path image to obtain a multimodal registered image. The image registration module 30 may further include: a feature point set matching unit for extracting a first feature point set from the first optical path image, extracting a second feature point set from the second optical path image, and matching the first feature point set and the second feature point set using feature descriptors to obtain multiple pairs of matching feature point pairs; and a spatial geometric transformation matrix calculation unit for filtering out correct matching pairs from the multiple pairs of matching feature point pairs and using the correct matching pairs... The spatial geometric transformation matrix from the second optical path image to the first optical path image is calculated; the registration unit is used to apply the spatial geometric transformation matrix to register the second optical path image, generating a second optical path registered image; the adaptive contrast enhancement processing unit is used to perform denoising and adaptive contrast enhancement processing on the first optical path image and the second optical path registered image respectively, to obtain a first enhanced optical path image and a second enhanced optical path image; the image stitching unit is used to stitch the first enhanced optical path image and the second enhanced optical path image along the channel dimension, and to perform inter-modal normalization processing on the stitched multi-channel image to generate the multi-modal registered image.

[0065] The image stitching unit may further include: the inter-modal normalization processing is Z-score normalization processing.

[0066] The detailed description of the specific configuration of the feature extraction and fusion module 40 is explained as follows: As mentioned above, based on the multimodal registration image, a first feature map and a second feature map are extracted and fused to generate a patterned multi-feature map. The feature extraction and fusion module 40 may further include: a feature map extraction unit for processing the first modality image data and the second modality image data in the multimodal registration image through a feature extraction network to obtain the first feature map and the second feature map; a global average pooling unit for concatenating the first feature map and the second feature map along the channel dimension and then performing global average pooling to obtain multiple global description vectors, wherein each channel corresponds to one global description vector; a weight coefficient output unit for inputting the multiple global description vectors into a multilayer perceptron to learn and output multiple weight coefficients; a normalization unit for normalizing the multiple weight coefficients using the Softmax function to obtain multiple channel attention weights; and a feature map weighting unit for weighting the corresponding channels of the first feature map and the second feature map using the multiple channel attention weights, and then merging the channels to generate the patterned multi-feature map.

[0067] The system may further include: a statistical analysis module for performing statistical analysis on the defect identification results of the same batch to generate a defect distribution map; a deviation source location module for locating the deviation source of the patterning process based on the defect distribution map; and a parameter control suggestion generation module for generating parameter control suggestions based on the deviation source.

[0068] The intelligent detection system for wafer pattern defects based on multimodal data fusion provided in this embodiment of the invention can execute the intelligent detection method for wafer pattern defects based on multimodal data fusion provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0069] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A smart detection method for wafer pattern defects based on multimodal data fusion, characterized in that, The method includes: Configure an industrial vision imaging system, wherein the industrial vision imaging system includes at least a first imaging optical path and a second imaging optical path; The industrial vision imaging system simultaneously acquires a first optical path image and a second optical path image of the patterned area of ​​the optical substrate wafer, wherein a preset optical functional pattern is formed on the surface of the optical substrate wafer. Spatial registration and image enhancement processing are performed on the first optical path image and the second optical path image to obtain a multimodal registered image; Based on the multimodal registration image, the first feature map and the second feature map are extracted and fused to generate a patterned multi-feature map; The pattern multi-feature map is compared and analyzed with a preset gold template to locate the candidate region of pattern anomaly; The pattern multi-feature map and the pattern anomaly candidate region are input into a pre-trained AI visual recognition model. Guided by the pattern anomaly candidate region, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map and outputs the defect recognition result.

2. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 1, characterized in that, The pattern multi-feature map is compared and analyzed with a preset gold template to locate candidate regions of pattern anomalies, including: The difference between the pattern multi-feature map and the gold template is calculated pixel by pixel to generate a pixel difference map; The pixel difference map is decomposed into multiple sub-maps by performing multi-scale Gaussian pyramid decomposition. Adaptive threshold segmentation is performed on the multiple difference sub-images respectively, and the segmented abnormal regions are mapped back to the original image scale for merging and morphological optimization. Isolated regions with an area smaller than a preset value are removed to obtain the pattern abnormality candidate regions.

3. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 2, characterized in that, The gold template is a gold template feature map, which is generated by inputting standard reference data into the same feature extraction and fusion process as that used to extract the pattern multi-feature map. The standard reference data is as follows: Multimodal registered images are generated from image data acquired using the industrial vision imaging system from a confirmed defect-free similar optical substrate wafer sample. Alternatively, based on the design layout data of the optical functional pattern, a multimodal registration simulation image can be generated by calculating and rendering using an optical imaging simulation model.

4. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 1, characterized in that, The pattern multi-feature map and the pattern anomaly candidate region are input into a pre-trained AI visual recognition model. Guided by the pattern anomaly candidate region, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map and outputs defect recognition results, including: The AI ​​visual recognition model is a twin network based on an encoder-decoder architecture and integrates a spatial attention guidance module; The spatial attention guidance module uses the anomaly candidate region of the pattern as a spatial attention mask to perform feature modulation on the input pattern multi-feature map, enhance the feature response within the candidate region, suppress the feature response of the non-candidate region, and generate an attention-modulated feature map. The first branch of the Siamese network takes the local feature block corresponding to the candidate region in the attention modulation feature map as input to extract the local detail features of the defect, and the second branch of the Siamese network takes the entire attention modulation feature map as input to extract the global context features containing the relationship between the candidate region and the surrounding pattern. The feature fusion layer of the Siamese network fuses the local detail features and the global context features, performs defect semantic segmentation and classification based on the fused features, and outputs the defect identification result.

5. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 1, characterized in that, The first imaging optical path is a bright-field or dark-field microscopic imaging optical path used to acquire high-contrast two-dimensional geometric morphology of the pattern; the second imaging optical path is a spectral imaging optical path used to acquire the material composition of the pattern region, a polarization imaging optical path used to acquire the film properties of the pattern region, or a confocal imaging optical path used to acquire subsurface information of the pattern region.

6. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 5, characterized in that, Spatial registration and image enhancement processing are performed on the first optical path image and the second optical path image to obtain a multimodal registered image, including: A first feature point set is extracted from the first optical path image, and a second feature point set is extracted from the second optical path image. The first feature point set and the second feature point set are matched using feature descriptors to obtain multiple pairs of matching feature points. Select the correct matching pairs from the multiple pairs of matching feature points, and use the correct matching pairs to calculate the spatial geometric transformation matrix from the second optical path image to the first optical path image; The second optical path image is registered using the spatial geometric transformation matrix to generate a second optical path registered image; The first optical path image and the second optical path registration image are respectively subjected to denoising and adaptive contrast enhancement processing to obtain the first enhanced optical path image and the second enhanced optical path image; The first enhanced optical path image and the second enhanced optical path image are stitched together along the channel dimension, and the stitched multi-channel image is subjected to inter-modal normalization processing to generate the multi-modal registration image.

7. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 6, characterized in that, The intermodal normalization process is Z-score normalization.

8. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 1, characterized in that, Based on the multimodal registration image, a first feature map and a second feature map are extracted, and after fusion, a patterned multi-feature map is generated, including: The first modality image data and the second modality image data in the multimodal registration image are processed by a feature extraction network to obtain the first feature map and the second feature map. The first feature map and the second feature map are concatenated along the channel dimension and then subjected to global average pooling to obtain multiple global description vectors, wherein each channel corresponds to one global description vector; The multiple global description vectors are input into a multilayer perceptron to learn and output multiple weight coefficients; The multiple weight coefficients are normalized using the Softmax function to obtain the attention weights for multiple channels; Multiple channel attention weights are used to weight the corresponding channels of the first feature map and the second feature map, and then the channels are merged to generate the pattern multi-feature map.

9. The intelligent detection method for wafer pattern defects based on multimodal data fusion as described in claim 1, characterized in that, Also includes: Statistical analysis is performed on the defect identification results of the same batch to generate a defect distribution map; Based on the defect distribution map, locate the source of deviation in the patterning process; Parameter control recommendations are generated based on the sources of the deviation.

10. A multimodal data fusion-based intelligent detection system for wafer pattern defects, characterized in that, The system is used to implement the intelligent detection method for wafer pattern defects based on multimodal data fusion as described in any one of claims 1-9, and the system comprises: An industrial vision imaging system configuration module is used to configure an industrial vision imaging system, wherein the industrial vision imaging system includes at least a first imaging optical path and a second imaging optical path. The optical path image acquisition module is used to simultaneously acquire a first optical path image and a second optical path image of the patterned area of ​​the optical substrate wafer through the industrial vision imaging system, wherein a preset optical functional pattern is formed on the surface of the optical substrate wafer. The image registration module is used to perform spatial registration and image enhancement processing on the first optical path image and the second optical path image to obtain a multimodal registered image; The feature extraction and fusion module is used to extract a first feature map and a second feature map based on the multimodal registration image, and then fuse them to generate a patterned multi-feature map; The pattern anomaly candidate region localization module is used to compare and analyze the pattern multi-feature map with a preset gold template to locate the pattern anomaly candidate region. The defect identification module is used to input the pattern multi-feature map and the pattern anomaly candidate region into a pre-trained AI visual recognition model. Guided by the pattern anomaly candidate region, the AI ​​visual recognition model performs depth detection on the pattern multi-feature map and outputs the defect identification result.

Images