Wafer image matching method and device and computer storage medium

Abstract

The embodiment of the invention provides a wafer image matching method and device and a computer storage medium, and the method comprises the steps: firstly carrying out the down-sampling of an obtained to-be-detected and template image, and rapidly calculating an initial matching position in a low-resolution space, so as to greatly reduce a search range; performing up-sampling on the low-resolution to-be-detected image by using a pre-trained neural network, and reconstructing a detail-enhanced high-resolution image; and finally, performing fine matching in an initial position neighborhood based on the high-quality image, and determining an optimal sub-pixel position. According to the method, the search space is efficiently compressed through low-resolution rough matching, the overall calculation complexity is remarkably reduced, and the speed is increased; meanwhile, by means of super-resolution recovery driven by the neural network, the problem that details of a traditional interpolation method are fuzzy is effectively solved, high-fidelity image data are provided for fine matching, and therefore the sub-pixel-level matching precision is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semiconductor detection, in particular to a wafer image matching method and device and computer storage medium. BACKGROUND

[0002] Image matching technology is a key link in high-precision semiconductor manufacturing detection, and is widely used in defect identification, size measurement and process monitoring of microscopic devices. The technology accurately aligns the collected detection image with the standard reference image to provide a benchmark for subsequent quality analysis. With the continuous reduction of semiconductor device feature size to the nanometer level, it puts forward very high requirements for the precision, robustness and efficiency of image matching.

[0003] The current wafer image matching method mainly uses feature template matching, normalized cross-correlation algorithm and traditional affine transformation model, which can complete the image alignment task in a simple scene to a certain extent, has the advantages of clear calculation structure and convenient implementation, and provides a foundation support for many industrial detection systems.

[0004] However, when facing the actual environment of semiconductor detection, the existing method still has certain limitations. Under strong noise interference, the edge feature retention ability of the traditional method is insufficient, and the smoothing operation easily leads to detail loss, making it difficult to achieve nanoscale positioning accuracy, and lacking effective modeling ability for complex nonlinear deformation and cross-modal image differences in the imaging process. In addition, some image matching methods combine deep learning models to show good performance in image reconstruction, but still cannot adaptively capture the spatial geometric relationship between image blocks, limiting the model's perception of image structure. SUMMARY

[0005] The present application provides a wafer image matching method, device and computer storage medium, which can solve the technical problem that in the image acquisition process of semiconductor detection, due to factors such as strong noise interference and mechanical vibration interference, the signal-to-noise ratio of the acquired image is reduced and the position offset error is caused, affecting the detection precision and reliability.

[0006] In a first aspect, the present application provides a wafer image matching method, comprising:

[0007] obtaining a to-be-tested image and a template image of a target object; performing down-sampling on the to-be-tested image and the template image respectively to obtain low-resolution images corresponding to the to-be-tested image and the template image respectively; determining an initial matching position of the to-be-tested image and the template image according to the low-resolution images corresponding to the to-be-tested image and the template image respectively; performing up-sampling on the low-resolution image corresponding to the to-be-tested image based on a pre-constructed neural network to obtain a high-resolution image corresponding to the to-be-tested image; and determining a best matching position of the to-be-tested image and the template image according to the high-resolution image corresponding to the to-be-tested image and the initial matching position.

[0008] In some embodiments, the determining of the initial matching position of the to-be-tested image and the template image according to the low-resolution images corresponding to the to-be-tested image and the template image respectively comprises:

[0009] performing noise reduction processing on the low-resolution images corresponding to the to-be-tested image and the template image respectively; performing pixel gradient calculation on the low-resolution images corresponding to the to-be-tested image and the template image after noise reduction by using a sobel operator to obtain gradient directions and gradient amplitudes of each pixel point; determining edge information of the low-resolution images corresponding to the to-be-tested image and the template image according to the gradient directions and the gradient amplitudes of each pixel point; and determining the initial matching position of the to-be-tested image and the template image according to the edge information.

[0010] In some embodiments, the noise reduction processing comprises at least one of Gaussian filtering processing, median filtering and local mean denoising.

[0011] The pixel gradient calculation on the low-resolution images corresponding to the to-be-tested image and the template image after noise reduction by using the sobel operator comprises performing pixel gradient calculation on the low-resolution images corresponding to the to-be-tested image and the template image after noise reduction by using sobel operators with different convolution kernels in X and Y directions.

[0012] In some embodiments, the pre-constructed neural network is a hybrid network model of a super-resolution convolutional neural network and a transformer based on a sliding window.

[0013] In some embodiments, the performing of the up-sampling on the low-resolution image corresponding to the to-be-tested image based on the pre-constructed neural network to obtain the high-resolution image corresponding to the to-be-tested image comprises:

[0014] dividing the low-resolution image corresponding to the to-be-tested image into a plurality of image blocks, and encoding each of the image blocks to obtain a sequence of feature vectors with spatial position information; inputting the sequence of feature vectors into the neural network to perform self-attention-based context feature modeling, and performing nonlinear transformation on the modeled features to obtain deep features of the to-be-tested image; and performing up-sampling processing on the deep features to reconstruct a high-resolution image corresponding to the to-be-tested image.

[0015] In some embodiments, the dividing the low-resolution image corresponding to the to-be-tested image into a plurality of image blocks, and encoding each of the image blocks to obtain a sequence of feature vectors with spatial position information comprises:

[0016] dividing a low-resolution image corresponding to a to-be-tested image into a plurality of image blocks of the same size and without overlap; mapping each of the image blocks to a vector space through linear projection to obtain a feature vector corresponding to each of the image blocks; and adding learnable spatial position encoding to each feature vector according to a pre-created learnable position encoding block to obtain a sequence of feature vectors with spatial position information.

[0017] In some embodiments, the pre-created position encoding block has the same size as the image block.

[0018] The adding learnable spatial position encoding to each feature vector according to a pre-created position encoding block comprises:

[0019] extracting a learnable position encoding vector in the position encoding block corresponding to the spatial position of each of the image blocks; and adding, element by element, the feature vector corresponding to each of the image blocks and the position encoding vector corresponding thereto to obtain the sequence of feature vectors with spatial position information.

[0020] In some embodiments, the learnable position encoding block is equipped with learnable scaling coefficients for controlling the strength of position information injection during model training.

[0021] In some embodiments, the inputting the sequence of feature vectors into the neural network to perform self-attention-based context feature modeling, and performing nonlinear transformation on the modeled features to obtain deep features of the to-be-tested image comprises:

[0022] The sequence of feature vectors is divided into a plurality of non-overlapping first windows, and self-attention is calculated in each of the first windows, the attention of each window is combined, and a first intermediate feature is obtained; the first window division is slid according to a preset offset, the first intermediate feature is divided into a plurality of second windows, and self-attention is calculated in each of the second windows, the shift attention features of each of the second windows are combined, and a second intermediate feature is obtained; the second intermediate feature is added to the initial sequence of feature vectors in a residual manner, and a fused feature is obtained; the fused feature is subjected to nonlinear transformation, and a deep feature is obtained.

[0023] In some embodiments, the determining the best matching position of the test image and the template image according to the high-resolution image corresponding to the test image and the initial matching position comprises:

[0024] According to the initial matching position, the corresponding coordinates of the initial matching position in the high-resolution image corresponding to the test image are determined; a local search window is constructed with a preset side length or radius as the center of the corresponding coordinates; the high-resolution image corresponding to the test image and the template image are subjected to normalized cross-correlation calculation in the local search window, and a correlation response map is obtained; according to the correlation response map, the best matching position of the test image and the template image is determined through centroid calculation.

[0025] In some embodiments, the nonlinear transformation of the fused feature comprises: using at least one convolution layer combined with an activation function for nonlinear transformation to perform channel transformation and enhancement on the feature; wherein the convolution layer comprises a 3x3 convolution, and the activation function comprises a ReLU or GELU function.

[0026] In a second aspect, the embodiments of the present application provide a wafer image matching device, comprising:

[0027] An image acquisition module is configured to acquire a test image and a template image of a target object.

[0028] A first processing module is configured to down-sample the test image and the template image respectively to obtain low-resolution images corresponding to the test image and the template image respectively.

[0029] A first matching module is configured to determine an initial matching position of the test image and the template image according to the low-resolution images corresponding to the test image and the template image respectively.

[0030] A second processing module is configured to up-sample the low-resolution image corresponding to the test image based on a pre-constructed neural network to obtain a high-resolution image corresponding to the test image.

[0031] The second matching module is configured to determine the optimal matching position of the to-be-tested image and the template image according to the high-resolution image corresponding to the to-be-tested image and the initial matching position.

[0032] In some embodiments, the neural network comprises an image patching layer, a linear embedding layer, a position encoding layer, a feature extraction layer, a nonlinear mapping layer, and a reconstruction layer.

[0033] The image patching layer is configured to divide the low-resolution image corresponding to the to-be-tested image into a plurality of image patches of the same size and without overlap; the linear embedding layer is configured to map each image patch to a vector space through linear projection to obtain a feature vector corresponding to each image patch; the position encoding layer comprises a learnable position encoding block, which is configured to add a learnable spatial position encoding to each feature vector according to the learnable position encoding block to obtain a feature vector sequence with spatial position information; the feature extraction layer is configured to input the feature vector sequence to the neural network for self-attention-based context feature modeling; the nonlinear mapping layer is configured to perform nonlinear transformation on the modeled features to obtain deep features of the to-be-tested image; and the reconstruction layer is configured to perform up-sampling processing on the deep features to reconstruct a high-resolution image corresponding to the to-be-tested image.

[0034] In some embodiments, the feature extraction layer comprises at least a first sliding window converter, a second sliding window converter, and a skip connection module.

[0035] The first sliding window converter is configured to divide the feature vector sequence into a plurality of non-overlapping first windows, calculate self-attention in each first window, and combine the attention of each window to obtain first intermediate features; the second sliding window converter is configured to slide the first window division according to a preset offset, divide the first intermediate features into a plurality of second windows, calculate self-attention in each second window, and combine the shifted attention features of each second window to obtain second intermediate features; and the skip connection module is configured to add a residual to the second intermediate features and the initial feature vector sequence to obtain fused features, and perform nonlinear transformation on the fused features to obtain deep features.

[0036] In a third aspect, the embodiments of the present application further provide a computer-readable storage medium, wherein a computer-executable program or instruction is stored in the computer-readable storage medium, and the computer-executable program or instruction is executed by a processor to implement the steps of the wafer image matching method according to any embodiment of the first aspect.

[0037] The wafer image matching method and device provided by the embodiment of the present application first performs down-sampling on the acquired template image and measured image, quickly calculates an initial matching position in a low-resolution space to greatly reduce the search range; then uses a pre-trained neural network to perform up-sampling on the low-resolution measured image, and reconstructs a high-resolution image with enhanced details; finally, performs fine matching in the initial position neighborhood based on the high-quality image, and determines the best sub-pixel position. The method efficiently compresses the search space through low-resolution coarse matching, significantly reduces the overall computational complexity, and improves the speed; at the same time, the neural network driven super-resolution recovery effectively overcomes the detail blur problem of the traditional interpolation method, provides high-fidelity image data for fine matching, thereby ensuring the sub-pixel level matching accuracy, and taking into account the processing efficiency, meets the stringent requirements of semiconductor manufacturing on online detection system high speed and high reliability.

[0038] In addition, the present application also provides a computer readable storage medium, which has the same beneficial effects as the wafer image matching method described above. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] Figure 1 The flowchart of the wafer image matching method provided by an embodiment of the present application.

[0041] Figure 2 The flowchart of the wafer image matching method provided by another embodiment of the present application.

[0042] Figure 3 The flowchart of the wafer image matching method provided by another embodiment of the present application.

[0043] Figure 4 The flowchart of the wafer image matching method provided by another embodiment of the present application.

[0044] Figure 5 The flowchart of the wafer image matching method provided by another embodiment of the present application.

[0045] Figure 6 The flowchart of the wafer image matching method provided by another embodiment of the present application.

[0046] Figure 7 The structure schematic diagram of the wafer image matching device provided by an embodiment of the present application.

[0047] Figure 8 The structure schematic diagram of the wafer image matching device provided by another embodiment of the present application.

[0048] Figure 9 Fig. 1 is a schematic diagram of a wafer image matching device according to an embodiment of the present application.

[0049] The specific embodiments of the present application have been shown by way of illustration in the drawings and more details will be described in the following. These drawings and detailed description are not intended to limit the scope of the present application in any manner, but to explain the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0050] The present application will be described in further detail by way of illustration in the following specific embodiments with reference to the accompanying drawings. Like reference numerals in different embodiments designate like elements. In the following embodiments, many details are described in order to provide a more thorough description of the present application. However, it will be apparent to persons skilled in the art that some features in different embodiments can be omitted, or replaced by other elements, materials, methods, etc. In some cases, some operations related to the present application are not shown or described in the specification in order to avoid the core of the present application from being obscured by too many descriptions, and it is not necessary to describe these related operations in detail for those skilled in the art based on the description in the specification and general technical knowledge in the art.

[0051] In addition, features, operations or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. Meanwhile, the steps or actions in the method description can also be adjusted or changed in order as can be apparent to those skilled in the art. Therefore, the various orders in the specification and the drawings are only for the purpose of clearly describing a certain embodiment, and do not mean a necessary order, unless otherwise stated that a certain order must be followed.

[0052] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind and are not limited in number, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally means that the objects before and after are in an "or" relationship. The present application means "connection" and "coupling", unless otherwise specified, including direct and indirect connection (coupling).

[0053] In wafer detection, image matching plays a key role in locating fiducials and aligning to a reference mark. It precisely aligns the acquired wafer image containing subtle process deformation to a perfect design template or "golden template" at pixel level. Only after the matching is completed, the semiconductor detection system can know "where to look" and establish a stable coordinate frame for subsequent defect scanning and dimension measurement. More importantly, the matched image can efficiently and reliably highlight abnormal areas inconsistent with the template through pixel comparison (i.e. image difference), which are potential defects such as short circuit, open circuit or particle contamination. Therefore, image matching technology is the prerequisite for realizing fully automatic, high-speed and high-precision wafer defect detection, which ensures the stability of the detection process and the repeatability of the results, and is a key technical guarantee for improving the yield of semiconductor manufacturing.

[0054] Compared with conventional image matching technology, due to the high repeatability of wafer surface structure and the requirement of nanometer-level matching accuracy, the technology usually adopts gray-based template matching or special algorithms based on pattern alignment. Wafer detection matching pursues extreme accuracy and stability, and it works in a highly controllable environment. The matching object is a known and regular artificial pattern. The technical challenge is how to overcome the slight process variation and noise, and firmly ignore the detected target (defect) itself, so as to realize nanometer-level repetitive positioning.

[0055] As described in the background, the conventional image matching method mainly adopts feature template matching, normalized cross-correlation algorithm and traditional affine transformation model, which can complete the image alignment task in a simple scene to a certain extent. However, under strong noise interference, smoothing operation is easy to cause detail loss, and it is difficult to realize nanometer-level positioning accuracy. Moreover, it lacks effective modeling ability for complex nonlinear deformation and cross-modal image difference in the imaging process.

[0056] Therefore, the present application provides a new wafer image matching method, which can solve the problem of signal-to-noise ratio reduction and position offset error of the acquired image caused by strong noise interference and mechanical vibration interference and other factors in the image acquisition process of semiconductor detection, and improve the detection accuracy and reliability.

[0057] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0058] Figure 1 A flowchart of the wafer image matching method provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the wafer image matching method provided by the embodiment specifically includes the following steps: Figure 1 ​

[0059] Step S10, obtaining a to-be-tested image and a template image of a target object.

[0060] In actual application scenarios such as wafer manufacturing and detection, image analysis needs to be performed on the to-be-tested wafer to determine its quality, position, and the like. Before wafer image matching, the target wafer can be photographed by various imaging devices such as a high-precision scanning electron microscope (SEM) to collect a to-be-tested image, which reflects the actual state of the wafer to be tested. A matching template is essential in the template matching process, and the template image is usually a nearly perfect reference image, which can be generated by directly rendering a design blueprint of a chip or collected from a "golden chip" confirmed to be defect-free. Obtaining these two types of images is the basis for subsequent matching analysis. Only when the clear and complete to-be-tested image and template image are accurately obtained, can the reliability of the subsequent matching result be ensured.

[0061] Step S20, respectively downsampling the to-be-tested image and the template image to obtain low-resolution images corresponding to the to-be-tested image and the template image.

[0062] In wafer image matching, a high-resolution image contains a large amount of pixel information, and direct processing thereof will consume a large amount of computing resources and time. Especially when processing large-scale wafer image data, the computing efficiency will be severely affected. Therefore, before obtaining the to-be-tested image and the template image of the target object for image matching, the pixel number of the image is first reduced by a preset algorithm to remove some subtle noise and unnecessary details in the image, and a low-resolution version with smaller size and more condensed information is generated. In the low-resolution scale, the details and noise of the image are weakened, but the overall structure contour and features are retained, and the main features are more prominent, which makes preparation for the next step of coarse matching, so that the approximate matching position can be quickly searched in a larger spatial range, and the huge computing overhead of full-range search on high-resolution massive data is avoided.

[0063] Step S30, determining an initial matching position of the to-be-tested image and the template image according to the low-resolution images corresponding to the to-be-tested image and the template image.

[0064] After obtaining the low-resolution versions of the to-be-tested image and the template image, due to the small amount of low-resolution image data and prominent features, a traditional image matching algorithm is used to quickly find the approximate corresponding position relationship between the to-be-tested image and the template image at a low computational cost, and the matching position between the two is preliminarily determined. The initial matching position calculated is a rough, pixel-level estimation, which is not accurate enough, but can narrow the range of subsequent fine search to a small neighborhood window, greatly reducing the computational amount required for final fine matching, and effectively avoiding possible mismatches in a large range

[0065] Step S40, based on the pre-constructed neural network, up-sampling the low-resolution image corresponding to the to-be-tested image to obtain a high-resolution image corresponding to the to-be-tested image.

[0066] After the initial matching position is quickly determined through the low-resolution images of the to-be-tested image and the template image, some details are lost in the process, and if the low-resolution image is directly enlarged, the resulting image will be edge blurred and details missing, which cannot meet the nanometer (sub-pixel) matching accuracy required for wafer detection. Therefore, in this embodiment, a pre-trained neural network is introduced for up-sampling, and the powerful non-linear mapping capability of the neural network is used to learn the complex mapping relationship from low resolution to high resolution from a large amount of wafer image data, aiming to intelligently reconstruct a high-resolution image with rich details and clear edges, restore or enhance feature details, realize the leap from "rough positioning" to "fine matching", and guarantee the sub-pixel level matching accuracy. It can be seen that, while improving the computational efficiency by using low-resolution images, the image obtained in this embodiment is close to the original high-resolution image quality, which provides more reliable image data for subsequent accurate determination of the best matching position.

[0067] Step S50, determining the best matching position of the to-be-tested image and the template image according to the high-resolution image corresponding to the to-be-tested image and the initial matching position.

[0068] After obtaining the high-resolution image of the to-be-tested image and the initial matching position, a high-precision matching algorithm is used to compare the high-resolution to-be-tested image reconstructed by the neural network with the original high-resolution template image in a very small neighborhood range centered on the initial matching position, and a more detailed search and matching is performed between the high-resolution image of the to-be-tested image and the template image. Through iterative optimization, the matching error is gradually reduced, and the sub-pixel level displacement offset is calculated, and finally the most accurate and matching position between the to-be-tested image and the template image, i.e. the best matching position, is found. The best matching position is of great significance to the actual application of wafer quality detection and positioning adjustment, and can provide accurate reference for subsequent production and detection processes.

[0069] In summary, the wafer image matching method provided by the embodiment first performs downsampling on the obtained to-be-tested image and the template image, quickly calculates an initial matching position in a low-resolution space to greatly reduce the search range, then uses a pre-trained neural network to upsample the low-resolution to-be-tested image to reconstruct a high-resolution image with enhanced details, and finally performs fine matching in the initial position neighborhood based on the high-quality image to determine the best sub-pixel position. The method efficiently compresses the search space through low-resolution coarse matching, significantly reduces the overall computational complexity, improves the speed, effectively overcomes the detail blurring problem of the traditional interpolation method with the aid of the neural network-driven super-resolution recovery, provides high-fidelity image data for fine matching, thereby guaranteeing the sub-pixel-level matching accuracy and taking into account the processing efficiency, and meets the stringent requirements of semiconductor manufacturing on the high speed and high reliability of the online detection system.

[0070] Figure 2 The flowchart of the wafer image matching method provided by another embodiment of the application is shown in FIG. 3. As shown in FIG. 3, the steps S30 and S40 are the same as those in the flowchart shown in FIG. 2. Figure 2 The step S30 of determining the initial matching position of the to-be-tested image and the template image according to the low-resolution images corresponding to the to-be-tested image and the template image respectively specifically includes the following steps.

[0071] The step S301 of performing noise reduction processing on the low-resolution images corresponding to the to-be-tested image and the template image respectively.

[0072] As described above, to improve the processing efficiency in the wafer image matching process, it is necessary to first perform preliminary downsampling operation on the to-be-tested image and the template image, convert the high-resolution image into a low-resolution image, and reduce the data amount. The low-resolution image after downsampling is reduced in size, but still contains noise inherited from the original image. Although the downsampling operation itself has a certain smoothing and noise suppression effect, these random noises will cause false gray value fluctuations between pixels. If gradient calculation is directly performed, these noise points will be misjudged as high-frequency edge features, a large number of irrelevant or even misleading “pseudo-gradients” will be generated, and the accuracy of the initial matching will be seriously interfered. Therefore, after downsampling, noise reduction processing needs to be performed on the low-resolution images corresponding to the to-be-tested image and the template image respectively to improve the quality of the low-resolution image and ensure the robustness of the matching.

[0073] In some embodiments, the noise reduction processing on the low-resolution images corresponding to the to-be-tested image and the template image respectively can use Gaussian filtering to perform weighted average on the neighborhood pixels according to the Gaussian function, maintain the image details while reducing the noise, or use median filtering or local mean denoising filtering algorithm to perform smoothing processing on the low-resolution image, effectively suppress the noise while striving to preserve the overall structure and main edge information of the image.

[0074] In step S302, pixel gradient of the low-resolution image corresponding to the denoised to-be-tested image and the template image is calculated by using a Sobel operator to obtain gradient direction and gradient amplitude of each pixel.

[0075] After the denoising processing of the low-resolution image is completed, in order to more accurately extract key features such as edges and textures with higher distinguishability from the gray image, a Sobel operator is used to calculate the pixel gradient. The Sobel operator calculates the differential approximation value of the image in the X and Y directions by convolution, thereby obtaining the gradient amplitude (intensity) and gradient direction of each pixel. After the gradient direction and gradient amplitude of each pixel are obtained by the Sobel operator, the conversion of the image from the original "gray space" to the "gradient space" that can better represent the structure information is realized. The generated gradient amplitude image highlights the edge information of the figure, and the gradient direction describes the trend of the edge. The gradient feature provides higher-quality and more stable input data for the subsequent initial matching, so that the matching algorithm can perform similarity measurement based on the clear structure contour, thereby greatly improving the accuracy and reliability of the coarse matching stage.

[0076] In the embodiment, for the low-resolution image corresponding to the to-be-tested image and the template image, a double-path framework is used to complete the feature extraction of the low-resolution image, and the best balance between denoising and feature reservation is realized. The main path uses Gaussian blur processing to effectively suppress the high-frequency noise in the low-resolution image, provides stable and smooth gray basis information for matching, and ensures the reliability of the overall structure. The secondary path extracts the image gradient field based on the improved Sobel operator to accurately obtain the edge information of the image and strengthen the perception ability of key structural features such as edges and contours, thereby avoiding the edge blurring problem that may be caused by the denoising operation in the traditional single path. The two paths are processed in parallel, and the denoised gray information and the enhanced edge gradient information are effectively combined to eliminate the interference of noise on the initial matching to the greatest extent, significantly improve the distinguishability and robustness of the features (especially the edge structure), and avoid the contradiction between denoising and edge extraction in the single path processing. The advantages of the two processing methods are fully utilized, and finally the accurate extraction of effective gradient information is realized, which lays a more accurate and reliable data foundation for the subsequent coarse matching and comprehensively improves the success rate and accuracy of the initial matching in the noise environment.

[0077] The Sobel operator is a commonly used discrete differential operator, which contains two sets of 3x3 convolution kernels (or filters) for calculating the gradients of the image in the horizontal and vertical directions, respectively. By performing convolution operations between the two matrices and the image, the approximate values of the gradients of the image in the horizontal and vertical directions can be obtained. Through the gradient values in the two directions, the gradient direction and the gradient amplitude of each pixel point can be further calculated, the gradient direction reflects the direction of the fastest gray change at the point in the image, and the gradient amplitude represents the degree of gray change.

[0078] In some embodiments, when calculating the pixel gradients of the low-resolution images corresponding to the denoised test image and the template image using the Sobel operator, the following expression can be used:

[0079]

[0080] where G(x) and G(y) are the two 3x3 convolution kernels of the Sobel operator, G(x) is the horizontal gradient operator for detecting the edge in the vertical direction, i.e., the change of the image in the X direction, and G(y) is the vertical gradient operator for detecting the edge in the horizontal direction, i.e., the change of the image in the Y direction. Through the convolution operations of G(x)*I and G(y)*I, the gradient component maps in the X and Y directions of the image are obtained, and the gradient components in the two directions are fused to finally obtain the pixel gradient of the image, The size of the gradient amplitude represents the total intensity of the edge at the point, and the larger the value, the more likely it is that the point is part of the edge; the smaller the value, the more likely it is that the point is located in a flat gray area. In short, the Sobel operator extracts the gradients in the X and Y directions through convolution to generate a feature image that is more conducive to image matching.

[0081] In step S303, the edge information of the low-resolution images corresponding to the test image and the template image is determined according to the gradient direction and the gradient amplitude of each pixel point.

[0082] Based on the gradient direction and the gradient amplitude of each pixel point in the image that have been obtained, the gradient direction indicates the direction of the fastest gray change at the pixel point, and the gradient amplitude reflects the degree of gray change. By comprehensively analyzing these two key parameters of each pixel point, it can be determined which pixel points constitute the edge of the image. Specifically, the pixel points with larger gradient amplitudes are often located at the edge of the image because the gray change is obvious at the edge; in combination with the gradient direction, the trend of the edge and other information can be further determined. According to this method, all pixel points in the low-resolution image are processed, and the edge information of the low-resolution images corresponding to the test image and the template image can be completely determined.

[0083] Step S304, determining the initial matching position of the to-be-tested image and the template image according to the edge information.

[0084] The edge information contains the key features of the image, such as the shape, direction and position of the edge, etc. Using these features, a variety of matching algorithms can be used to determine the initial matching position. Specifically, a similarity measurement-based method can be used to calculate the similarity degree between the edges of the to-be-tested image and the template image, and the position with the highest similarity degree can be used as the initial matching position. A geometric transformation-based method can also be used to find the position that makes the edges of the two images as aligned as possible as the initial matching position by analyzing the geometric relationship of the edges, such as rotation, translation, etc. Determining the initial matching position can reduce the calculation amount and time of matching, improve the efficiency and accuracy of matching, and provide a favorable starting point for subsequent more accurate matching algorithms.

[0085] In some embodiments, the pre-constructed neural network is a hybrid network model of a super-resolution convolutional neural network and a transformer based on a sliding window.

[0086] This model combines the excellent pixel-level reconstruction capability of SRCNN (super-resolution convolutional neural network) with the powerful global context feature modeling capability of SwinTransformer. The SRCNN structure is mainly responsible for effectively recovering high-frequency details from low-resolution input, while the Swin Transformer structure can efficiently capture long-range, complex semantic relationship-dependent features in the image through its hierarchical, sliding window-based self-attention mechanism. This hybrid design achieves complementary advantages and aims to significantly improve the visual quality and geometric accuracy of the final high-resolution reconstructed image.

[0087] Figure 3 A flowchart of a wafer image matching method according to another embodiment of the present application is provided. As shown in Figure 3 Step S40, based on a pre-constructed neural network, up-sampling a low-resolution image corresponding to the to-be-tested image to obtain a high-resolution image corresponding to the to-be-tested image, specifically including the following steps:

[0088] Step S401, dividing the low-resolution image corresponding to the to-be-tested image into a plurality of image blocks, and encoding each image block to obtain a sequence of feature vectors with spatial position information.

[0089] In order to restore the low-resolution image corresponding to the feature-extracted to-be-tested image to a high-resolution image, the low-resolution image needs to be structured and preprocessed to prepare for deep feature extraction. Specifically, the low-resolution to-be-tested image is divided into multiple image blocks, each image block is taken as a basic processing unit, the global image problem is converted into a local processing task, a convolutional neural network (SRCNN, super-resolution convolutional neural network) or an already coded encoder is used to code each image block, and the pixel values therein are converted into a high-dimensional feature vector. These feature vectors not only contain visual information within the block, but also explicitly add position encoding to each feature vector to inject absolute or relative position information of the image block in the original image, so that the feature vectors corresponding to all image blocks are arranged in order to form a feature sequence that retains spatial structure information, which completely describes the content of the whole image.

[0090] In step S402, the feature vector sequence is input into a neural network for self-attention-based context feature modeling, and the modeled features are nonlinearly transformed to obtain deep features of the to-be-tested image.

[0091] The neural network of the Transformer architecture can receive a feature vector sequence with spatial position information and model the context features of the feature vector sequence based on self-attention, i.e., allowing each feature vector in the sequence to interact and weightedly aggregate with all other vectors in the sequence. Specifically, the model strengthens the relevance of key features by calculating the attention weights of the query (Query), key (Key) and value (Value).

[0092] In some embodiments, the calculation formula of the self-attention mechanism (Self-Attention) can be:

[0093]

[0094] where Q (Query) is a query matrix representing the target feature for which attention needs to be calculated, K (Key) is a key matrix representing the index vectors of all input features, used to match the relevance with Q, and V (Value) is a value matrix representing the feature content that needs to be weightedly aggregated. QK T The dot product of the query vector Q and all key vectors K is calculated to obtain a similarity score matrix, and the dot product reflects the relevance of Q and each K, and the greater the value, the more similar (closer in direction) the two are; d is the vector dimension (i.e., feature dimension) of Q and K; B is an optional bias matrix used to explicitly model the position relationship; SoftMax normalization is applied to each row to convert the similarity score into a probability distribution; the output vector is obtained by weightedly summing V with the attention weight, the output vector fuses global information and pays more attention to the part of V with high relevance to Q.

[0095] In simple terms, self-attention dynamically computes the correlation between features (QK T ), adaptively aggregates global information (V), thereby capturing long-range dependencies (such as the association of distant pixels in an image or the dependence of distant words in text). The scaling and bias terms optimize numerical stability and position modeling, respectively, and finally output the feature representation that fuses the context.

[0096] Subsequently, a nonlinear transformation is performed on these attention-weighted features, further enhancing the model's expression ability and complexity, and finally extracting deep features that fuse local details and global semantics. This is a feature representation that has been deeply refined and enhanced, and it contains the key knowledge of how to intelligently reconstruct high-resolution details from low-resolution information.

[0097] In some embodiments, the modeled features can be subjected to nonlinear transformation through structures such as feedforward neural networks, residual connections, and normalization layers.

[0098] Step S403, the deep features are upsampled to reconstruct the high-resolution image corresponding to the to-be-tested image.

[0099] Further, the deep features that have been deeply modeled are upsampled, i.e., the deep features are upsampled to the size of the original high-resolution image. In some embodiments, a learnable upsampling layer such as transposed convolution, sub-pixel convolution, or interpolation algorithm can be used to gradually enlarge the resolution of the feature map, while simultaneously fusing shallow detail information through a skip connection. The final output high-resolution image not only has a larger size, but more importantly, it is more visually realistic and contains restored clear edges and rich texture details.

[0100] Figure 4 A flowchart of a wafer image matching method according to another embodiment of the present application is provided. As shown in Figure 4 the above step S401, the low-resolution image corresponding to the to-be-tested image is divided into a plurality of image blocks, and each image block is encoded to obtain a feature vector sequence with spatial position information, which specifically includes:

[0101] Step S4011, the low-resolution image corresponding to the to-be-tested image is divided into a plurality of image blocks of the same size and without overlap.

[0102] When encoding the low-resolution image, the low-resolution image to be processed is first uniformly cut into a plurality of regular image blocks of the same size and without overlap in the spatial dimension. The non-overlapping block strategy not only reduces the huge memory overhead when the model processes high-resolution images, but also preserves the most critical local structure and texture information in the downsampled image.

[0103] Step S4012, mapping each image block to a vector space through linear projection to obtain a feature vector corresponding to each image block.

[0104] Each cut-out two-dimensional image block is flattened and projected into a high-dimensional vector space to generate a one-dimensional vector, that is, the spatial structure of the image block is converted into a vectorized feature representation, which is mapped to a high-dimensional vector space through a learnable linear projection as a direct input of the subsequent Transformer model.

[0105] Step S4013, adding a learnable spatial position encoding to each feature vector according to a pre-created learnable position encoding block to obtain a feature vector sequence with spatial position information.

[0106] In order to compensate for the inherent permutation invariance of the self-attention mechanism, that is, the defect of being unable to perceive the input order, before modeling the self-attention, spatial position information is injected into each feature vector in the sequence, which is specifically realized by a pre-created learnable position encoding block. The position encoding vector corresponding to each image block is merged with the feature vector, and each vector in the finally output feature vector sequence contains both the visual content information of the image block and the absolute spatial position information of the image block in the original image.

[0107] In some embodiments, the size of the pre-created position encoding block is the same as the size of the image block, that is, the encoding block can be a trainable parameter matrix as long as the feature vector sequence, and each vector in the matrix uniquely corresponds to a specific spatial position.

[0108] In some embodiments, adding a learnable spatial position encoding to each feature vector specifically includes: extracting a learnable position encoding vector in the position encoding block that corresponds one-to-one to the spatial position of each image block; and element-wise adding the feature vector corresponding to each image block to the position encoding vector corresponding thereto to obtain a feature vector sequence with spatial position information.

[0109] Specifically, a trainable position encoding parameter matrix corresponding to the input image grid space size is obtained, and each vector in the matrix uniquely and fixedly represents an absolute coordinate position. During forward propagation, according to the coordinate of the current image block in the original image, the position encoding vector corresponding thereto is indexed (extracted) from the position encoding block stored therein, and then the position encoding vector is fused with the feature vector representing the visual content of the image block obtained through linear projection by element-wise addition. The feature vector sequence thus generated enables the subsequent Transformer model to clearly distinguish between image blocks at different positions, so that when the self-attention mechanism is used for global context modeling, the inherent two-dimensional spatial geometric relationship of the image can be correctly understood and utilized.

[0110] In some embodiments, the learnable position encoding block is equipped with learnable scaling coefficients for controlling the strength of position information injection during model training.

[0111] In the learnable position encoding block equipped with dynamically adjustable scaling coefficients, the scaling coefficients can be understood as a trainable parameter. Multiplying the scaling coefficients with the position encoding values can finely regulate the strength or weight of the position information injected into the image block features during model training. By regulating, the network is allowed to pay more attention to the content features of the image block itself at the early stage of training. As the training progresses, the value of the scaling coefficient is automatically learned and gradually increased through gradient descent backpropagation, thereby smoothly and adaptively increasing the importance of position information, improving the stability and convergence efficiency of training, enabling the model to ultimately achieve an optimal balance between content and position information, and enhancing the geometric perception ability of feature representation.

[0112] Figure 5 A flowchart of a wafer image matching method according to another embodiment of the present application is provided. As shown in Figure 5 The step S402 of inputting the sequence of feature vectors into the neural network for self-attention-based context feature modeling and performing nonlinear transformation on the modeled features to obtain deep features of the to-be-tested image includes the following steps:

[0113] Step S4021, divide the sequence of feature vectors into a plurality of non-overlapping first windows, and calculate self-attention in each first window, merge the attention of each window, and obtain first intermediate features.

[0114] Considering the problem of quadratic growth of computational complexity with sequence length in visual tasks, after obtaining the sequence of feature vectors with spatial position information through step S401, the sequence is further divided into a plurality of non-overlapping first windows, such as dividing 64 feature vectors into 8 groups, each group having 8 vectors, i.e., rearranging the sequence of feature vectors of the entire image in two-dimensional space; after dividing the windows, self-attention is calculated independently in each window, i.e., capturing the local dependency relationship between features in the window through the dot product interaction of query-key-value (Q-K-V), and then merging the attention outputs of all windows to generate first intermediate features, which retain the window-level local context information.

[0115] Step S4022, slide the first window division according to a preset offset, divide the first intermediate features into a plurality of second windows, and calculate self-attention in each second window, merge the shifted attention features of each second window, and obtain second intermediate features.

[0116] The first window is divided by sliding according to a preset offset (such as half of the window size), so as to generate a set of second windows different from the first window and rearranged, the second windows being combined from feature vector sequences from different first windows, such as from 8 windows to 12 partially overlapping windows; and then self-attention calculation is performed again in the second window, so that the feature vector sequences originally in different first windows can directly interact, information fusion across windows is realized, and the outputs of all second windows are combined, so that a second intermediate feature with global perception capability is obtained, which fuses a wider range of spatial context.

[0117] In step S4023, the second intermediate feature is added to the initial feature vector sequence in residual, to obtain a fused feature.

[0118] In order to alleviate the gradient vanishing problem in the deep network and as much as possible to retain the original detail information of the feature vector sequence after multiple transformations in the deep network, the second intermediate feature after the transformation by the two-layer complex window attention mechanism is further added to the original feature vector sequence input to the module in residual, and the fused feature is a fused feature containing both original low-level features and refined high-level global context information.

[0119] In step S4024, the fused feature is subjected to nonlinear transformation to obtain a deep feature.

[0120] Finally, the fused feature is further processed and enhanced, and each feature vector sequence is independently and identically subjected to nonlinear transformation, and the finally output deep feature is a highly abstract feature representation after sufficient global context modeling and nonlinear enhancement, which provides rich context information for the final high-resolution image reconstruction. This process realizes an effective balance between computational efficiency and modeling capability through the alternating iteration of window attention and residual fusion.

[0121] In some embodiments, the nonlinear transformation of the fused feature includes channel transformation and enhancement of the feature by using at least one convolution layer combined with an activation function; wherein the convolution layer includes a 3x3 convolution, and the activation function includes a ReLU or GELU function.

[0122] In the nonlinear transformation of the fused features, at least one calculation unit composed of a convolution layer and an activation function in series can be used to process the fused features. The main role of the convolution layer (such as a 3x3 convolution) is to perform the change of the channel number of the feature map and the local feature extraction, which can effectively fuse the information points adjacent in space. The activation function such as ReLU or GELU is applied to the convolution output, and the key role is to introduce the nonlinear expression capability for the whole transformation process. After the convolution layer and the activation function, the representation capability of the features can be enhanced, and the detail information such as texture and edge in the image can be better captured and expressed.

[0123] Figure 6 A flowchart of a wafer image matching method is provided for another embodiment of the present application. As shown in FIG. 5, step S50, according to the initial matching position and the high-resolution image corresponding to the to-be-tested image, the best matching position of the to-be-tested image and the template image is determined, which specifically includes the following steps: Figure 3

[0124] Step S501, according to the initial matching position, the corresponding coordinate of the initial matching position in the high-resolution image corresponding to the to-be-tested image is determined.

[0125] In the image matching process, first, the initial matching position calculated on the low-resolution image is mapped to the high-resolution image space corresponding to the to-be-tested image to complete the matching on the high-resolution image reconstructed by the neural network. The mapping process can be realized by simple coordinate scaling transformation, that is, multiplying the coordinates under the low resolution by a scaling factor (the size ratio between the high resolution and the low resolution), so as to obtain a roughly corresponding target region center point coordinate in the high-resolution image, which can be regarded as the starting point of the fine search on the high-resolution image.

[0126] Step S502, a local search window is constructed with a preset side length or radius as the center of the corresponding coordinate.

[0127] A square or circular local search window with a fixed side length is constructed in the high-resolution image with the mapped coordinate in the high-resolution image as the center, so as to realize the reduction of the search space of the image matching, and no longer need to perform blind global search in the whole high-resolution image range, thereby reducing the calculation complexity from being proportional to the image area to being proportional to the window area, which is an important guarantee for realizing efficient and accurate matching.

[0128] The size of the local search window needs to balance the calculation efficiency and the matching accuracy. If the side length is too large, irrelevant regions will be introduced and the calculation burden will be increased; if the side length is too small, the real matching position may be missed. Therefore, the size of the window should be large enough to accommodate the error of a few pixels that may exist in the initial coarse matching, and should be small enough to minimize the calculation amount and ensure the real-time performance of the matching.​

[0129] Step S503: Within the local search window, perform normalized cross-correlation calculation on the high-resolution image corresponding to the image to be tested and the template image to obtain the correlation response map.

[0130] Within the local search window, the high-resolution image patch and the original high-resolution template image are subjected to normalized cross-correlation calculation. In other words, the template image is used as a sliding window, sliding pixel by pixel across the high-resolution image to be tested within the local search window. At each offset position, the normalized cross-correlation coefficient between the corresponding regions of the template image and the image to be tested is calculated. Finally, the NCC coefficient values ​​at all offset positions constitute a correlation response map, where each pixel value represents the confidence of the template matching the image to be tested at that position, and the position of the peak indicates the best matching point.

[0131] In some embodiments, the calculation expression for normalized cross-correlation can be:

[0132]

[0133] Where R(i,j) represents the normalized cross-correlation coefficient between the local region of pixel (i,j) in the test image and the template image; S(i,j) represents the gray value of pixel (i,j) in the test image; T(m,n) represents the gray value of pixel (m,n) in the template image; S i , j(m,n) represents the pixel gray value at coordinates (m,n) in a local image patch of size M x N starting at (i,j) in the image to be tested.

[0134] Step S504: Based on the relevant response map, determine the optimal matching position between the image to be tested and the template image through centroid calculation.

[0135] To improve sub-pixel matching accuracy, sub-pixel localization algorithms such as centroid calculation or quadratic surface fitting are used within the peak region of the response map to determine the optimal matching position between the test image and the template image. Taking the centroid method as an example, it treats the response values ​​near the peak as a quality distribution. Using the maximum value of the response map as the center, it extracts the local neighborhood and calculates its centroid coordinates. By weighted averaging the response value distribution, noise interference is suppressed, resulting in a more accurate sub-pixel offset. Finally, the centroid coordinates are superimposed with the initial mapped coordinates to obtain the optimal matching position between the test image and the template image, achieving high-precision alignment between them.

[0136] Figure 7 This is a schematic diagram of the structure of a wafer image matching device provided in one embodiment of this application. Figure 7As shown, the wafer image matching device provided in this embodiment includes an image acquisition module 710, a first processing module 720, a first matching module 730, a second processing module 740, and a second matching module 750.

[0137] In this embodiment, the image acquisition module 710 is configured to acquire a to-be-tested image and a template image of a target object.

[0138] The first processing module 720 is configured to down-sample the to-be-tested image and the template image respectively to obtain low-resolution images corresponding to the to-be-tested image and the template image respectively.

[0139] The first matching module 730 is configured to determine an initial matching position of the to-be-tested image and the template image according to the low-resolution images corresponding to the to-be-tested image and the template image respectively.

[0140] The second processing module 740 is configured to up-sample the low-resolution image corresponding to the to-be-tested image based on a pre-constructed neural network to obtain a high-resolution image corresponding to the to-be-tested image.

[0141] The second matching module 750 is configured to determine a best matching position of the to-be-tested image and the template image according to the high-resolution image corresponding to the to-be-tested image and the initial matching position.

[0142] The image matching device provided in this embodiment proposes a wafer image matching process based on a "from coarse to fine" hybrid architecture. First, the to-be-tested image and the template image are input by the image acquisition module, and the first processing module is used to down-sample the images to obtain low-resolution images. Then, the first matching module is used to perform fast calculation at the low-resolution scale to determine the initial matching position at the pixel level, and the search range is compressed from global to local. Subsequently, the second processing module is used to intelligently up-sample the low-resolution to-be-tested image by using the pre-trained neural network to reconstruct a high-resolution image with rich details. Finally, the second matching module is used to perform fine matching in the small neighborhood of the high-resolution image starting from the initial position to determine the best matching position at the sub-pixel level. The device optimizes the calculation efficiency and matching accuracy in cooperation, realizes the significant improvement of the processing speed while ensuring the nanoscale positioning performance, and effectively meets the stringent requirements of high reliability and high efficiency of online detection in semiconductor manufacturing.

[0143] Figure 8 FIG. 2 shows a structural schematic diagram of a wafer image matching device provided in another embodiment of the present application. As shown, the wafer image matching device provided in this embodiment includes an image acquisition module 210, a first processing module 220, a first matching module 230, a second processing module 240, and a second matching module 250. Figure 8 As shown, the neural network provided in this embodiment is configured to up-sample the low-resolution image corresponding to the to-be-tested image to obtain a high-resolution image corresponding to the to-be-tested image, and the neural network includes an image blocking layer 810, a linear embedding layer 820, a position encoding layer 830, a feature extraction layer 840, a nonlinear mapping layer 850, and a reconstruction layer 860.

[0144] Specifically, the image segmentation layer 810 is used to divide the low-resolution image corresponding to the image under test into multiple image blocks of the same size that do not overlap.

[0145] The linear embedding layer 820 is used to map each image patch to a vector space through linear projection to obtain the feature vector corresponding to each image patch.

[0146] The location coding layer 830 includes a learnable location coding block, which is used to add a learnable spatial location code to each feature vector based on the learnable location coding block, so as to obtain a feature vector sequence with spatial location information.

[0147] The feature extraction layer 840 is used to input the feature vector sequence into the neural network for self-attention-based context feature modeling.

[0148] The nonlinear mapping layer 850 is used to perform nonlinear transformations on the modeled features to obtain deep features of the image under test.

[0149] Reconstruction layer 860 is used to upsample deep features and reconstruct a high-resolution image corresponding to the image under test.

[0150] Figure 9 This is a schematic diagram of the structure of a wafer image matching device provided in another embodiment of this application. Figure 9 As shown, in this embodiment, the feature extraction layer 840 further includes a first sliding window converter 8401, a second sliding window converter 8402, and a jump connection module 8403.

[0151] The first sliding window converter 8401 is used to divide the feature vector sequence into multiple non-overlapping first windows, calculate self-attention in each first window, merge the attention of each window, and obtain the first intermediate feature.

[0152] The second sliding window converter 8402 is used to slide the first window division according to a preset offset, divide the first intermediate feature into multiple second windows, calculate self-attention in each second window, merge the shift attention features of each second window, and obtain the second intermediate feature.

[0153] The skip connection module 8403 is used to add the second intermediate feature to the initial feature vector sequence by residual addition to obtain the fused feature, and to perform nonlinear transformation on the fused feature to obtain deep features.

[0154] In any of the above embodiments, first, the low-resolution input image is divided into non-overlapping image blocks by the image blocking layer; each image block is projected into a high-dimensional feature vector by the linear embedding layer; the position encoding layer injects learnable spatial position information into these feature vectors to form a feature sequence that preserves spatial structure; the feature extraction layer uses a self-attention mechanism to model the global context of the sequence and capture long-range dependencies; the non-linear mapping layer enhances the feature expression capability through a feedforward network; and the final reconstruction layer up-samples the deep features to generate a high-resolution image with high fidelity. Through the deep fusion of structure perception and semantic understanding, intelligent conversion from low-resolution input to high-quality reconstruction is achieved, providing a reliable data foundation for subsequent accurate matching.

[0155] The various modules in the above image matching device can be implemented in whole or in part by software, hardware, and combinations thereof. The various modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to the various modules.

[0156] The embodiment also provides a computer readable storage medium, which stores a program or instructions executed by a computer. When the program or instructions executed by the computer is executed by a processor, each process of any embodiment of the wafer image matching method can be implemented, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0157] The processor can be a central processing unit (CPU) or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0158] Those skilled in the art can understand that all or part of the functions of various methods in the above embodiments can be realized by hardware or by a computer program. When all or part of the functions are realized by a computer program, the program can be stored in a computer readable storage medium, which can include a read-only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, and the like. The above functions are realized by executing the program by a computer. For example, the program is stored in a memory of a device, and the above functions are realized by executing the program in the memory by a processor. In addition, when all or part of the functions are realized by a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash disk, or a mobile hard disk, and is saved in a memory of a local device by downloading or copying, or the system of the local device is updated, and the above functions are realized by executing the program in the memory by a processor. The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative but not restrictive. Those skilled in the art can make simple deductions, modifications, or replacements according to the idea of the present application without departing from the scope of the present application and the protection scope of the claims.

Claims

1. A wafer image matching method characterized by, The method comprises the following steps: obtaining a to-be-tested image and a template image of a target object; down-sampling the to-be-tested image and the template image respectively to obtain low-resolution images corresponding to the to-be-tested image and the template image respectively; determining an initial matching position of the to-be-tested image and the template image according to the low-resolution images corresponding to the to-be-tested image and the template image respectively; up-sampling the low-resolution image corresponding to the to-be-tested image based on a pre-constructed neural network to obtain a high-resolution image corresponding to the to-be-tested image; determining an optimal matching position of the to-be-tested image and the template image according to the high-resolution image corresponding to the to-be-tested image and the initial matching position.

2. The wafer image matching method of claim 1, wherein, The step of determining the initial matching position of the to-be-tested image and the template image according to the low-resolution images corresponding to the to-be-tested image and the template image respectively comprises the following steps: respectively performing noise reduction processing on the low-resolution images corresponding to the to-be-tested image and the template image; calculating pixel gradients of the low-resolution images corresponding to the to-be-tested image and the template image after noise reduction by using a sobel operator to obtain gradient directions and gradient amplitudes of each pixel point; determining edge information of the low-resolution images corresponding to the to-be-tested image and the template image according to the gradient directions and gradient amplitudes of each pixel point; determining the initial matching position of the to-be-tested image and the template image according to the edge information.

3. The wafer image matching method of claim 2, wherein, The noise reduction processing comprises at least one of Gaussian filtering processing, median filtering and local mean denoising; The step of calculating pixel gradients of the low-resolution images corresponding to the to-be-tested image and the template image after noise reduction by using a sobel operator comprises the following steps: calculating pixel gradients of the low-resolution images corresponding to the to-be-tested image and the template image after noise reduction by using sobel operators with different convolution kernels in X and Y directions.

4. The wafer image matching method of claim 1, wherein, The pre-constructed neural network is a hybrid network model of a super-resolution convolutional neural network and a transformer based on a sliding window.

5. The wafer image matching method of claim 4, wherein, The step of up-sampling the low-resolution image corresponding to the to-be-tested image based on the pre-constructed neural network to obtain a high-resolution image corresponding to the to-be-tested image comprises the following steps: dividing the low-resolution image corresponding to the to-be-tested image into a plurality of image blocks, and encoding each image block to obtain a feature vector sequence with spatial position information; inputting the feature vector sequence into the neural network to perform self-attention based context feature modeling, and performing nonlinear transformation on the modeled features to obtain deep features of the to-be-tested image; performing up-sampling processing on the deep features to reconstruct the high-resolution image corresponding to the to-be-tested image.

6. The wafer image matching method of claim 5, wherein, The step of dividing the low-resolution image corresponding to the to-be-tested image into a plurality of image blocks and encoding each image block to obtain a feature vector sequence with spatial position information comprises the following steps: dividing the low-resolution image corresponding to the to-be-tested image into a plurality of image blocks of the same size and without overlapping; mapping each image block to a vector space through linear projection to obtain a feature vector corresponding to each image block. According to the pre-created learnable position encoding block, learnable spatial position encodings are added to each feature vector to obtain a sequence of feature vectors with spatial position information.

7. The wafer image matching method of claim 6, wherein, The size of the pre-created position encoding block is the same as the image block size. The adding of the learnable spatial position encodings to each feature vector according to the pre-created position encoding block comprises: extracting a learnable position encoding vector in the position encoding block corresponding to the spatial position of each image block; adding the feature vector corresponding to each image block to the corresponding position encoding vector element by element to obtain the sequence of feature vectors with spatial position information.

8. The wafer image matching method of claim 6, wherein, The learnable position encoding block is equipped with learnable scaling coefficients for controlling the strength of position information injection during model training.

9. The wafer image matching method of claim 5, wherein, The sequence of feature vectors is input into the neural network for self-attention-based context feature modeling, and the modeled features are subjected to nonlinear transformation to obtain deep features of the image to be tested, comprising: dividing the sequence of feature vectors into a plurality of non-overlapping first windows, and calculating self-attention in each first window, and merging the attention of each window to obtain first intermediate features; sliding the first window division according to a preset offset, dividing the first intermediate features into a plurality of second windows, and calculating self-attention in each second window, and merging the shift attention features of each second window to obtain second intermediate features; adding the second intermediate features to the initial sequence of feature vectors in residual to obtain fused features; nonlinearly transforming the fused features to obtain deep features.

10. The wafer image matching method of claim 9, wherein, The nonlinear transformation of the fused features comprises: using at least one convolutional layer combined with an activation function for nonlinear transformation to transform and enhance the features in the channel; wherein the convolutional layer comprises a 3x3 convolution, and the activation function comprises a ReLU or GELU function.

11. The wafer image matching method of claim 1, wherein, The determining of the best matching position of the image to be tested and the template image according to the high-resolution image corresponding to the image to be tested and the initial matching position comprises: mapping the corresponding coordinates of the initial matching position in the high-resolution image corresponding to the image to be tested according to the initial matching position; constructing a local search window with a preset side length or radius centered on the corresponding coordinates; performing normalized cross-correlation calculation on the high-resolution image corresponding to the image to be tested and the template image in the local search window to obtain a correlation response map; determining the best matching position of the image to be tested and the template image through centroid calculation according to the correlation response map.

12. A wafer image matching apparatus characterized by comprising: Comprise: an image acquisition module for acquiring a test image and a template image of a target object; a first processing module for down-sampling the test image and the template image respectively to obtain low-resolution images corresponding to the test image and the template image respectively; a first matching module for determining an initial matching position of the test image and the template image according to the low-resolution images corresponding to the test image and the template image respectively; The second processing module is configured to perform up-sampling on the low-resolution image corresponding to the to-be-tested image based on a pre-constructed neural network, to obtain a high-resolution image corresponding to the to-be-tested image. The second matching module is configured to determine a best matching position of the to-be-tested image and the template image according to the high-resolution image corresponding to the to-be-tested image and the initial matching position.

13. The wafer image matching apparatus according to claim 12, wherein The neural network comprises: An image block division layer configured to divide the low-resolution image corresponding to the to-be-tested image into a plurality of image blocks of the same size and without overlap; A linear embedding layer configured to map each of the image blocks to a vector space through linear projection, to obtain a feature vector corresponding to each of the image blocks; A position encoding layer comprising a learnable position encoding block, configured to add learnable spatial position encoding to each of the feature vectors according to the learnable position encoding block, to obtain a feature vector sequence with spatial position information; A feature extraction layer configured to input the feature vector sequence to the neural network for self-attention-based context feature modeling; A nonlinear mapping layer configured to perform nonlinear transformation on the modeled features, to obtain deep features of the to-be-tested image; A reconstruction layer configured to perform up-sampling processing on the deep features, to reconstruct the high-resolution image corresponding to the to-be-tested image.

14. The wafer image matching apparatus according to claim 13, wherein The feature extraction layer at least comprises a first sliding window converter, a second sliding window converter and a skip connection module; The first sliding window converter is configured to divide the feature vector sequence into a plurality of non-overlapping first windows, and calculate self-attention in each of the first windows, to combine the attention of each window, to obtain first intermediate features; The second sliding window converter is configured to divide the first window division in a sliding manner according to a preset offset, to divide the first intermediate features into a plurality of second windows, and calculate self-attention in each of the second windows, to combine the shift attention features of each of the second windows, to obtain second intermediate features; The skip connection module is configured to add a residual to the second intermediate features and the initial feature vector sequence, to obtain fused features, and perform nonlinear transformation on the fused features, to obtain deep features.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer-executed programs or instructions, which are executed by the processor to implement the steps of the wafer image matching method according to any one of claims 1-11.

Images