A large-size image splicing method, a wafer detection system and a storage medium

Abstract

This application relates to the field of semiconductor inspection technology, specifically a wafer surface defect detection method, including an ultra-large image stitching method, a wafer inspection system, and a storage medium. The embodiments of this application process overlapping regions using a frequency domain phase correlation method, achieving sub-pixel-level translation calculation and providing high-precision local relative constraints for global stitching. By constructing a weighted graph optimization model and introducing a confidence-based screening and iterative reweighting mechanism, erroneous local matches can be automatically identified and eliminated, effectively suppressing error accumulation and ensuring that even in large-scale scenarios where local registration failures are possible, the globally optimal stitching result can still be output. The embodiments of this invention significantly reduce the search range and complexity of subsequent image registration, thereby significantly improving the image stitching efficiency and accuracy of ultra-large wafers under high-magnification inspection, laying a reliable foundation for subsequent accurate defect localization and classification.

Description

Technical Field

[0001] This application relates to the field of semiconductor inspection technology, specifically a wafer surface defect detection method, an ultra-large image stitching method, a wafer inspection system, and a storage medium. Background Technology

[0002] In the process of AOI inspection of wafer semiconductors, under certain process conditions, the field of view of the camera imaging is often insufficient for a single complete die, especially for ultra-large dies (up to 100mm*100mm and above). In order to ensure the detection capability of defects, high magnification lenses of 10x and above are required, and the field of view of the camera will be within 2mm or even smaller.

[0003] In the field of wafer AOI inspection, the performance of the equipment is guaranteed to a certain extent, and the repeatability accuracy of the motion axis is generally around 0.1µm. However, due to environmental factors such as flash delay and equipment vibration, the final image position can generally be controlled within 5µm. At this point, an algorithm is needed to perform secondary high-precision position determination. Summary of the Invention

[0004] To address the technical problems in the prior art, embodiments of this application provide a method for stitching ultra-large images, a wafer inspection system, and a storage medium, through which...

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, a method for stitching ultra-large images is provided. The method includes: determining the overlapping region of multiple sub-images based on a preset standard overlapping region ratio, and determining the offset of multiple adjacent sub-images with respect to the overlapping region; constructing the relative positions and confidence scores between adjacent sub-images based on the offsets; using the spatial coordinates of each sub-image as a global position constraint, using the confidence scores between each sub-image as a local position constraint, and constructing a global optimization model to determine the optimal stitching position of each sub-image in the global image; and stitching all the sub-images together based on the optimal stitching position to obtain a global image.

[0006] In some specific implementations, determining the offset of multiple adjacent sub-images with respect to the overlapping region includes: determining the adjacent image pair corresponding to each sub-image based on a first direction and a second direction, determining the overlapping region image between each sub-image in each adjacent image pair, obtaining the image features of the overlapping region image, and determining the offset based on the distribution of the image features.

[0007] In some specific implementations, the image features are the spatial distribution of overlapping region images, and the distribution includes the cyclic correlation peaks of the overlapping region images.

[0008] In some specific implementations, obtaining the image features of the overlapping region includes: performing a discrete Fourier transform on the image of the overlapping region to obtain the frequency domain of the image of the overlapping region, and performing cross-power spectrum calculation and inverse transform on the frequency domain of the image of the overlapping region to obtain the cyclic correlation peak of the image of the overlapping region.

[0009] In some specific implementations, determining the offset based on the image feature distribution includes: fitting the sub-pixel position of the cyclic correlation peak to obtain the offset of the adjacent image pair with respect to a first direction or a second direction.

[0010] In some specific implementations, the global optimization model includes nodes and edges. The nodes are used to represent the sub-image, and the edges are used to represent the constraint relationships between the nodes. The constraint relationships include global position constraints and local position constraints.

[0011] In some specific implementations, determining the optimal stitching position of each sub-image in the global image includes: constructing initial coordinate points using vertex sub-images, obtaining the prior constraint edges and relative constraint edges corresponding to each sub-image based on a preset path, and determining the optimal stitching position sequentially based on the weight classification of the relative constraint edges; the weight of the relative constraint edges is determined based on the confidence score.

[0012] Secondly, a wafer inspection system is provided for surface inspection of a wafer, wherein the wafer includes multiple units to be inspected, and the system includes: an image acquisition device configured with an image acquisition field of view, which acquires images of each unit to be inspected in the wafer based on an acquisition path, and obtains multiple sub-images of the unit to be inspected within the image acquisition field of view; and an image processing device that receives the multiple sub-images and executes the ultra-large image stitching method described in any of the above-mentioned methods to obtain a complete image of the unit to be inspected.

[0013] In some specific implementations, the image processing device includes: determining the overlapping region of multiple sub-images based on a preset standard overlapping region ratio, and determining the offset of multiple adjacent sub-images with respect to the overlapping region; constructing the relative position and confidence score between adjacent sub-images based on the offset; using the spatial coordinates of each sub-image as a global position constraint, using the confidence score between each sub-image as a local position constraint, and constructing a global optimization model to determine the optimal stitching position of each sub-image in the global image; and stitching all the sub-images together based on the optimal stitching position to obtain the global image.

[0014] Thirdly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer-executable instructions, which, when called and executed by a processor, cause the processor to implement the ultra-large image stitching method described in any of the above claims.

[0015] The embodiments of the present invention bring the following beneficial effects: In the technical solution provided by this application, the overlapping region is processed by the frequency domain phase correlation method, which enables sub-pixel level translation calculation and provides high-precision local relative constraints for global stitching. By constructing a weighted graph optimization model and introducing a confidence-based screening and iterative reweighting mechanism, erroneous local matches can be automatically identified and eliminated, effectively suppressing error accumulation and ensuring that the globally optimal stitching result can still be output even in large-scale situations where local registration failures may occur. This application embodiment greatly reduces the search range and complexity of subsequent image registration, thereby significantly improving the image stitching efficiency and accuracy of ultra-large wafers under high-magnification detection, laying a reliable foundation for subsequent accurate defect location and classification.

[0016] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0017] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] The methods, systems, and / or procedures shown in the accompanying drawings will be further described with reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein example figures represent similar mechanisms in the various views of the drawings.

[0020] Figure 1 This is a schematic diagram of a wafer inspection system.

[0021] Figure 2 This is a schematic diagram of the distribution of sub-images.

[0022] Figure 3 This is a schematic diagram of the image stitching method.

[0023] Figure 4 This is a schematic diagram of the scanning path.

[0024] Figure 5 This is a schematic diagram of the image processing device.

[0025] Figure 6 This is a schematic diagram of a readable medium structure. Detailed Implementation

[0026] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0027] In the detailed description below, numerous specific details are illustrated with examples to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that this application can be practiced without these details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high level without detail to avoid unnecessarily obscuring aspects of this application.

[0028] This application uses flowcharts to illustrate the execution process performed by a system according to embodiments of this application. It should be clearly understood that the execution processes in the flowcharts may not be executed sequentially. Instead, these execution processes may be executed in reverse order or simultaneously. Additionally, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.

[0029] This application provides a wafer inspection system for detecting defects on the surface of a wafer, including abnormalities such as breakage, scratches, and contamination. Specifically, the system employs automated optical inspection, acquiring image information of the wafer surface to be inspected and determining whether the wafer has corresponding defects based on pixel features in the image. In this embodiment, "wafer" generally refers to a substrate formed of semiconductor or non-semiconductor materials. Examples include (but are not limited to) single-crystal silicon, gallium arsenide, gallium nitride, and indium phosphide. Such substrates are typically found and / or processed in semiconductor manufacturing facilities. In some cases, a wafer may contain only a substrate (i.e., a bare wafer). Alternatively, a wafer may contain one or more different material layers formed on the substrate. The one or more layers formed on the wafer may be "patterned" or "unpatterned." For example, a wafer may contain multiple bare wafers with repeatable pattern features.

[0030] A wafer has multiple dies. Typically, the target of automated optical inspection of a wafer is each die. This can be understood as the wafer inspection system provided in this application acquiring an image of each die on the wafer to be inspected, and determining whether each die has defined defects based on the pixel features expressed in the die image.

[0031] Furthermore, in certain wafer fabrication processes, the imaging field of view of wafer inspection systems often cannot meet the requirements of a single complete die, especially for ultra-large dies with dimensions of 100mm*100mm and above. To ensure defect detection capabilities, high-magnification lenses of 10x or higher are required, resulting in a camera field of view of less than 2mm or even smaller. Therefore, for such wafer fabrication processes, this embodiment requires image acquisition of each die within the wafer to be inspected based on the corresponding acquisition path using the image acquisition field of view, obtaining multiple sub-images of each die, and then stitching these multiple sub-images together to obtain a complete image of the die. This solution needs to address a pressing technical problem: how to achieve seamless stitching of massive amounts of high-resolution sub-images with sub-pixel precision to ensure the accuracy and reliability of defect detection.

[0032] Generally, under conditions of stable wafer spatial positioning and accurate imaging field of view control, multiple sub-images have strict spatial relationships, which can be used to stitch together multiple sub-images. However, it is worth noting that although the mechanical motion axes of the wafer inspection system have extremely high repeatability accuracy, this is not equivalent to the actual positioning accuracy of the image. During high-speed scanning and imaging, dynamic environmental factors such as flash delay, machine vibration, and thermal drift can introduce unpredictable errors. The accumulation of these errors may cause the actual position of the image to deviate from the expected position. For submicron-level defect detection scenarios, this level of misalignment is sufficient to cause artifacts at the stitching seam or to segment tiny defects into two sub-images that cannot be identified, resulting in serious missed or false detections.

[0033] In existing technologies, image stitching methods mainly rely on two approaches. One is mechanical stitching of multiple images using platform coordinates, the accuracy of which is limited by the machine's motion error. The second is a pixel-level registration algorithm based on image grayscale features. However, due to the high repetition, low contrast, and complex texture backgrounds of wafer images, this method faces difficulties in feature extraction, resulting in a high matching error rate. Furthermore, the second algorithm can only achieve pixel-level integer displacement accuracy, failing to meet sub-pixel-level stitching requirements. Moreover, when calculating the global position of all image acquisition fields, it lacks a quantitative assessment of the reliability of each registration result, making it prone to global stitching failure due to local mismatches.

[0034] Furthermore, the inspection of ultra-large wafers requires processing massive amounts of high-resolution images; each wafer may require processing hundreds of thousands of images, and the inspection process demands extremely high throughput, for example, wafer inspection needs to be completed within sub-seconds. These processing requirements pose a significant challenge to the system's computing power.

[0035] To solve the above technical problems, please refer to Figure 1 This embodiment provides a wafer inspection system 10 for wafer surface inspection. The system includes an image acquisition device 11, a server 12, and an image processing device 13. The image acquisition device acquires multiple sub-images corresponding to each die on the wafer to be inspected according to an acquisition path, and transmits these sub-images to the server. The server stores the sub-images according to a task and transmits them to the image processing device. The image processing device is equipped with an image processing method. This method determines the offset between the multiple sub-images, determines the spatial relationship of the multiple sub-images under the die layout conditions based on the offset, and stitches the multiple sub-images together based on this spatial relationship to construct a complete image corresponding to the unit to be inspected.

[0036] The image acquisition device is a camera with a field of view. It acquires images of each unit to be inspected on the wafer based on a pre-set image acquisition path, obtaining multiple sub-images of the unit in a single acquisition task. The acquisition path is a serpentine path, using any edge corner of the unit to be inspected as the starting position for scanning, and acquiring multiple sub-images of the unit based on this serpentine path.

[0037] For details regarding the acquisition path and the distribution of each sub-image, please refer to [link / reference]. Figure 2 ,pass Figure 2As can be seen, the path of the image acquisition device in this embodiment starts from the upper left corner of the unit to be detected and traverses the unit to be detected row by row and column by column according to the path from left to right and from right to left, thereby obtaining multiple sub-images of the unit to be detected. Each sub-image can be called an independent field-of-view image. In this embodiment, there are nine independent field-of-view images for the unit to be detected. Of course, in other embodiments, depending on the changes in the image acquisition field of view and the size of the unit to be detected, there may be other numbers of independent field-of-view images for the unit to be detected. In this embodiment, there is no limitation on the number of sub-images in the unit to be detected; this number is configured accordingly based on the die size and image acquisition field of view in the actual process.

[0038] The image processing device receives multiple sub-images acquired by the above acquisition logic and stitches the multiple sub-images together to obtain a complete image of the unit to be detected.

[0039] Furthermore, regarding the image processing device's process of stitching sub-images, please refer to [link / reference needed]. Figure 3 This includes the following steps: Step S31. Determine the overlapping area of ​​multiple sub-images based on a preset standard overlapping area ratio, and determine the offset of multiple adjacent sub-images with respect to the overlapping area. Construct the relative position and confidence score between adjacent sub-images based on the offset.

[0040] In this embodiment, during the image acquisition phase, the image acquisition device acquires images of the unit to be detected according to a preset scanning path, obtaining a large number of high-resolution sub-images. Each sub-image has the same size based on the image acquisition field of view.

[0041] The overlap ratio refers to the proportion of overlapping areas between adjacent sub-images caused by the image acquisition field-of-view configuration logic at the edge positions. This image acquisition field-of-view configuration logic, in order to ensure that information from all sub-images is acquired and to prevent wafer displacement due to system vibration, updates the image acquisition position by setting an overlap region between adjacent sub-images. Sub-image acquisition is then performed based on the updated image acquisition position and the image acquisition field of view.

[0042] Furthermore, in this embodiment, the overlapping region includes the overlapping region between sub-images distributed along the first direction and the overlapping region between sub-images distributed along the second direction. Here, the first direction refers to the direction in which the wafer to be inspected is set laterally, and the second direction refers to the direction in which the wafer to be inspected is set vertically. This can be understood as multiple sub-images being distributed in rows and columns formed along the first and second directions.

[0043] For details, please refer to Figure 2 This diagram shows the distribution of multiple sub-images within the unit to be detected. Each sub-image represents a sub-image acquired under a specific image acquisition field of view. Based on the size of the image acquisition field of view in this diagram, the unit to be detected can be understood as consisting of nine sub-images, each configured with a corresponding sequence number according to the image scanning path. Specifically, the sequence numbers in this embodiment are set based on the corresponding field of view. For example, the image within the first field of view in the image scanning path is the first field of view image, with the sequence number Fov.1. Similarly, the corresponding sub-images are Fov.2, Fov.3, Fov.4, Fov.5, Fov.6, Fov.7, Fov.8, and Fov.9.

[0044] In this embodiment, the image scanning path refers to the scanning imaging path for acquiring the aforementioned multiple sub-images, through... Figure 4 As can be seen, the scanning path in this embodiment is a serpentine scanning path. Specifically, this scanning path starts from the upper left corner of the unit to be detected and completes the scan along a zigzag route. Based on this scanning path, the sub-images form a corresponding row and column distribution, and for each sub-image, the upper left sub-image can be used as the starting coordinate point to construct the initial relative coordinate position of multiple sub-images. It can be understood that the spatial coordinate position of the upper left sub-image is set as an absolute position, and then the multiple sub-images determine their relative positions with respect to each of the other sub-images by obtaining the spatial relativity with respect to the upper left sub-image, and then the stitching of each sub-image is guided based on this relative position.

[0045] Specifically, the purpose of the splicing logic in this embodiment is to obtain the relative positional relationship between sub-images and to splice multiple sub-images based on this relative positional relationship.

[0046] In this embodiment, the offset of the overlapping region is used to evaluate whether the sub-images conform to the standard overlap distribution. If the offset of the overlapping region between the current sub-images conforms to the standard overlap distribution, it indicates that there is no systematic error between the two sub-images, and their positional relationship can be directly calculated using the set standard offset. If the offset of the current overlapping region does not conform to the standard overlap distribution, it indicates that there is a deviation between the two sub-images as described in the background art, and this deviation needs to be fused when calculating their spatial position.

[0047] The determination of the overlapping region offset involves first pairing each adjacent sub-image to form a sub-image pair. Then, based on the positional distribution of the adjacent sub-images, the overlapping region between them is determined as either a first overlapping region or a second overlapping region. Further, the first overlapping region refers to the overlap of the sub-image pair along a first direction, and the second overlapping region refers to the overlap of the sub-image pair along a second direction. Then, based on a set standard overlap distribution, the corresponding overlapping region image on the two sub-images is determined. The standard overlap distribution refers to the proportion of the overlapping region to the total image distribution of the sub-images. The overlapping region is obtained by performing image segmentation on the corresponding sub-image based on the standard overlap distribution to obtain the overlapping region image corresponding to that sub-image. For example, in this embodiment, the standard overlap distribution is 5%. Therefore, in this embodiment, the overlapping region is determined by cropping the image at a 5% area ratio in the adjacent positions of the two sub-images in the sub-image pair. The resulting cropped image is the overlapping region image corresponding to the overlapping region.

[0048] Furthermore, the overlapping region image is obtained using a standard overlapping region ratio. However, under actual operating conditions, the wafer under inspection and / or the image acquisition device may have motion errors or deviations, resulting in a certain amount of offset in the overlapping region. Therefore, in this embodiment, it is necessary to calculate the offset of the overlapping region. This offset is determined based on the distribution of image features within the overlapping region.

[0049] Specifically, if there are no errors or deviations in the wafer under inspection and / or the image acquisition device, the image corresponding to the overlapping area should contain two sub-images with a fixed proportion, meaning the two sub-images actually overlap. However, if there are errors in the wafer under inspection and / or the image acquisition device, the proportions of the two sub-images in the image corresponding to the overlapping area will not meet the preset requirements. For example, relative to the first and second sub-images, under the condition that there are no errors or deviations in the system, the image of the overlapping area based on the standard overlapping area proportion should include 5% of the first sub-image and 5% of the second sub-image, with the sub-images of the above two proportions forming the corresponding overlapping area image. In reality, when there are errors or deviations in the system, for example, if the image acquisition device moves to the right during image acquisition, the overlapping area between the acquired second sub-image and the first sub-image decreases accordingly, resulting in less image information about the first sub-image in the overlapping area determined based on the standard overlapping area proportion.

[0050] Therefore, in this embodiment, the determination of the offset of a sub-image is based on the distribution of image features in the image corresponding to the overlapping region to determine whether an offset exists. Furthermore, in this embodiment, the offset is not an absolute offset position, but a relative degree of offset.

[0051] Specifically, determining this offset first involves acquiring image features of the overlapping region images. In this embodiment, these image features refer to the spatial distribution of the overlapping region images. Specifically, the spatial distribution is determined by performing a discrete Fourier transform on the overlapping region images to obtain their frequency domain representation. Then, the cross-power spectrum of the overlapping region images in adjacent images is calculated and inversely transformed to obtain the cyclic correlation peaks of the overlapping region images in adjacent images.

[0052] Unlike traditional methods that directly match feature points in the spatial domain, this step transforms the overlapping region images into the frequency domain for processing. Translation relationships in the spatial domain manifest as phase changes in the frequency domain. Utilizing this characteristic, the frequency domain transformation method is highly robust to interferences such as image noise and uneven illumination, and can more accurately capture the geometric displacement relationships between images.

[0053] Specifically, the determination of the cyclic correlation peak first involves performing a Discrete Fourier Transform on two adjacent sub-images to obtain their corresponding frequency domain values. Based on this, the cross-power spectrum of the two images is calculated and normalized, preserving their phase information. Finally, an Inverse Fourier Transform is performed on the cross-power spectrum to obtain the cyclic correlation peak in the spatial domain. This cyclic correlation peak typically presents as a sharp impulse function, with its peak position directly corresponding to the integer pixel offsets between the two images in the horizontal and vertical directions. It should be understood that the frequency domain phase correlation method utilizes the translation invariance of the Fourier Transform, accurately detecting displacement information even in images with repetitive textures or low-contrast regions, effectively avoiding mismatch problems that easily occur in spatial domain feature matching.

[0054] Furthermore, due to hardware errors and / or mechanical vibrations in the system, the actual offset between adjacent sub-images is often not at the integer pixel level, but rather a tiny displacement at the sub-pixel level. Therefore, directly using the integer coordinates of the cyclic correlation peak as the offset would introduce quantization errors, failing to meet the requirements of high-precision stitching. Therefore, after obtaining the integer peak position of the cyclic correlation peak, this embodiment performs surface fitting within the neighborhood of the corresponding peak. Preferably, this embodiment employs a Gaussian surface fitting method, fitting a continuous Gaussian surface function using the correlation response values ​​of the peak point and its surrounding points. By solving for the coordinates of the extreme points of this Gaussian surface, the sub-pixel level offset can be obtained. This setup aims to improve the geometric accuracy of image stitching, providing high-precision observation data for subsequent global optimization.

[0055] Furthermore, the obtained sub-pixel offsets are constructed as a two-dimensional affine transformation matrix, which formally describes the relative geometric positional relationship between adjacent sub-images. Simultaneously, a confidence score for this registration is calculated based on this sub-pixel offset. The confidence score is determined based on the positional relationship between the sub-pixel offset and a pre-defined overlapping region; it can be understood as the degree of deviation between the positional relationship between the current sub-images and the standard overlap relationship. The smaller the deviation, the higher the confidence score. Therefore, by using sub-pixel offsets, the confidence score between each sub-image can be determined.

[0056] Step S32. Use the spatial coordinates of each sub-image as a global position constraint, use the confidence score between each sub-image as a local position constraint, and construct a global optimization model to determine the optimal stitching position of each sub-image in the global image.

[0057] In this embodiment, this process is the core of the stitching method. This process achieves the fusion and error elimination of multiple sub-images by constructing a global optimization model. The spatial coordinates of each sub-image are typically derived from hardware feedback from the image acquisition system. These coordinates provide an absolute position estimate of the sub-image in the global coordinate system, constituting a global position constraint.

[0058] The role of global position constraints is to provide a stable anchor point for the entire stitching network. In the process of long-distance and large-scale stitching, the continuous accumulation of phase position errors leads to severe deformation of the global image.

[0059] Furthermore, the local positional constraints are derived from the relative positional relationships between adjacent sub-images. Since the relative position is calculated based on the acquired image content, its accuracy is higher than that of the mechanical platform's positioning accuracy, thus effectively correcting local errors between adjacent sub-images. However, local constraints also have uncertainties, meaning that incorrect matching may occur. Therefore, in this embodiment, the confidence score of adjacent sub-images is used as a weight in the global optimization model for local positional constraints. Relative positional constraints with higher confidence scores have a greater weight in the optimization and a greater impact on the final result.

[0060] In the global optimization model, global and local positional constraints complement each other: global constraints ensure macroscopic positional accuracy and suppress the generation of cumulative errors; local constraints ensure seamless stitching between adjacent images at the microscopic level and correct local errors. By solving this global optimization model, the system can calculate the optimal stitching position of each sub-image in the global image, which conforms to both the macroscopic layout of the global coordinate system and the high-precision relative positional relationship between adjacent images.

[0061] In this embodiment, the global optimization model includes nodes and edges. Specifically, nodes represent sub-images, and edges represent the constraint relationships between nodes, including global positional constraints and local positional constraints.

[0062] This embodiment can be understood as modeling the large-scale image stitching problem as a mathematical optimization problem. In the graph model, each sub-image is set as a node, and the state variable of this node is the global pose to be determined, usually represented as two-dimensional planar coordinates. The edges in the graph model connect different nodes and are used to express the geometric constraints between nodes. Furthermore, in this embodiment, the initial node is set as the first image in the upper left corner. It can be understood that the first image in the upper left corner has initial coordinates (0,0), and the subsequent stitching of nodes in the global model is based on this initial coordinate to determine the spatial position.

[0063] In this embodiment, edges are mainly divided into two categories. The first category is prior constraint edges, which correspond to global position constraints. These edges connect each sub-image node to a virtual global anchor point. Furthermore, the spatial positional relationship of these edges originates from the initial spatial position coordinates provided by the image acquisition device, i.e., the hardware feedback coordinates. The role of prior constraint edges is to provide an absolute coordinate system reference for the entire optimization system, preventing overall drift in the optimization results and ensuring that the stitched global image corresponds to the actual physical location. The second category is relative constraint edges, which correspond to local position constraints. These edges connect adjacent sub-image nodes, and the spatial positional relationship of these edges originates from the relative positions between adjacent sub-images calculated in step S31. The role of relative constraint edges is to correct local geometric relationships using high-precision image registration results, eliminating stitching gaps between adjacent images. It should be understood that each relative constraint edge has a weight, which is determined by the confidence score calculated in this embodiment. The higher the confidence score, the greater the update weight of the edge in the optimization.

[0064] Specifically, when determining the optimal stitching position, the first sub-image in the upper left corner is first obtained as the base position, and all sub-images and their corresponding prior constraint edges and relative constraint edges are traversed row by row and column by column. Then, the optimal stitching position is determined sequentially based on the weight classification of the relative constraint edges.

[0065] In this context, weighted grading refers to prioritizing the optimal stitching position of nodes corresponding to high-weight relative constraint edges when determining the stitching position. Further, firstly, nodes with relative constraint edge weights of 0.9 or higher are selected from all sub-points. These selected nodes are used as the first-round selection nodes. The associated sub-images corresponding to these selected nodes and their offsets relative to the selected nodes are determined. Then, based on the offsets and the prior constraints corresponding to the selected nodes, the relative position with respect to these sub-images is determined. Here, "associated sub-images" refers to sub-images that are related to this relative constraint edge.

[0066] Furthermore, in this embodiment, the weight configuration in the weight grading includes 0.9-0.3. This can be understood as follows: after filtering nodes with a weight of 0.9 or higher, nodes corresponding to weights of 0.8-0.9 are filtered based on the above process until the filtering of nodes corresponding to a weight of 0.3 is completed. When the weight of a node is less than 0.3, the corresponding subgraph is a low-evaluation image, indicating that there is no overlapping area with evaluation value, and the spatial position corresponding to the prior constraint variable can be directly used as its relative spatial position.

[0067] In this embodiment, the process of determining the optimal stitching position is a forward process. In other embodiments, the position corresponding to the first image at the bottom right corner of the object to be detected can be used as the base position, and all sub-images and the prior constraint edges and relative constraint edges corresponding to each sub-image can be traversed in reverse order row by row and column by column. Then, the optimal stitching position is determined sequentially based on the weight hierarchy of the relative constraint edges. The processing method for this process is the same as that of the forward process, and the spatial relative positions corresponding to multiple nodes are determined sequentially based on the weight hierarchy.

[0068] Step S33. Based on the optimal stitching position, stitch all the sub-images together to obtain the global image.

[0069] In this embodiment, after determining the optimal stitching position for each sub-image, all sub-images are mapped to a unified global coordinate system. In the projected image, a weighted fusion method is used for overlapping areas. Specifically, the Hanning window function is used for stitching fusion in this embodiment. The Hanning window function assigns a function that gradually decays from the center to the edge to the overlapping portion of each sub-image near the edge. By weighting and averaging pixel values ​​at the same location from different viewpoints according to their respective window function weights, a smooth and seamless filter can be generated, effectively eliminating stitching seams and ghosting.

[0070] Furthermore, the generated seamless, high-resolution panoramic image of the ultra-large grain can be input into a subsequent defect detection algorithm to detect and identify defects in the image, and the detection and identification results are synchronized to the user interface. The defect detection algorithm is configured within the server and is invoked and executed based on the corresponding scenario and detection task. The processing details of this method will not be elaborated upon in this embodiment.

[0071] In summary, this application provides a method for stitching ultra-large images. By processing overlapping regions using the frequency domain phase correlation method, sub-pixel-level translation calculations can be achieved, providing high-precision local relative constraints for global stitching. By constructing a weighted graph optimization model and introducing a confidence-based filtering and iterative reweighting mechanism, erroneous local matches can be automatically identified and eliminated, effectively suppressing error accumulation and ensuring that the globally optimal stitching result can still be output even in large-scale situations where local registration failures may occur. This application significantly reduces the search range and complexity of subsequent image registration, thereby significantly improving the efficiency and accuracy of image stitching for ultra-large wafers under high-magnification detection, laying a reliable foundation for subsequent accurate defect localization and classification.

[0072] For further details, please refer to [link / reference]. Figure 5 This is a schematic diagram of the structure of an image processing apparatus. This apparatus is used to perform the processing steps S31-S33. The image processing apparatus 13 includes: The offset evaluation module 131 determines the overlapping area of ​​multiple sub-images based on a preset standard overlapping area ratio, and determines the offset of multiple adjacent sub-images with respect to the overlapping area. Based on the offset, it constructs the relative position and confidence score between adjacent sub-images. The position update module 132 uses the spatial coordinates of each sub-image as a global position constraint, the confidence score between each sub-image as a local position constraint, and constructs a global optimization model to determine the optimal stitching position of each sub-image in the global image. The stitching module 133 stitches all the sub-images together based on the optimal stitching position to obtain a global image. See Figure 6 The present invention also provides a readable medium 60, which stores computer-readable instructions 601, including instructions for performing the aforementioned ultra-large image stitching method.

[0073] The functions and technical effects of the readable medium 60 provided in this embodiment of the invention can be referred to the technical effects of the calibration method in the foregoing embodiments, and will not be repeated here.

[0074] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and / or methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units / modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0075] The units / modules described as separate components may or may not be physically separate. The components shown as units / modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units / modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] Furthermore, in the various embodiments of the present invention, the functional units / modules can be integrated into one processing unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated into one unit / module. The integrated unit / module described above can be implemented in hardware or in the form of hardware plus software functional units / modules.

[0077] The integrated unit / module implemented as a software functional unit / module described above can be stored in a computer-readable storage medium. The software functional unit, stored in a storage medium, includes several instructions to cause one or more processors of a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention.

[0078] The integrated unit / module implemented as a software functional unit / module described above can be stored in a computer-readable storage medium. The software functional unit, stored in a storage medium, includes several instructions to cause one or more processors of a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention.

[0079] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for stitching ultra-large images, characterized in that, The method includes: The overlapping regions of multiple sub-images are determined based on a preset standard overlapping region ratio, and the offsets of multiple adjacent sub-images with respect to the overlapping regions are determined. The relative positions and confidence scores between adjacent sub-images are constructed based on the offsets. The spatial coordinates of each sub-image are used as global position constraints, the confidence scores between each sub-image are used as local position constraints, and a global optimization model is constructed to determine the optimal stitching position of each sub-image in the global image. All the sub-images are stitched together based on the optimal stitching position to obtain the global image.

2. The ultra-large image stitching method according to claim 1, characterized in that, Determining the offset of multiple adjacent sub-images with respect to an overlapping region includes: determining the adjacent image pair corresponding to each sub-image based on a first direction and a second direction, determining the overlapping region image between each sub-image in each adjacent image pair, obtaining the image features of the overlapping region image, and determining the offset based on the image feature distribution.

3. The ultra-large-scale image stitching method according to claim 2, characterized in that, The image features are the spatial distribution of overlapping region images, and the distribution includes the cyclic correlation peaks of the overlapping region images.

4. The ultra-large-scale image stitching method according to claim 3, characterized in that, Obtaining the image features of the overlapping region includes: performing a discrete Fourier transform on the image of the overlapping region to obtain the frequency domain of the image of the overlapping region, and performing cross-power spectrum calculation and inverse transform on the frequency domain of the image of the overlapping region to obtain the cyclic correlation peak of the image of the overlapping region.

5. The ultra-large image stitching method according to claim 3, characterized in that, Determining the offset based on the image feature distribution includes: fitting the subpixel position of the cyclic correlation peak to obtain the offset of the adjacent image pair with respect to a first direction or a second direction.

6. The ultra-large image stitching method according to claim 1, characterized in that, The global optimization model includes nodes and edges. The nodes are used to represent the sub-image, and the edges are used to represent the constraint relationships between the nodes. The constraint relationships include global position constraints and local position constraints.

7. The ultra-large image stitching method according to claim 6, characterized in that, Determining the optimal stitching position of each sub-image in the global image includes: constructing initial coordinate points using vertex sub-images, obtaining the prior constraint edges and relative constraint edges corresponding to each sub-image based on a preset path, and determining the optimal stitching position sequentially based on the weight classification of the relative constraint edges; the weight of the relative constraint edges is determined based on the confidence score.

8. A wafer inspection system, characterized in that, The system is applied to wafer surface inspection, wherein the wafer includes multiple units to be inspected, and the system includes: An image acquisition device is configured with an image acquisition field of view, and performs image acquisition on each of the units to be inspected in the wafer to be inspected based on the acquisition path, thereby acquiring multiple sub-images of the units to be inspected within the image acquisition field of view. An image processing device receives a plurality of said sub-images and executes the ultra-large image stitching method according to any one of claims 1-7 to obtain a complete image of the unit to be detected.

9. The wafer inspection system according to claim 8, characterized in that, The image processing device includes: The offset evaluation module is used to determine the overlapping area of ​​multiple sub-images based on a preset standard overlapping area ratio, and to determine the offset of multiple adjacent sub-images with respect to the overlapping area. Based on the offset, the module constructs the relative position and confidence score between adjacent sub-images. The position update module is used to take the spatial position coordinates of each sub-image as a global position constraint, take the confidence score between each sub-image as a local position constraint, and construct a global optimization model to determine the optimal stitching position of each sub-image in the global image. The stitching module is used to stitch all the sub-images together based on the optimal stitching position to obtain a global image.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the ultra-large image stitching method according to any one of claims 1-7.

Images