A method and apparatus for reconstructing image alignment
By combining block registration and similarity score screening with morphological completion mechanism, the problem of insufficient image integrity and accuracy caused by non-rigid deformation in large-size industrial images is solved, realizing high-precision image reconstruction at the sub-pixel level and improving the robustness and visual integrity of the detection system.
Patent Information
- Application Number
- CN202611097534.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-23
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to accurately describe non-uniform offset distributions in large-size industrial images, resulting in insufficient integrity and detection accuracy of reconstructed images. In particular, when faced with local non-rigid deformations, existing methods lack a quantitative assessment of matching reliability and a collaborative mechanism to ensure the continuity of the offset field, leading to the inability to effectively eliminate abnormal matching points and uncontrollable quality of invalid region completion.
By registering the image to be registered with the reference image in blocks to obtain local offset information, and combining similarity score filtering and morphological completion mechanism to construct a spatially continuous offset field, a remapping relationship is established to achieve sub-pixel level image reconstruction, low-confidence abnormal offset points are eliminated, and invalid regions are adaptively filled by morphological dilation and iterative diffusion to construct a continuous offset field.
It significantly improves the alignment accuracy and reconstruction quality of large-size industrial images under non-rigid deformation conditions, meets the requirements of detection systems for high precision, high robustness and visual integrity, and overcomes the physical distortion and void residue problems caused by simple interpolation.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This application relates to the field of semiconductor detection technology, specifically an intelligent semiconductor detection method and apparatus, which is a method and apparatus for reconstructing image alignment. Background Technology
[0002] In industrial applications such as semiconductor wafer inspection, PCB automated optical inspection (AOI), and display panel defect identification, image registration is a fundamental step in achieving high-precision defect detection. Its basic principle is to spatially align the image to be inspected with a pre-acquired reference image, establishing a precise correspondence between the two images at the pixel level, thereby extracting true defect information through differential comparison. As manufacturing processes continue to evolve towards higher density and miniaturization, the structural features of the objects to be inspected are constantly shrinking, raising the requirements for image registration accuracy to the sub-pixel level.
[0003] While existing technologies offer reconstruction methods for large-size images, they all face common limitations when dealing with the prevalent local non-rigid deformations found in large industrial images. On the one hand, most methods rely on global parameter models or single deformation assumptions, making it difficult to accurately describe spatially dependent non-uniform offset distributions. On the other hand, even when using local block strategies to obtain sparse displacement samples, there is a lack of quantitative evaluation of matching reliability and a collaborative mechanism to ensure the continuity of the offset field. This results in the inability to effectively remove abnormal matching points and uncontrollable quality of invalid region completion, ultimately affecting the integrity and detection accuracy of the reconstructed image. Summary of the Invention
[0004] To address the technical problems in the prior art, this application provides a method and apparatus for reconstructing image alignment. The method involves performing block registration between the image to be registered and a reference image to obtain local offset information, constructing a spatially continuous offset field by combining similarity score filtering and morphological completion mechanisms, and establishing a remapping relationship based on the continuous offset field to achieve sub-pixel-level image reconstruction. This enables high-precision non-rigid registration and continuous offset field reconstruction of large-size industrial images.
[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, a method for reconstructing image alignment is provided. The method includes: retrieving a reference image corresponding to the image to be registered; obtaining sub-image pairs corresponding to the image to be registered and the reference image based on spatial correspondence; obtaining a set of relative offset points and corresponding similarity scores for the two sub-images in each sub-image pair; determining the offset results in the offset point set based on the relationship between a preset threshold and the similarity scores; constructing an initial offset field based on the effective offset results; and completing the initial offset field based on morphology to construct a continuous offset field; constructing a corresponding remapping relationship through the continuous offset field; determining the source coordinates corresponding to each pixel in the sub-image; and determining the spatial position of the sub-image based on the source coordinates.
[0006] In some specific implementations, obtaining the sub-image pairs corresponding to the image to be registered and the reference image based on spatial correspondence includes: performing image segmentation on the image to be registered and the reference image based on unit size to obtain multiple sub-images, and constructing the sub-image pairs by constructing the sub-image pairs with the sub-images in the same spatial relationship.
[0007] In some specific implementations, the method further includes: setting the edge position of each of the sub-images in the sub-image pair along a first direction as an overlapping sub-region according to a preset standard size, and obtaining the offset point set of the two sub-images in each sub-image pair and the corresponding similarity score based on the overlapping sub-region.
[0008] In some specific implementations, obtaining the offset point set of the two sub-images in each sub-image pair includes: obtaining the frequency domain distribution result corresponding to each overlapping region, and determining the displacement estimate between the two sub-images and the offset point set based on the frequency domain result.
[0009] In some specific implementations, obtaining the frequency domain distribution results corresponding to the two sub-images in each sub-image pair includes: performing Fourier transform on the two sub-images respectively and determining the normalized cross power spectrum, and calculating the inverse Fourier transform to obtain the correlation plane.
[0010] In some specific implementations, the method further includes edge weighting of the overlapping region using a Hanning window.
[0011] In some specific implementations, the offset result includes valid offset results and invalid offset results. The offset result in the offset point set is determined based on the relationship between a preset threshold and the similarity score, including: determining whether each offset point is valid based on a preset matching threshold, determining the number of valid offset points and determining the offset results with a number of offset points greater than a preset requirement as the valid offset results, and generating invalid offset points as invalid region masks.
[0012] In some specific implementations, the initial offset field is completed based on morphology to construct a continuous offset field, including: performing neighborhood smoothing and morphological dilation iteration on the effective offset points in the initial offset field in sequence, and reconstructing the continuous offset field through bilinear interpolation.
[0013] In some specific implementations, constructing the corresponding remapping relationship through the continuous offset field includes: determining the pixel coordinates corresponding to any pixel in the sub-image, determining multiple spatially adjacent associated sub-images of the pixel, determining multiple center distances between the pixel and each associated sub-image, determining offset update weights based on the center distances, and constructing the remapping relationship based on the offset update weights and the continuous offset field.
[0014] Secondly, an image reconstruction alignment apparatus is provided, the apparatus comprising: an image processing module, configured to retrieve a reference image corresponding to an image to be registered, obtain sub-image pairs corresponding to the image to be registered and the reference image based on spatial correspondence, and obtain a set of relative offset points and a corresponding similarity score for the two sub-images in each sub-image pair; an offset field construction module, configured to determine the offset result in the offset point set based on a preset threshold and the similarity score, construct an initial offset field based on the effective offset result, and complete the initial offset field based on morphology to construct a continuous offset field; and an image reconstruction module, configured to construct a corresponding remapping relationship through the continuous offset field, determine the source coordinates corresponding to each pixel in the sub-image, and determine the spatial position of the sub-image based on the source coordinates.
[0015] The embodiments of the present invention bring the following beneficial effects: The technical solution provided in this application introduces a joint discrimination mechanism of similarity score and preset threshold, which fundamentally eliminates abnormal offset points with low confidence, avoiding interference from noise and defects in the construction of the offset field. Furthermore, by utilizing a completion strategy combining morphological dilation and iterative diffusion, invalid regions are adaptively filled based on the principle of spatial proximity, overcoming the physical distortion and void residue problems caused by simple interpolation. Further, through a continuous offset field remapping mechanism based on center distance weighting, discrete sub-block offsets are fused into a globally continuous deformation description, fundamentally eliminating stitching seams and sub-pixel alignment errors. The synergistic effect of these technical means enables this application to significantly improve the alignment accuracy and reconstruction quality of large-size industrial images under non-rigid deformation conditions while ensuring computational efficiency, meeting the stringent requirements of detection systems for high precision, high robustness, and visual integrity.
[0016] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.
[0017] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] The methods, systems, and / or procedures shown in the accompanying drawings will be further described with reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein example figures represent similar mechanisms in the various views of the drawings.
[0020] Figure 1 This is a schematic diagram of a method for reconstructing image alignment provided in this application.
[0021] Figure 2 This is a schematic diagram illustrating the correspondence between an image to be registered and a reference image, as provided in this application.
[0022] Figure 3 This is a diagram illustrating the segmentation provided in this application.
[0023] Figure 4 A schematic diagram of the alignment process provided in this application.
[0024] Figure 5 This is a schematic diagram illustrating the spatial relationship between the target pixels and the image provided in this application.
[0025] Figure 6 This is a schematic diagram of a device for reconstructing image alignment.
[0026] Figure 7 This is a schematic diagram of a detection system structure.
[0027] Figure 8 This is a schematic diagram of a computer system architecture. Detailed Implementation
[0028] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.
[0029] In the detailed description below, numerous specific details are illustrated with examples to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that this application can be practiced without these details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high level without detail to avoid unnecessarily obscuring aspects of this application.
[0030] This application uses flowcharts to illustrate the execution process performed by a system according to embodiments of this application. It should be clearly understood that the execution processes in the flowcharts may not be executed sequentially. Instead, these execution processes may be executed in reverse order or simultaneously. Additionally, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.
[0031] This embodiment provides a method for reconstructing image alignment, applied in the field of industrial inspection, specifically in a semiconductor inspection scenario, to achieve image registration of semiconductor products. In existing technologies, mainstream image registration methods mainly include the following technical paths: Feature point matching-based methods extract significant features such as corners and edges from images and construct corresponding relationships to achieve registration; their accuracy depends on the quality of feature extraction and the robustness of the matching strategy. Cross-correlation-based methods calculate similarity metrics between images in the spatial or frequency domain to estimate displacement; the calculation process is intuitive, but it is prone to getting trapped in local extrema under large displacement or complex deformation conditions. Optical flow estimation-based methods describe continuous deformation between images by establishing pixel-level motion fields; they are suitable for small displacement scenarios but are sensitive to noise and illumination changes. Deep learning-based end-to-end registration methods utilize convolutional neural networks to directly learn deformation parameters from image pairs; although they have strong nonlinear modeling capabilities, they require a large amount of labeled data for training and the model's interpretability is insufficient.
[0032] Among the methods described above, phase-correlation-based registration is widely used in industrial scenarios due to its high computational efficiency and insensitivity to linear grayscale changes. However, all of these methods face common limitations when dealing with the local non-rigid deformations prevalent in large-size industrial images. Specifically, these methods mainly rely on global parameter models or assumptions about single deformations, making it difficult to accurately describe spatially dependent non-uniform offset distributions. Furthermore, even when using a local block strategy to obtain sparse displacement samples, there is a lack of quantitative evaluation of matching reliability and a collaborative mechanism to ensure the continuity of the offset field. This results in the inability to effectively remove abnormal matching points and uncontrollable quality of invalid region completion, ultimately affecting the integrity of the reconstructed image and the detection accuracy.
[0033] Furthermore, in large-size image inspection scenarios, the physical size of the object to be inspected is typically large. For example, a 12-inch wafer has a diameter of approximately 300mm, a large PCB board can reach hundreds of millimeters, and a display panel can have a diagonal size of tens of inches. The corresponding industrial inspection image resolution often reaches tens of thousands or even hundreds of thousands of pixels. During the acquisition of such large-size images, due to the wide imaging field of view and long acquisition time, the object to be inspected is easily affected by the superposition of multiple factors, resulting in spatially dependent local non-rigid deformation. For example, thermal drift caused by local heating during wafer inspection exhibits differences in different areas. This deformation shows significant differences in direction and amplitude in different regions of the image, which is a typical non-rigid offset.
[0034] To address local non-rigid deformation, some existing technologies introduce a block registration strategy, dividing the image into several sub-regions and independently performing phase correlation operations within each sub-region to obtain local displacement estimates. However, this strategy faces the following technical problems in practical applications. First, the sub-block division often uses a fixed grid and lacks overlapping settings, leading to phase jumps and matching breaks at the sub-block boundaries due to spectral truncation. In large images, the boundary effect is further amplified due to the large number of sub-blocks. Second, a quantitative reliability assessment mechanism for the matching results is not established, and abnormal displacement points generated in texture-poor, repetitive, or interfering regions cannot be effectively identified and eliminated, directly contaminating the subsequent migration field construction process. Third, the migration field reconstruction stage often relies on simple interpolation to fill invalid regions, failing to guarantee the spatial consistency and directional continuity of the hole-filling results. In large images, due to the wide distribution and large area of invalid regions, the reconstructed image ultimately contains stitching seams, holes, or artifacts, severely restricting the accuracy and stability of the detection system.
[0035] Furthermore, in order to solve the above-mentioned technical problems, see [reference needed]. Figure 1 An exemplary flowchart of a method for reconstructing image alignment according to an embodiment of the present disclosure is shown. Based on this diagram, it can be seen that the method includes the following steps: Step S11. Retrieve the reference image corresponding to the image to be registered, obtain the sub-image pairs corresponding to the image to be registered and the reference image based on the spatial correspondence, and obtain the relative offset point set of the two sub-images in each sub-image pair and the corresponding similarity score.
[0036] In this embodiment, the image to be registered refers to an image containing potential defects or deformations acquired through an industrial camera, while the reference image refers to a pre-established standard template image or a gold sample image. For details regarding the image to be registered and the reference image, please refer to [link to relevant documentation]. Figure 2 As shown, the two images share consistency in overall image features. However, since large-size images often exhibit local non-rigid deformations, direct global registration struggles to capture local details. Therefore, this step decomposes the complex global nonlinear problem into multiple local linear or approximately linear problems by discretizing the entire image into multiple sub-image pairs. Spatial correspondence refers to the clear mapping relationship between the sub-images in the image to be registered and the sub-images in the reference image in physical space. This relationship can be based on row and column index correspondence using a regular grid or on topological correspondence of feature regions; this embodiment does not limit this. It should be understood that the image features on the image to be registered and the reference image are only one implementation method. In other implementations, the circuit layout and image features on the image to be registered and the reference image may differ from those shown in the figures, but this does not mean that different circuit layouts or image features are not within the scope of this embodiment. This embodiment does not exhaustively list the aforementioned layouts and features.
[0037] After acquiring the sub-image pairs, the relative offset point set between them is calculated. This point set represents the displacement vector distribution within the local region. Simultaneously, a corresponding similarity score is obtained. This score is a key indicator for quantifying the reliability of the matching result; its physical meaning is to measure the degree of texture consistency or phase correlation between the two sub-images at the current position. Introducing the similarity score provides data support for subsequent steps, distinguishing between high-confidence true deformation and low-confidence noise interference, thus avoiding blindly trusting all calculated offsets.
[0038] Furthermore, in this embodiment, a high-precision local offset acquisition mechanism is constructed to solve the problems of discontinuous boundary matching and spectral truncation in large-size images under non-rigid deformation.
[0039] As one implementation method, sub-image pairs are obtained by segmenting the image to be registered and the reference image based on the unit size to obtain multiple sub-images, and constructing sub-image pairs with sub-images that have the same spatial relationship. Specifically, in this embodiment, the unit size refers to the reference pixel size used for grid division, which can be a pre-set fixed value or an adaptive value dynamically adjusted according to the image texture complexity. When the image texture is rich, a smaller unit size can be used to improve spatial resolution; when the texture is poor, the unit size is increased to ensure the recognizability of the main peak in the frequency domain. The same spatial relationship means that the (i,j)th sub-image in the image to be registered and the (i,j)th sub-image in the reference image with the same row and column index logically constitute a set of matching units. The segmentation process and the corresponding distribution of the grid division can be found in [reference needed]. Figure 3 ,pass Figure 3 It can be seen that multiple dividing grids are generated on the image to be registered, and the segmentation processing of the image to be registered is achieved along the corresponding segmentation path.
[0040] This regularized grid partitioning strategy decouples the non-rigid registration problem of the entire image into multiple local rigid or near-rigid registration problems, providing a standardized data foundation for subsequent large-scale parallel computing and significantly improving processing efficiency. It should be understood that although this embodiment preferably uses a regular rectangular grid for partitioning, in other embodiments, regular grids or irregular adaptive blocks can be used based on the geometric features of the object being detected, as long as a clear spatial topological correspondence can be established between the image to be registered and the reference image.
[0041] To further improve the continuity of boundary matching, this embodiment also sets the edge position of each sub-image in the sub-image pair along the first direction as an overlapping sub-region according to a preset standard size, and obtains the offset point set of the two sub-images in each sub-image pair and the corresponding similarity score based on the overlapping sub-region.
[0042] Here, the first direction refers to the row direction of the image, and the preset standard size defines the overlap range between adjacent sub-images. In this embodiment, the standard size is 20% of the width of the sub-image. Specifically, the physical significance of introducing overlapping sub-regions in this embodiment is to increase the information redundancy at the boundaries. In non-rigid deformation scenarios, if the sub-images are completely independent and do not overlap, the displacement field at the boundaries is prone to abrupt changes or discontinuities, resulting in obvious stitching seams in the reconstructed image. By setting overlapping areas, the same physical region is covered by multiple adjacent sub-images, and the final offset of each pixel can be obtained by fusing the matching results of multiple sub-images, thereby ensuring the spatial continuity of the offset field from the data acquisition source. In addition, the overlapping areas also provide more sample support for subsequent performance evaluation, which helps to eliminate abnormal matches at the boundaries.
[0043] In the specific implementation of obtaining the offset point set, this embodiment adopts a frequency domain analysis strategy. Specifically, the offset point sets of the two sub-images in each sub-image pair are obtained. By obtaining the frequency domain distribution results corresponding to each overlapping region, the displacement estimate and offset point set between the two sub-images are determined based on the frequency domain results. In this embodiment, the distribution of the offset point set can be found in [reference needed]. Figure 4 As shown in the figure, the arrows represent the displacement vector of each pixel, providing a visual representation of pixel-level displacement. The entire arrowhead represents the continuity of the offset field. Specifically, the direction of the arrow indicates the direction of pixel movement; for example, the arrows pointing upwards at the top and to the right on the right indicate that the middle block needs to move upwards / to the right to achieve alignment. In this embodiment, by controlling the direction, length, and continuity of the arrows, image remapping from local deformation to fine alignment can be achieved, ultimately solving the alignment problem caused by non-rigid displacement.
[0044] Please continue reading. Figure 4 ,against Figure 4 The processing steps described in the image show that the left view is the original view, containing local deformation. The image shows that the local deformation is a misalignment between the central block and its surroundings. For this misalignment, an offset field is obtained through the above processing. Based on this offset field, after displacement processing, the right view is obtained as a remapped view. In the right view, the central block is aligned with its surroundings, and the edges are smooth.
[0045] It is worth noting that this embodiment chooses a frequency domain method instead of a spatial domain cross-correlation method because the frequency domain phase correlation algorithm is insensitive to linear changes in grayscale and has excellent noise resistance and computational efficiency, making it suitable for complex environments in industrial settings with uneven lighting or surface contamination. The frequency domain distribution results characterize the phase difference information of sub-image pairs in the frequency domain, which directly corresponds to the translation amount in the spatial domain and is unaffected by changes in the overall brightness or contrast of the image.
[0046] Furthermore, the frequency domain distribution results corresponding to the two sub-images in each sub-image pair are determined by performing Fourier transforms on the two sub-images respectively, determining the normalized cross-power spectrum, and calculating the inverse Fourier transform to obtain the phase correlation surface. The peak position on this phase correlation surface corresponds to the sub-pixel level displacement vector between the two sub-images. This embodiment, through normalization processing, can effectively eliminate the influence of amplitude changes caused by light source fluctuations or differences in material reflectivity, ensuring that displacement estimation depends only on the relative positional relationship of structural features, thereby significantly improving robustness in the detection environment.
[0047] It is worth noting that directly performing a Fourier transform on the truncated sub-image introduces a spectral truncation effect, leading to spurious high-frequency components in the frequency domain and affecting matching accuracy. Therefore, this embodiment uses a Hanning window for edge weighting of the overlapping region. Before performing a Fourier transform on the sub-image, it is multiplied element-wise by the Hanning window function, which smoothly transitions the grayscale values at the sub-image edges to zero, eliminating signal discontinuities caused by rectangular window truncation and effectively suppressing the Gibbs phenomenon and high-frequency leakage. Without a window or using only a rectangular window, sidelobe energy in the spectrum may mask the true main peak or generate spurious peaks near strong texture edges, leading to mismatches. This embodiment, through Hanning window weighting, significantly improves the signal-to-noise ratio and sharpness of the main peak in the relevant plane, making peak localization more accurate, which is crucial for achieving sub-pixel-level alignment accuracy. It should be understood that although the Hanning window is preferred in this embodiment, other types of smoothing window functions such as the Hamming window and Blackman window can be selected in other implementations based on spectral characteristics, as long as they can suppress edge effects and improve frequency domain matching stability.
[0048] Step S12. Determine the offset results in the offset point set based on the relationship between the preset threshold and the similarity score, construct an initial offset field based on the effective offset results, and complete the initial offset field based on morphology to construct a continuous offset field.
[0049] Specifically, due to the complex industrial environment, the sub-image matching process is inevitably affected by texture repetition, lighting changes, or defects themselves, causing some offset results to deviate from the true value. This step first uses a preset threshold to judge the similarity score, dividing the offset results into valid and invalid offset results. Specifically, the preset threshold in this embodiment is a static judgment boundary used to filter out reliable data points that meet the similarity score requirements. Although the initial offset field constructed based on these valid offset results is numerically accurate, it is often sparse and discontinuous in spatial distribution, with a large number of holes caused by the removal of invalid points. In order to restore the spatial continuity of the offset field, this embodiment further completes the initial offset field based on morphology.
[0050] Morphological completion is a processing mechanism that uses spatial neighborhood information to infer the values of missing regions. It assumes that the deformation of an object's surface is smooth and continuous within a local range; therefore, the offset at the hole can be derived from the offsets of surrounding valid points using specific geometric or algebraic rules. Through this process, the discrete, hole-filled initial offset field is reconstructed into a fully covered continuous offset field. This not only fills in the missing data but also suppresses isolated noise points, providing a complete deformation description model for subsequent image reconstruction.
[0051] Furthermore, the offset results in this embodiment include valid offset results and invalid offset results. Specifically, determining the offset results in the offset point set based on the relationship between a preset threshold and the similarity score includes: determining whether each offset point is valid based on a preset matching threshold, determining the number of valid offset points, and defining offset results with offset point data exceeding a preset number as valid offset results, while generating invalid region masks for invalid offset points.
[0052] Specifically, this embodiment employs a dual verification mechanism to filter reliable offset data. The first verification process determines the absolute value of the similarity score of a single offset point based on a preset matching threshold. The similarity score is jointly composed of the normalized cross-correlation coefficient and the sharpness of the phase correlation peak, where the peak sharpness reflects the sharpness of the frequency domain correlation peak and can effectively distinguish between true matches and false matches in flat regions. When the overall similarity score of an offset point is lower than the matching threshold, the point is initially marked as a suspected invalid point. However, relying solely on a single threshold is difficult to achieve when dealing with periodically repeating textures or highly reflective areas, as these areas may generate extremely high similarity scores but the corresponding displacement vectors are incorrect. Therefore, this embodiment introduces a second verification, namely, determining the number of valid offset points and defining offset results with a number of offset points greater than a preset requirement as valid offset results. Here, the number of valid offset points refers to the statistical count of offset points that pass the first verification within a local neighborhood or the same sub-image pair. The preset requirement is a constraint condition that assumes that true non-rigid deformation has continuity in local space; therefore, valid offset points should be clustered rather than isolated in spatial distribution. If, within a certain region, some points have high similarity scores, but the overall number of valid points is sparse and does not meet the preset requirement, it indicates that the region is highly likely to be affected by periodic texture interference or random noise. The system will then forcibly classify all offset results within that region as invalid. This dual verification mechanism ensures data reliability from both individual quality and group consistency dimensions, exhibiting higher stability compared to threshold-based filtering mechanisms.
[0053] Furthermore, for offset points deemed invalid, this embodiment generates an invalid region mask. This invalid region mask is structurally represented as a binary matrix with the same size as the initial offset field, where elements with a value of 1 indicate that the offset result at the corresponding position is valid, and elements with a value of 0 indicate that the corresponding position is an invalid hole. This mask serves as a necessary input for the subsequent morphological completion step. During the subsequent completion process, based on the spatial position represented by the mask, interpolation or diffusion operations are performed only in the hole regions with a value of 0, while the original calculated values remain unchanged in the valid regions with a value of 1. This embodiment, through the aforementioned mask-based directional completion strategy, avoids secondary contamination of high-confidence valid data by traditional global smoothing processing, ensuring that the final constructed continuous offset field not only fills in missing information but also fully preserves the true local deformation details. It should be understood that although this embodiment preferentially uses a binary matrix as the data carrier for the mask, in other embodiments, soft masks such as probability maps or confidence weight maps can also be used, as long as they can clearly distinguish between valid regions and regions to be completed and provide spatial guidance for subsequent processing. Furthermore, the matching threshold and the preset quantity requirement can be fixed empirical values or dynamic parameters that are adaptively calculated based on the overall signal-to-noise ratio or texture complexity of the image. In this embodiment, there are no restrictions on these.
[0054] As one implementation method, the specific processing approach for morphological completion of the initial offset field involves sequentially performing neighborhood smoothing and morphological dilation stacking on the effective offset points in the initial offset field, and then reconstructing a continuous offset field using bilinear interpolation. This processing flow is not a simple step-by-step stacking, but a multi-level data restoration mechanism designed for the non-rigid deformation characteristics of industrial images. Neighborhood smoothing is primarily used to suppress high-frequency noise and enhance local consistency; iterative morphological dilation is used to quickly fill small-scale holes and establish a long-range connected skeleton while maintaining the geometric structure; and bilinear interpolation is responsible for transforming the discrete or blocky data processed in the first two steps into a pixel-level continuous offset field. These three processes work together to achieve a progressive reconstruction from denoising and skeleton restoration to full-field continuity.
[0055] Specifically, this embodiment introduces a weighted average method with constraints when performing neighborhood smoothing to prevent excessive edge blurring caused by conventional smoothing. For example, a Gaussian kernel can be used to perform a weighted average of the neighborhoods of valid offset points, with the weight coefficients decreasing exponentially with spatial distance. The smoothing process is only performed when specific neighborhood validity conditions are met, such as requiring that the current valid point has at least three other valid neighbors within its 8-neighborhood, and limiting the maximum search radius to a preset range. This constraint mechanism is suitable for regions with rich texture and dense matching points, effectively suppressing random measurement noise through multi-neighbor averaging; however, in edge transition regions with severe deformation or sparse valid points, the smoothing process is skipped because the minimum number of neighbors requirement is not met, thus fully preserving the key gradient features of local non-rigid deformation.
[0056] After the above processing, for the remaining invalid void regions, this embodiment employs morphological dilation iterative processing for directional completion. Specifically, firstly, based on the generated invalid region mask, a morphological dilation operation is performed on the valid offset mask using predefined structuring elements. In each round of dilation, for the newly expanded void location, instead of simply assigning a fixed value, the nearest valid source point in its neighborhood is found, and the offset vector of that source point is assigned to the current location, while the propagation distance from the source point to the current location is recorded. This process can be iterated multiple times to cover most isolated or small-scale void regions.
[0057] Furthermore, to handle large areas of invalid regions and ensure the physical plausibility of the completion results, this embodiment introduces a refined reconstruction based on an iterative diffusion mechanism on top of morphological dilation. In each iteration, all pixels still in a holed state are traversed, and their offset values are updated to the weighted average of their effective neighborhood offset values, with the weights decaying exponentially with spatial distance. In particular, to prevent contamination by long-distance erroneous data, additional attenuation factors are applied to points whose propagation distance exceeds a preset threshold to suppress long-range error propagation. The iteration process has dual termination conditions, automatically stopping when the maximum number of iterations is reached or the overall average offset change is less than a preset residual threshold.
[0058] Although the offset field after the above smoothing and completion processing is topologically complete, it may exhibit gradient effects in its numerical distribution due to grid partitioning. Therefore, this embodiment uses bilinear interpolation for final reconstruction, mapping discrete offset sampling points onto a full-resolution pixel grid to generate a displacement field that is continuously differentiable in both the X and Y directions. It should be understood that while this embodiment preferably uses bilinear interpolation as the reconstruction method, bicubic interpolation or spline interpolation can also be used in scenarios with higher accuracy requirements. Similarly, the specific values for iterative diffusion can be selected using implicit methods or multigrid methods depending on computational resources, as long as adaptive hole filling based on neighborhood information and controllable error propagation can be achieved. This embodiment uses a four-layer processing architecture—constrained smoothing, morphological skeleton, diffusion processing, and continuous reconstruction—to solve the problems of residual holes, edge distortion, and noise amplification caused by simple interpolation in existing technologies, providing a data foundation for high-precision image reconstruction.
[0059] Step S13. Construct the corresponding remapping relationship through the continuous offset field, determine the source coordinates corresponding to each pixel in the sub-image, and determine the spatial position of the sub-image based on the source coordinates.
[0060] In this embodiment, the continuous offset field is a function field that defines the displacement vectors of each point on the image plane. However, to apply it to image reconstruction, it needs to be converted into a pixel-level coordinate mapping relationship. The remapping relationship describes the source coordinate position of any target pixel in the output image in the original image to be registered. By querying the offset vector of this position in the continuous offset field and superimposing it on the target coordinates, the source coordinates can be accurately calculated. Based on these source coordinates, this embodiment obtains the pixel grayscale value through interpolation, thereby determining the corrected spatial position of each pixel in the sub-image. This process realizes the transformation from abstract offset field data to specific graphic geometric transformations, ensuring that the reconstructed image can accurately compensate for non-rigid deformations caused by factors such as thermal drift and mechanical errors. Compared with the traditional block-by-block stitching method, the remapping relationship constructed based on the continuous offset field ensures the continuous differentiability of the full-field transformation function, fundamentally eliminating stitching seams and misalignments at the boundaries of sub-blocks, making the final output image highly consistent both visually and geometrically.
[0061] As one implementation method, the remapping relationship is reconstructed through a continuous offset field, including: determining the pixel coordinates of any pixel in a sub-image, determining multiple spatially adjacent associated sub-images of the pixel, determining multiple center distances between the pixel and each associated sub-image, determining offset update weights based on the center distances, and constructing a remapping relationship based on the offset update weights and the continuous offset field.
[0062] Specifically, constructing the remapping relationship involves transforming the abstract continuous offset field into a concrete image transformation operation. First, for any target pixel in the corrected image to be matched, its pixel coordinates in the image coordinate system are obtained. For details regarding the spatial relationship between the target pixel and the image, please refer to [link to relevant documentation]. Figure 5 As shown, multiple spatially adjacent associated sub-images are identified for a given pixel. An associated sub-image refers to all sub-image blocks whose coverage area spatially includes the pixel under the grid partitioning and overlap strategy. Due to the overlapping window strategy, pixels within the overlapping region are typically covered by two or four adjacent sub-images, while pixels within the non-overlapping region belong to only a single sub-image.
[0063] Further, the center distances between the pixel and each associated sub-image are calculated. The center distance refers to the Euclidean distance from the pixel to the geometric center of each associated sub-image. Then, the offset update weights are determined based on the center distances. In this embodiment, the offset update weights are negatively correlated with the center distances, preferably using the inverse distance weighting method or the Gaussian distance weighting method. When the pixel is exactly located at the center of a sub-image, the weight of that sub-image approaches 1; when the pixel is located at the exact midpoint between the boundaries of two sub-images, the weights of both are approximately 0.5. This embodiment uses a geometric distance weighting mechanism, ensuring that the final offset of the output pixel is smoothly derived from the local offsets of multiple surrounding sub-images, rather than simply taking a rigid estimate of a single sub-block. Finally, a remapping relationship is established based on the offset update weights and the continuous offset field. Specifically, the final offset vector corresponding to the target pixel is the weighted sum of the offsets of each associated sub-image at that position. The source coordinates are then determined, and the source image pixel values are obtained through bilinear interpolation to complete the resampling. It should be understood that although this embodiment preferentially uses the inverse distance as the basis for weight calculation, in other embodiments, additional correction factors can be introduced based on the matching confidence or texture complexity of the sub-image, or a bicubic spline kernel function can be used to replace the simple inverse distance, as long as the principle of spatial correlation that the closer the distance, the greater the contribution can be reflected.
[0064] This embodiment introduces distance weighting within the overlapping region, making the offset at the boundary a convex combination of the offsets of adjacent sub-blocks, thereby forcing the first-order continuity of the offset field. Furthermore, since the weights are based on sub-pixel level precise distance calculations, even if the offset estimates of the sub-blocks themselves are at the integer pixel level, the final offset after weighted fusion naturally exhibits sub-pixel resolution, significantly improving the accuracy of non-rigid deformation compensation. In terms of engineering implementation, since the remapping calculations for each output pixel are independent and involve only a small number of neighborhood searches and arithmetic operations, this algorithm is naturally suitable for deployment on parallel computing architectures such as GPUs. For example, CUDA technology can be used to distribute the weight calculation and interpolation mapping of each pixel to independent thread cores for execution, thereby meeting the stringent real-time requirements of industrial inspection while ensuring high accuracy.
[0065] In summary, this application's embodiments, by introducing a joint discrimination mechanism of similarity score and preset threshold, fundamentally eliminate low-confidence abnormal offset points, avoiding interference from noise and defects in the offset field construction. Furthermore, by employing a completion strategy combining morphological dilation and iterative diffusion, invalid regions are adaptively filled based on the principle of spatial proximity, overcoming the physical distortion and void residue problems caused by simple interpolation. Moreover, through a continuous offset field remapping mechanism based on center distance weighting, discrete sub-block offsets are fused into a globally continuous deformation description, fundamentally eliminating stitching seams and sub-pixel alignment errors. The synergistic effect of these techniques enables this application to significantly improve the alignment accuracy and reconstruction quality of large-size industrial images under non-rigid deformation conditions while ensuring computational efficiency, meeting the stringent requirements of detection systems for high precision, high robustness, and visual integrity.
[0066] For further details, please refer to [link / reference]. Figure 6 This embodiment provides an image reconstruction alignment device 60, which solidifies the aforementioned method flow into a reusable product form through modular functional unit settings. The device includes: an image processing module 61, an offset field construction module 62, and an image reconstruction module 63.
[0067] Image processing module 61 is used to retrieve the reference image corresponding to the image to be registered, obtain the sub-image pairs corresponding to the image to be registered and the reference image based on the spatial correspondence, and obtain the relative offset point set of the two sub-images in each sub-image pair and the corresponding similarity score.
[0068] Specifically, the image processing unit, serving as the data front-end and local matching engine of the entire device, can be further divided into an image segmentation submodule and a frequency domain matching submodule. The image segmentation submodule is configured to perform gridded partitioning and overlapping window generation operations, transforming massive pixel data into a standardized sequence of sub-image pairs, laying the foundation for subsequent parallel computing. The frequency domain matching submodule is configured to perform Fourier transform, normalized cross-power spectrum calculation, and inverse transform operations to extract high-precision local offset information. In actual hardware deployment, the image processing unit is preferably implemented using a graphics processing unit (GPU) or field-programmable gate array (FPGA) that supports large-scale parallel computing. Its multi-core concurrency characteristics allow for the simultaneous processing of matching tasks for multiple sub-image pairs, thereby meeting the stringent real-time requirements of industrial AOI inspection. It should be understood that although this embodiment describes it as an independent unit, in some integrated designs, the image processing unit can also be represented as a multi-threaded software module running on a general-purpose processor, as long as it can complete the aforementioned data acquisition and preprocessing functions.
[0069] The offset field construction module 62 is used to determine the offset result in the offset point set based on the relationship between the preset threshold and the similarity score, construct an initial offset field based on the effective offset result, and complete the initial offset field based on morphology to construct a continuous offset field.
[0070] Specifically, the offset field construction module is the core processing hub ensuring registration robustness and geometric continuity. Internally, this unit logically includes a confidence discriminator, a morphological operator, and a diffusion solver. The confidence discriminator is configured to perform a dual verification mechanism, combining a similarity score threshold and a valid point count threshold to filter reliable data and generate an invalid region mask. The morphological operator is configured to perform iterative neighborhood smoothing and dilation operations to quickly fill small-scale holes and establish a data skeleton. The diffusion solver is configured to execute an iterative diffusion algorithm based on the heat conduction equation, adaptively completing large-area invalid regions while ensuring physical plausibility. From a data flow perspective, this unit receives sparse discrete data from the image processing unit and outputs a full-coverage continuous vector field, playing a crucial role in bridging the gap between the preceding and following steps. In engineering implementation, the offset field construction module can adopt a heterogeneous collaboration between CPU and GPU. Logical judgment and mask generation are performed by the CPU, while smoothing and diffusion calculations involving numerous neighborhood operations are offloaded to the GPU for acceleration, balancing control flexibility and computational throughput.
[0071] The image reconstruction module 63 is used to construct a corresponding remapping relationship through the continuous offset field, determine the source coordinates corresponding to each pixel in the sub-image, and determine the spatial position of the sub-image based on the source coordinates.
[0072] Specifically, the image reconstruction module is the final output engine of the device, responsible for transforming abstract offset field data into a visualized corrected image. This module internally includes a weighted fusion calculator and a resampling interpolator. The weighted fusion calculator dynamically calculates offset update weights based on the distance between pixels and the centers of associated sub-images, eliminating stitching seams at sub-block boundaries. The resampling interpolator, based on the fused precise source coordinates, extracts pixel values from the original image using bilinear or bicubic interpolation algorithms to generate the final seamless reconstructed image. To further improve performance, the image reconstruction module can also be configured with a lookup table (LUT) generation submodule, which pre-calculates and caches the remapping relationship table for the entire image, allowing subsequent image reconstructions with the same parameters to be performed directly by looking up the table, avoiding redundant calculations. It should be understood that although this embodiment describes a specific module division method, in other embodiments, the above units or modules can also be merged into an integrated processing unit or split into more fine-grained sub-units, as long as their overall functionality covers the technical solution described in this application.
[0073] Through the aforementioned device architecture, this embodiment achieves fully automated processing from data acquisition and offset field reconstruction to image reconstruction. Data interaction between functional units is achieved through clearly defined interfaces, ensuring not only the strict execution of the algorithm logic but also facilitating system upgrades and maintenance. For example, when adapting to detection objects with different texture characteristics, only the block parameters in the image processing unit or the threshold strategy in the offset field construction module need to be adjusted, without reconstructing the entire system. Furthermore, this device supports decoupled hardware and software deployment, serving as a core component of an independent embedded detection device or as part of a cloud-based image processing service, exhibiting broad applicability and excellent scalability.
[0074] This embodiment provides a specific implementation method for applying the aforementioned reconstructed image alignment method to wafer surface inspection in advanced packaging processes. Advanced packaging technologies, such as fan-out wafer-level packaging, 2.5D / 3D heterogeneous integration, and through-silicon via (TSV) interconnects, are key pathways for the semiconductor industry to improve chip integration density and system performance. Their process characteristics place more stringent requirements on the alignment accuracy of surface inspection.
[0075] Specifically, in the advanced packaging manufacturing process, wafers need to undergo multiple processes, including thinning, bump fabrication, redistribution layer photolithography, chip mounting, molding packaging, and temporary bonding / debonding. These processes give wafers a unique physical state that differs from traditional front-end manufacturing. On the one hand, the thickness of wafers after back-side thinning is usually only 100 micrometers to 50 micrometers or even thinner, and their mechanical stiffness is greatly reduced. During transport, adsorption and processing, they are very prone to non-uniform warping deformation due to clamping stress and their own weight. Moreover, this deformation shows a significant gradient difference between the center and edge regions of the wafer. On the other hand, there is a difference in the coefficient of thermal expansion between the organic substrate, molding compound and silicon material widely used in advanced packaging. After high-temperature processes such as reflow soldering or thermosetting bonding, complex non-rigid deformation caused by thermal mismatch of multiple materials will remain on the wafer surface. This deformation not only includes translational components, but also local rotation and scaling components. Its spatial distribution is closely related to the layout of the packaging structure.
[0076] Against this backdrop, when performing defect detection on the surface of advanced packaged wafers, the geometric deviation between the image to be registered and the reference image exhibits a highly non-uniform spatial distribution. For example, in fan-out wafer-level packaging scenarios, the deformation patterns of the molding area and the silicon chip area are drastically different: the local depressions caused by the curing shrinkage of the molding compound cause the offset vector in this area to converge centripetally, while the silicon chip area, due to its higher stiffness, only exhibits slight translation. Similarly, in redistribution layer inspection, since the linewidth / spacing of the redistribution layer has been reduced to 2 micrometers / 2 micrometers or even smaller, the registration accuracy requirement reaches the subpixel level. Any uncompensated local deformation will result in systematic residuals at the wiring edges after Golden subtraction, which may be misjudged as short circuits or open circuits.
[0077] In this embodiment, a high-resolution area scan camera or line scan camera acquires the image to be registered on the advanced packaging wafer, and a pre-stored standard reference image corresponding to the process stage is retrieved. High-precision alignment and reconstruction are performed according to the methods described in steps S11-S13. As a preferred implementation parameter configuration, considering that the surface of the advanced packaging wafer simultaneously contains a large area of molding and a dense wiring area, the sub-image size is set to 128×128 pixels, and a 20% overlap sub-region is set in both the horizontal and vertical directions to enhance the matching continuity at the location of maximum deformation gradient at the junction of the molding area and the chip area. Hanning windows are applied to the overlapping areas for edge weighting to suppress spectral sidelobe interference caused by the low texture contrast of the molding area. In the confidence discrimination stage, the matching confidence threshold is set to 0.65 to accommodate the special case where the molding area has weak texture but still has effective matching information; at the same time, the preset requirement for the number of effective points is dynamically adjusted according to the type of the sub-image area, with the threshold appropriately lowered for the molding area and a higher standard maintained for the dense wiring area. For the selected valid offset results, Gaussian kernel neighborhood smoothing, three iterations of morphological dilation, and diffusion completion based on the heat conduction equation are performed sequentially. The maximum number of iterations in the diffusion process is set to 12 to accommodate the large void range caused by the large molding area in advanced packaging wafers. Finally, a continuous offset field covering the entire image is constructed, and a remapping relationship is generated through a fusion strategy based on the inverse of the center distance. Subpixel-level image resampling is then performed using bilinear interpolation.
[0078] Because the non-rigid deformation amplitude of advanced packaging wafers is typically greater than that of traditional front-end wafers, and the deformation gradient is particularly severe at the chip-molding interface, the Golden subtraction residual of traditional global affine registration methods in such scenarios is often several times greater than that in front-end scenarios. By adopting the solution described in this application, the continuous offset field can accurately track the gradual transition from the chip region to the molding region, making the Golden subtraction residual distribution more uniform and effectively eliminating the ring artifacts generated at the chip edges due to insufficient deformation compensation. Simultaneously, based on the mechanism of morphological dilation and iterative diffusion collaborative completion, it can effectively handle the large-area matching failure problem caused by extremely poor texture in the molding region, ensuring that the offset field in this region will not have holes or abrupt changes, and avoiding misjudging normal texture fluctuations in the molding region as surface defects.
[0079] Furthermore, since the inspection targets of advanced packaging production lines cover various packaging forms and process stages, key parameters such as sub-image size, overlap ratio, and confidence threshold in this application's solution can be flexibly configured according to the specific packaging structure without redesigning the algorithm framework, exhibiting good process adaptability and scalability. It should be understood that although this embodiment uses fan-out wafer-level packaging and redistribution layer inspection as examples, the technical solution of this application is also applicable to other surface inspection stages in advanced packaging processes such as through-silicon via interconnect inspection, bump array inspection, chip mounting offset inspection, and molding surface defect inspection; this application does not impose any limitations on these aspects.
[0080] See Figure 7 This embodiment provides a detection system 70. The detection system integrates the image reconstructing alignment method corresponding to steps S11-S13, and integrates the optical imaging hardware and image processing algorithm in the physical architecture and data processing link to achieve high-precision automated detection of the surface of advanced packaged wafers.
[0081] Specifically, the detection system includes a hardware component 71 and a processing unit 72. The hardware component further includes an optical component 710 and a carrier component. The carrier component includes a carrier stage 701 for supporting and fixing the packaged wafer to be inspected. In actual deployment, the carrier stage typically uses vacuum adsorption to fix the wafer to its surface to ensure that the wafer remains relatively stationary during imaging. The carrier stage is also equipped with a precision motion platform capable of micrometer-level stepping movement along the X and Y axes in the horizontal plane, and focusing movement along the Z axis, thereby achieving area-by-area scanning imaging of the wafer surface. It should be understood that the fixing method of the carrier stage is not limited to vacuum adsorption; in other embodiments, electrostatic adsorption or mechanical clamping can also be used, as long as the wafer's position can be kept stable during imaging.
[0082] The optical section, located above the carrier section, includes an optical lens assembly 711 and a detector 712. The optical lens assembly images the microstructural features of the wafer surface onto the photosensitive surface of the detector. Its optical magnification and numerical aperture are configured according to the detection resolution requirements. For example, in advanced packaging redistribution layer inspection scenarios, the magnification of the optical lens assembly can be configured to 5x to 20x, and the numerical aperture not less than 0.3, to meet the clear imaging requirements of 2-micron linewidth features. The detector is an image sensor used to convert the optical image formed by the optical lens assembly into a digital image signal. The detector can be a CCD image sensor or a CMOS image sensor, and its pixel size and array size are selected according to the field of view and resolution requirements. In area array imaging mode, the detector can acquire a complete image of a rectangular area on the wafer surface in a single exposure; in line array scanning mode, the detector, in conjunction with the continuous movement of the carrier stage, acquires images line by line and stitches them into a complete wafer surface image. The optical section transmits the acquired surface image of the packaged wafer to be inspected to the processing unit.
[0083] The processing unit 74 is communicatively connected to the optical part and is used to receive the surface image transmitted by the optical part and execute the reconstructed image alignment method described in steps S11-S13 above to achieve high-precision non-rigid alignment between the image to be registered and the reference image. Specifically, after receiving the surface image, the processing unit first uses it as the image to be registered and retrieves the standard reference image corresponding to the process stage from the storage medium. Subsequently, the processing unit performs sub-image segmentation, frequency domain matching, confidence screening, morphological completion, and remapping reconstruction according to the aforementioned method, and outputs a corrected image after deformation compensation. In terms of hardware implementation, the processing unit may include a heterogeneous computing architecture of a central processing unit and a graphics processing unit, wherein the CPU is responsible for process scheduling and logical judgment, and the GPU is responsible for large-scale parallel computing tasks such as sub-image matching, offset field construction, and image resampling to meet the throughput requirements of production line inspection. It should be understood that the implementation form of the processing unit is not limited to an independent industrial control computer or server. In other embodiments, it can also be integrated into the main control board of the inspection equipment or deployed on a cloud computing platform to interact with the inspection equipment through a network. This application does not limit this.
[0084] After image alignment is completed, the processing unit further performs detection processing based on the aligned image. Specifically, the processing unit performs pixel-by-pixel difference operations between the deformation-compensated corrected image and the reference image to generate a residual image. Since the aforementioned alignment method has accurately compensated for the non-rigid deformation of the wafer surface, the background region in the residual image exhibits a uniform low-amplitude noise distribution, while the real defects stand out in the residual image with significantly higher amplitudes than the background. The processing unit applies adaptive threshold segmentation or a machine learning classification model to the residual image, extracts candidate defect regions, and outputs the location, size, and type information of the defects. Because the improved alignment accuracy directly reduces systematic artifacts in the residual image, the false alarm rate of the detection algorithm is significantly reduced, while the detection rate of small defects is improved, thereby improving the overall reliability and sensitivity of the detection system.
[0085] This embodiment integrates optical imaging, precision motion control, image alignment algorithms, and defect detection logic into a unified detection system architecture, forming a complete processing loop from image acquisition to defect determination. The high-quality raw images acquired by the optical component provide a sufficient information foundation for subsequent alignment algorithms. The high-precision alignment method in the processing unit eliminates the interference of non-rigid deformation on the detection results, while the detection processing based on the aligned image fully utilizes the signal-to-noise ratio gain brought by deformation compensation. The synergistic work of these three components enables the detection system to achieve a balance between accuracy and efficiency in advanced packaging wafer inspection scenarios.
[0086] It should be understood that although this embodiment describes the specific configuration of the inspection system using advanced packaged wafer inspection as an example, the architecture of this inspection system is also applicable to other industrial inspection scenarios such as PCB board inspection, display panel inspection, and photovoltaic cell inspection. It is only necessary to adjust the magnification of the optical part, the travel range of the stage, and the algorithm parameters in the processing unit according to the characteristics of the object being inspected. This application does not impose any limitations on this.
[0087] See Figure 8 This invention also provides a computer system 800, which includes one or more client devices 810, a server 820, and one or more communication networks 830 coupling the one or more client devices to the server. The client devices can be configured to execute one or more applications.
[0088] In this embodiment, the server is used to run one or more services or software applications that enable the execution of agent-based methods for model training according to embodiments of this disclosure.
[0089] Specifically, a server may include one or more components that implement the functions performed by the server. These components include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices can sequentially interact with the server using one or more client applications to utilize the services provided by these components. The server may include one or more general-purpose computers, dedicated server computers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Furthermore, the server may include one or more virtual machines running a virtual operating system, or other computing architectures designed for virtualization.
[0090] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and / or methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units / modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0091] The units / modules described as separate components may or may not be physically separate. The components shown as units / modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units / modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0092] Furthermore, in the various embodiments of the present invention, the functional units / modules can be integrated into one processing unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated into one unit / module. The integrated unit / module described above can be implemented in hardware or in the form of hardware plus software functional units / modules.
[0093] The integrated unit / module implemented as a software functional unit / module described above can be stored in a computer-readable storage medium. The software functional unit, stored in a storage medium, includes several instructions to cause one or more processors of a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention.
[0094] The integrated unit / module implemented as a software functional unit / module described above can be stored in a computer-readable storage medium. The software functional unit, stored in a storage medium, includes several instructions to cause one or more processors of a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention.
[0095] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the invention, and not to limit it. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the invention.
Claims
1. A method for reconstructing image alignment, characterized in that, The method includes: Retrieve the reference image corresponding to the image to be registered, obtain the sub-image pair corresponding to the image to be registered and the reference image based on the spatial correspondence, and obtain the relative offset point set of the two sub-images in each sub-image pair and the corresponding similarity score; The offset results in the offset point set are determined based on the relationship between the preset threshold and the similarity score, and an initial offset field is constructed based on the effective offset results. The initial offset field is then completed based on morphology to construct a continuous offset field. By constructing the corresponding remapping relationship through the continuous offset field, the source coordinates corresponding to each pixel in the sub-image are determined, and the spatial position of the sub-image is determined based on the source coordinates.
2. The image reconstructing alignment method according to claim 1, characterized in that, Obtaining the sub-image pair corresponding to the image to be registered and the reference image based on spatial correspondence includes: performing image segmentation on the image to be registered and the reference image based on unit size to obtain multiple sub-images, and constructing the sub-image pair by the sub-images that are in the same spatial relationship.
3. The image reconstructing alignment method according to claim 2, characterized in that, The method further includes: setting the edge position of each of the sub-images in the sub-image pair along a first direction as an overlapping sub-region according to a preset standard size, and obtaining the offset point set of the two sub-images in each sub-image pair and the corresponding similarity score based on the overlapping sub-region.
4. The image reconstructing alignment method according to claim 3, characterized in that, Obtaining the offset point set of the two sub-images in each sub-image pair includes: obtaining the frequency domain distribution result corresponding to each overlapping sub-region, and determining the displacement estimate between the two sub-images and the offset point set based on the frequency domain distribution result.
5. The image reconstructing alignment method according to claim 4, characterized in that, Obtaining the frequency domain distribution results corresponding to the two sub-images in each sub-image pair includes: performing Fourier transform on the two sub-images respectively and determining the normalized cross power spectrum, and calculating the inverse Fourier transform to obtain the correlation plane.
6. The image reconstructing alignment method according to claim 3, characterized in that, The method also includes edge weighting of the overlapping sub-regions using a Hanning window.
7. The image reconstructing alignment method according to claim 1, characterized in that, The offset results include valid offset results and invalid offset results. The offset results in the offset point set are determined based on the relationship between a preset threshold and the similarity score, including: determining whether each offset point is valid based on a preset matching threshold, determining the number of valid offset points and determining the offset results with a number of offset points greater than a preset requirement as the valid offset results, and generating invalid offset points as invalid region masks.
8. The image reconstructing alignment method according to claim 1, characterized in that, The initial offset field is morphologically completed to construct a continuous offset field, including: performing neighborhood smoothing and morphological dilation iteration on the effective offset points in the initial offset field in sequence, and reconstructing the continuous offset field by bilinear interpolation.
9. The image reconstruction alignment method according to claim 1, characterized in that, Constructing the corresponding remapping relationship through the continuous offset field includes: determining the pixel coordinates corresponding to any pixel in the sub-image, determining multiple spatially adjacent associated sub-images of the pixel, determining the center distance between the pixel and each associated sub-image, determining the offset update weight based on the center distance, and constructing the remapping relationship based on the offset update weight and the continuous offset field.
10. An image reconstruction alignment device, characterized in that, The device includes: The image processing module is used to retrieve the reference image corresponding to the image to be registered, obtain the sub-image pairs corresponding to the image to be registered and the reference image based on the spatial correspondence, and obtain the relative offset point set of the two sub-images in each sub-image pair and the corresponding similarity score. The offset field construction module is used to determine the offset result in the offset point set based on the relationship between the preset threshold and the similarity score, and construct an initial offset field based on the effective offset result, and complete the initial offset field based on morphology to construct a continuous offset field; the image reconstruction module is used to construct the corresponding remapping relationship through the continuous offset field, determine the source coordinates corresponding to each pixel in the sub-image, and determine the spatial position of the sub-image based on the source coordinates.
