An automatic optical inspection image registration method and system based on block matching

Abstract

This disclosure relates to an automatic optical detection image registration method and system based on block matching. The method divides the sample image and template image into multiple current sub-block images, and determines the current selection weight of each current sub-block image based on registration quality indicators and historical selection weights to identify multiple target sub-block images. It leverages the consistent graph structure and local image repetition characteristics to reduce the computational cost of matching invalid sub-blocks, improve the efficiency of acquiring effective matching points and the stability of global registration, while also considering registration speed, edge coverage, and robustness. Furthermore, it performs coordinate transformation processing on the matching point pairs of each target sub-block image based on the offset, obtaining a global matching point pair set. This ensures that the sample images are aligned under the same transformation model, thereby reducing boundary artifacts and improving the continuity and interpretability of the overlay results. It also avoids local unsolvability or numerical instability caused by insufficient points within a single block, thus guaranteeing the stability and accuracy of the registered image.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of image processing technology for automatic optical inspection, and in particular relate to an automatic optical inspection image registration method and system based on block matching. Background Technology

[0002] Automated Optical Inspection (AOI) equipment is widely used for defect detection in board-level manufacturing processes such as PCB, FPC, and semiconductor packaging. When performing defect detection, AOI equipment typically acquires sample images of the product under test using high-resolution linear or area array cameras. These sample images are then precisely aligned and registered with a standard template image (which can be a standard image obtained by rasterizing Gerber data, or a template slice image formed by segmenting the standard image). This allows for overlay comparison, differential localization, and defect determination within a unified coordinate system.

[0003] Image registration is a core preliminary step in automated optical inspection, and its accuracy and stability directly determine the reliability of defect detection. To achieve registration between sample and template images, existing technologies mostly employ geometric alignment methods based on alignment markers, template matching methods based on pixel similarity, and registration methods based on feature point extraction and matching. However, these methods suffer from problems such as insufficient registration stability, the tendency for local unsolvability during geometric transformations, inconsistencies in transformations between regions leading to discontinuous registration results, and an inability to simultaneously meet the requirements of computational efficiency and processing latency in online inspection scenarios, thus affecting the accuracy and stability of image registration. Summary of the Invention

[0004] The embodiments of this disclosure present an automatic optical detection image registration method and system based on block matching.

[0005] In a first aspect of this disclosure, an automatic optical detection image registration method based on block matching is provided. The method includes dividing the image set to be registered into grids to obtain multiple current sub-block images, and determining the offset of each current sub-block image in a global coordinate system; the image set to be registered includes sample images and template images. The method further includes determining current selection weights based on registration quality indices of historical sub-block image sets and historical selection weights corresponding to each current sub-block image, and determining multiple target sub-block images from all current sub-block images based on all current selection weights. The method also includes performing feature matching on each target sub-block image to obtain multiple matching point pairs in a local coordinate system, and performing coordinate transformation processing on all matching point pairs based on all offsets to obtain a global matching point pair set. The method further includes determining multiple global homography matrices based on the global matching point pair set, and determining a target homography matrix from all global homography matrices based on the set of interior points corresponding to each global homography matrix. Furthermore, the method includes performing perspective transformation processing on the sample images based on the target homography matrix to obtain the registered image.

[0006] In a second aspect of this disclosure, an automatic optical detection image registration system based on block matching is provided. The system includes an offset determination module configured to perform grid division processing on a set of images to be registered, obtaining multiple current sub-block images, and determining the offset of each current sub-block image in a global coordinate system; the set of images to be registered includes sample images and template images. The system also includes a target sub-block selection module configured to determine current selection weights based on registration quality indicators of historical sub-block image sets and historical selection weights corresponding to each current sub-block image, and to determine multiple target sub-block images from all current sub-block images based on all current selection weights. The system further includes a matching point pair determination module configured to perform feature matching on each target sub-block image, obtaining multiple matching point pairs in a local coordinate system, and to perform coordinate transformation processing on all matching point pairs based on all offsets to obtain a global matching point pair set. The system also includes a homography matrix determination module, configured to determine multiple global homography matrices based on a global matching point pair set, and to determine a target homography matrix from all global homography matrices based on the set of interior points corresponding to each global homography matrix. Furthermore, the system includes a registration image generation module, configured to perform perspective transformation processing on the sample image based on the target homography matrix to obtain a registration image.

[0007] In a third aspect of this disclosure, a computer program product is provided, comprising a computer program that is executed by a processor to implement the method according to the first aspect.

[0008] In a fourth aspect of this disclosure, a machine-readable storage medium is provided. The machine-readable storage medium stores machine-executable instructions, which are executed by a processor to implement the method provided according to a first aspect of this disclosure.

[0009] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 A flowchart illustrating an automatic optical detection image registration method based on block matching, according to some embodiments of this disclosure, is shown.

[0012] Figure 2 A comparative illustration of the effect of a set of images to be registered according to some embodiments of the present disclosure is shown;

[0013] Figure 3 This illustration shows a grid division effect of a set of images to be registered according to some embodiments of the present disclosure;

[0014] Figure 4 A schematic diagram illustrating the selection effect of a target sub-block image according to some embodiments of the present disclosure is shown;

[0015] Figure 5 A schematic diagram illustrating the matching effect of a target sub-block image according to some embodiments of the present disclosure is shown;

[0016] Figure 6 This diagram illustrates the output effect of a registered image according to some embodiments of the present disclosure;

[0017] Figure 7 A block diagram of an automatic optical detection image registration system based on block matching, according to some embodiments of the present disclosure, is shown.

[0018] Figure 8 A block diagram of an electronic device that can implement several embodiments of the present disclosure is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0020] The terms “comprising” and “having”, and any variations thereof, in this specification, claims, and the foregoing drawings are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. Depending on the context, the word “if” as it applies herein may be interpreted as “when”, “when”, “in response to determination”, or “in response to detection”.

[0021] As mentioned above, to achieve registration between sample images and template images, existing technologies mostly employ geometric alignment methods based on alignment markers, template matching methods based on pixel similarity, and registration methods based on feature point extraction and matching. Among these, the geometric alignment method based on alignment markers typically acquires an image containing alignment markers, uses image processing algorithms (such as template matching, edge detection, or centroid methods) to locate the center coordinates of each alignment marker in the image, and calculates the translation and rotation amounts based on the center coordinates of all alignment markers in the image and their theoretical coordinates in the template image. This method is highly dependent on the visibility of the markers; if the markers are contaminated, reflective, occluded, worn, or the marker area in the acquired image is incomplete, it can easily lead to recognition failure. In addition, it has requirements on the number and spatial distribution of markers, making it difficult to reflect local imaging differences in the entire image. Furthermore, it has poor adaptability to products without markers or with non-standard markers; if some products lack standard labels or the marker positions are inconsistent between different batches or models, it will increase engineering adaptation costs.

[0022] Pixel similarity-based template matching methods typically solve for the mapping relationship by initializing geometric transformation parameters and minimizing pixel differences or similarity cost functions (such as Normalized Cross-Correlation (NCC) and its multi-scale variants, pyramid search, or sliding window strategies) after the transformation. This method can achieve high alignment accuracy in some scenarios, but it often requires iterative search within a large parameter space, resulting in high computational cost and slow processing speed. At the same time, it is sensitive to imaging differences such as scale changes, brightness changes, rotation, and local occlusion, and is prone to unstable matching responses or getting trapped in local optima, especially under ultra-large format image conditions, where the search range and computational cost increase significantly. In addition, the presence of duplicate textures, local missing parts, noise interference, or occlusion may also lead to mismatches, positioning drift, or unstable matching windows.

[0023] Registration methods based on feature point extraction and matching typically extract and match feature points in the sample and template images, then combine robust estimation methods such as RANSAC to solve the homography matrix, thereby completing image alignment. This method has certain advantages in terms of efficiency and adaptability to scale and rotation changes, but its registration effect is highly dependent on the number, quality, and spatial distribution of matching point pairs. For example, when the number of matching points is insufficient, the distribution is too concentrated, or the mismatch ratio is high, the estimation accuracy of the homography matrix will decrease significantly, or even fail to solve. In addition, in automatic optical inspection applications, images usually have ultra-high resolution, sparse texture in local areas, or obvious repetitive line structures, and may have limited effective overlapping areas due to reflection, contamination, or incomplete camera coverage. All of these factors can make it difficult to generate stable and reliable corresponding points in some areas, thereby reducing the overall registration success rate and stability.

[0024] Building upon the aforementioned existing technologies, some existing techniques have attempted to divide images into multiple sub-regions for block matching or local modeling to improve the matching success rate of local regions. However, existing block matching methods typically still solve geometric transformation parameters independently within a single sub-region, or only use the block matching results for local alignment and auxiliary positioning. When there are insufficient matching points in the sub-blocks or significant differences in imaging between regions, problems such as local unsolvability, inconsistent transformations between regions, and discontinuous registration results can easily arise, making it difficult to meet the comprehensive requirements of stability, consistency, and processing efficiency for online automatic optical inspection.

[0025] In summary, existing techniques suffer from problems such as insufficient registration stability, local unsolvability during geometric transformations, inconsistencies between region transformations and resulting discontinuities in registration results, and an inability to meet the requirements of computational efficiency and processing latency in online detection scenarios, thus affecting the accuracy and stability of image registration.

[0026] To address this, embodiments of this disclosure propose an automatic optical detection image registration method based on block matching. The method includes dividing the image set to be registered into a grid to obtain multiple current sub-block images, and determining the offset of each current sub-block image in the global coordinate system; the image set to be registered includes sample images and template images. The method further includes determining current selection weights based on registration quality indices of historical sub-block image sets and historical selection weights corresponding to each current sub-block image, and determining multiple target sub-block images from all current sub-block images based on all current selection weights. The method also includes performing feature matching on each target sub-block image to obtain multiple matching point pairs in the local coordinate system, and performing coordinate transformation processing on all matching point pairs based on all offsets to obtain a global matching point pair set. The method further includes determining multiple global homography matrices based on the global matching point pair set, and determining a target homography matrix from all global homography matrices based on the set of interior points corresponding to each global homography matrix. Furthermore, the method includes performing perspective transformation processing on the sample images based on the target homography matrix to obtain the registered image.

[0027] In this way, the set of images to be registered, containing sample images and template images, is divided into multiple current sub-block images. The current selection weight of each current sub-block image is determined based on the registration quality index and historical selection weights to identify multiple target sub-block images. This approach leverages the consistent image structure and local image repetition characteristics of images with the same part number, thereby reducing the computational load for matching invalid or low-quality sub-blocks, improving the efficiency of obtaining effective matching points, and enhancing global registration stability (compared to re-randomizing or fixing blocks for each registration, this approach gradually converges to a sub-block selection distribution more suitable for the current part number during mass production testing). It also balances registration speed, edge coverage, and robustness. Furthermore, coordinate transformation is performed on multiple matching point pairs of each target sub-block image based on the offset to obtain a global matching point pair set. This ensures that the sample images are aligned under the same transformation model, reducing the sources of cross-regional inconsistencies at the transformation model level. This reduces boundary artifacts and improves the continuity and interpretability of the overlay results, while avoiding local unsolvability or numerical instability caused by insufficient points within a single block, thus guaranteeing the stability and accuracy of the registered images.

[0028] Please see Figure 1 The flowchart shown illustrates an automatic optical inspection image registration method based on block matching, according to some embodiments of the present disclosure. Method 100 is executed, for example, by a processing terminal that can be deployed on a cloud management platform. The processing terminal establishes a communication connection with an automatic optical inspection device to acquire, based on the camera of the automatic optical inspection device, an image (i.e., a sample image) of the surface of the product under test (such as multiple printed circuit boards with the same part number).

[0029] like Figure 1As shown in box 102, method 100 can perform grid division processing on the image set to be registered to obtain multiple current sub-block images, and determine the offset of each current sub-block image in the global coordinate system. Here, the image set to be registered can be understood as an image set composed of sample images acquired by the automated optical inspection equipment and template images corresponding to the sample images. The template image can be a corresponding Gerber standard image (i.e., an image obtained by rasterizing Gerber data) retrieved from the template library based on the associated metadata of the sample image (such as the model of the product under test, the inspection surface, the camera number, the field of view number, the acquisition sequence number, and the work order number); or, considering that the image width of the template image is larger than the shooting width of a single camera of the automated optical inspection equipment, the template image can also be a template segmented image obtained by segmenting the aforementioned Gerber standard image (such as segmenting along the image height direction and / or segmenting along the image width direction) (each Gerber standard image has multiple template segmented images), and the template switching image corresponding to the sample image can be determined by the camera number of the sample image.

[0030] Please see Figure 2 The illustration shows a comparison of a set of images to be registered according to some embodiments of the present disclosure, such as... Figure 2 As shown, the right side of the image set 200 to be registered shows a sample image, and the left side of the image set 200 to be registered shows a template image corresponding to the sample image. The template image is the template segmentation image obtained by segmenting the Gerber standard image mentioned above.

[0031] It is understood that each current sub-block image includes any sample sub-block image obtained by meshing the sample image, and the template sub-block image corresponding to the sample sub-block image after meshing the template image (that is, the position of the template sub-block image in the template image is the same as the position of the sample sub-block image in the sample image); the offset of each sample sub-block image in the global coordinate system can be, for example, the coordinates of the upper left corner of each sample sub-block image in the global coordinate system of the sample image, which can be, for example, an origin at the upper left corner of the sample image, and a coordinate system at the upper left corner of the sample image. The axes are in a Cartesian coordinate system parallel to the edges of the sample image (e.g., the x-axis points to the right horizontally and the y-axis points downward vertically, with units of pixels). The offset of each template sub-block image in the global coordinate system can be, for example, the coordinates of the upper left corner of each template sub-block image in the global coordinate system of the template image. This global coordinate system can be, for example, a Cartesian coordinate system with the origin at the upper left corner of the template image and axes parallel to the edges of the template image (e.g., the x-axis points to the right horizontally and the y-axis points downward vertically, with units of pixels).

[0032] For example, taking the global coordinate system of the sample image as an example, for a sample image with a width of 10,000 pixels and a height of 30,000 pixels, the coordinates of the top left corner are (0, 0), the coordinates of the top right corner are approximately (9999, 0), the coordinates of the bottom left corner are approximately (0, 29999), and the coordinates of the bottom right corner are (9999, 29999). If the top left corner of a certain sample sub-block image is located at the 2000th column and the 5000th row of the sample image, the offset of the sample sub-block image in the global coordinate system can be determined to be (2000, 5000).

[0033] To ensure the validity of each sample sub-block image obtained from the mesh partitioning process, the sample image and the template image can be preprocessed to ensure that the sample image and the template image have consistent pixel size and the same effective field of view.

[0034] In some implementations, when the processing terminal performs grid division on the image set to be registered to obtain multiple current sub-block images, it specifically identifies invalid regions from the sample images of the image set to be registered and performs cropping processing on the sample images based on the invalid regions. Here, invalid regions can be understood as black borders, blank borders, fixture areas, non-plate areas, abnormal exposure areas, and / or filled areas outside the camera's acquisition range in the sample images. Invalid regions can be identified from the sample images through image recognition algorithms, and cropping processing is performed on the sample images based on these invalid regions.

[0035] Next, a scaling factor is determined based on camera calibration parameters, template resolution, and linewidth parameters. This scaling factor is then used to scale both the processed sample image and the template image of the image set to be registered. In one example, the sample size corresponding to each pixel in the sample image can be converted based on the camera calibration parameters, and the template size corresponding to each pixel in the template image can be converted based on the template resolution (i.e., the rasterized resolution of the template image). The ratio between the sample size and the template size is then used as the main scaling factor to scale the cropped sample image, ensuring that the sample size corresponding to each pixel in the scaled sample image is the same as the template size.

[0036] Next, based on linewidth parameters (such as the minimum linewidth of the product under test and the preset pixel linewidth corresponding to defect detection), the linewidth (i.e., the number of pixels) after preliminary scaling can be obtained by combining the minimum linewidth of the product under test, the sample size corresponding to each pixel in the sample image mentioned above, and the main scaling factor. It can then be determined whether this linewidth is significantly larger than the preset pixel linewidth. Understandably, when the linewidth is significantly larger than the preset pixel linewidth, it indicates that the sample image still has some compression space. Therefore, based on the detection time requirements, an auxiliary scaling factor can be further set (e.g., a smaller auxiliary scaling factor is set when the detection time requirement is high to compress more and detect only larger defects; a larger auxiliary scaling factor is set when the detection time requirement is low to accommodate smaller defects). The sample image is then further scaled based on this auxiliary scaling factor (ensuring that the linewidth of the sample image after further scaling is still greater than the preset pixel linewidth), thereby meeting the detection time requirements.

[0037] In addition, the template image can be scaled based on the size of the scaled sample image so that the size of the scaled template image is the same as or approximately the same as the size of the sample image, and is not limited thereto.

[0038] Next, vertical projection processing is performed on the processed sample image and template image respectively, and the width boundary region is determined based on the vertical projection result. Here, vertical projection processing can be understood as accumulating (or averaging) the gray values ​​of all pixels in each column of the image to obtain a gray value vector with a length equal to the width of the image.

[0039] After determining the grayscale value vectors of the sample image and the template image respectively, the corresponding edge regions (such as the columns corresponding to the grayscale value jumping from near 0 to a larger value, and the columns corresponding to the grayscale value jumping from a larger value back to near 0) can be determined based on the grayscale value vector of the sample image, and the corresponding edge regions (such as the columns corresponding to the grayscale value jumping from near 0 to a larger value, and the columns corresponding to the grayscale value jumping from a larger value back to near 0) can be determined based on the grayscale value vector of the template image. The maximum value of the left edge in the two edge regions is taken as the common left boundary, and the minimum value of the right edge is taken as the common right boundary. Then, the region formed by the common left boundary and the common right boundary is determined as the width boundary region.

[0040] Subsequently, the processed sample image and template image are aligned based on the width boundary region, and then gridded to obtain multiple current sub-block images. It is understandable that after determining the width boundary region, the processed sample image and template image can be cropped based on this region to a common width range; alternatively, they can be padded based on the width boundary region to make their widths the same, and this is not a limitation.

[0041] In addition, for each sample image and the corresponding template image, the grid division process is performed according to the preset grid division rules; or, the number of sub-block images is calculated based on the preset target sub-block size and image resolution, and the grid division process is performed based on the number of sub-block images, so that each sample image is divided into the same number of sample sub-block images, and each template image is divided into the same number of template sub-block images, and each sample sub-block image has a corresponding template sub-block image.

[0042] Please see Figure 3 The diagram shown illustrates the grid division effect of an image set to be registered according to some embodiments of the present disclosure, such as... Figure 3 As shown, the right side of the image set 300 to be registered shows a sample image, and the left side of the image set 300 to be registered shows a template image corresponding to the sample image. The sample image is divided into 12 sample sub-block images by grid division, and the template image is divided into 12 template sub-block images by grid division, and each sample sub-block image has a corresponding template sub-block image.

[0043] In box 104, method 100 can determine the current selection weight based on the registration quality index of the historical sub-block image set and the historical selection weight corresponding to each current sub-block image, and determine multiple target sub-block images from all current sub-block images based on all current selection weights. It is understandable that, since multiple products to be tested (distinguished by different product numbers) with the same part number need to be continuously tested in an automated optical inspection production line, the consistent image structure and local image repetition characteristics of products with the same part number can be utilized. For each historical sample image of the previous multiple products to be tested, a preset sampling strategy (such as checkerboard sampling, equally spaced sampling, uniform random sampling, or edge-biased sampling) can be used to determine the historical selection weight corresponding to each sample sub-block image in each historical sample image. After obtaining the corresponding registered images, the registration quality index corresponding to each sample sub-block image is analyzed. Therefore, for the sample images of subsequent products to be tested, the current selection weight corresponding to each sample sub-block image can be finally obtained based on the historical selection weight and registration quality index corresponding to each sample sub-block image. This approach effectively reduces the matching calculations for invalid or low-quality sub-blocks, improves the efficiency of obtaining effective matching points and the stability of global registration. Compared to re-randomizing or fixing the selection of blocks for each product under test, it can gradually converge to a sub-block selection distribution that is more suitable for the current part number during mass production testing, taking into account registration speed, edge coverage and robustness.

[0044] In some implementations, when determining the current selection weight based on the registration quality index of the historical sub-block image set and the historical selection weights corresponding to each current sub-block image, the processing terminal specifically determines whether the product number corresponding to the image set to be registered exceeds a preset number threshold. Here, based on the product number corresponding to the image set to be registered, it can be determined whether the current product under test belongs to the first few products under test produced with the same part number. For example, if the product number exceeds the preset number threshold, it indicates that the current product under test does not belong to the first few products under test produced with the same part number. Then, based on the historical selection weights and registration quality index corresponding to each sample sub-block image, the current selection weight corresponding to each sample sub-block image is finally obtained. If the product number does not exceed the preset number threshold, it indicates that the current product under test belongs to the first few products under test produced with the same part number. Then, based on a preset sampling strategy (such as checkerboard sampling, equally spaced sampling, uniform random sampling, or edge-biased sampling), the current selection weight corresponding to each sample sub-block image is determined, and it is not limited to this.

[0045] Subsequently, in response to the determination that the product number exceeds a preset number threshold, a registration quality score corresponding to each current sub-block image is determined based on the registration quality index of the historical sub-block image set. Here, the registration quality index of the historical sub-block image set can be understood as the registration quality index of each sample sub-block image after obtaining the corresponding historical registration images, which corresponds to the previous sample test product with the same part number and adjacent to the current test product. This registration quality index may include, for example, the number of registration inliers, the number of matching point pairs, and the average reprojection error. The number of registration inliers can be the number of matching point pairs selected as inliers when obtaining the historical registration images, among all matching point pairs corresponding to each sample sub-block image. The number of matching point pairs is the total number of matching point pairs corresponding to each sample sub-block image. The average reprojection error can be the average reprojection error corresponding to the target homography matrix applied when generating the historical registration images, among all matching point pairs selected as inliers corresponding to each sample sub-block image.

[0046] In one example, taking the registration quality metrics as having the number of in-registration points, the number of matched point pairs, and the average reprojection error, the registration quality score corresponding to each sample sub-block image can be determined based on the registration quality metrics of each sample sub-block image in the historical sample images corresponding to the previous test product adjacent to the current test product, using the expression shown below:

[0047]

[0048] In the above formula, The registration quality score is given to the image corresponding to the sample sub-block in the i-th row and j-th column. The number of registration inliers corresponding to the sample sub-block image in the i-th row and j-th column. This represents the number of matching point pairs corresponding to the sample sub-block image in the i-th row and j-th column. The average reprojection error corresponding to the sample sub-block image in the i-th row and j-th column is... The preset attenuation coefficient (e.g., 0.5).

[0049] Next, the current selection weight is determined based on the historical selection weights and registration quality scores corresponding to each current sub-block image. Here, the current selection weight corresponding to the corresponding sample sub-block image (i.e., the current selection weight corresponding to each current sub-block image) can be determined based on the historical selection weights corresponding to each sample sub-block image and the registration quality scores of all sample sub-block images, using the expression shown below:

[0050]

[0051] In the above formula, The current selection weight is the sub-block image in the i-th row and j-th column of the sample image of the t-th product to be tested (i.e., the current product to be tested). Weights are selected for the historical data corresponding to the sample sub-block image in the i-th row and j-th column. The preset forgetting factor parameter (e.g., 0.7). The registration quality score is the image corresponding to the sample sub-block in the i-th row and j-th column of the historical sample image of the (t-1)-th test product (i.e., the previous test product adjacent to the current test product). It is the maximum value of all registration quality scores in the historical sample images of the (t-1)th test product (i.e., the previous test product adjacent to the current test product).

[0052] It is understandable that when the previous test product adjacent to the current test product belongs to the previous multiple test products produced with the same part number, the historical selection weight corresponding to each sample sub-block image can be determined based on a preset sampling strategy (such as checkerboard sampling, equal interval sampling, uniform random sampling, or edge bias sampling). For example, taking edge bias sampling as an example, the historical selection weight corresponding to the corresponding current sub-block image can be determined based on the coordinates of each sample sub-block image (center) in the historical sample image and the image center coordinates of the historical sample image, through the expression shown below.

[0053]

[0054] In the above formula, Weights are selected for the historical data corresponding to the sample sub-block image in the i-th row and j-th column. and Let be the coordinates of the sample sub-block image in the i-th row and j-th column within the historical sample images. and The coordinates of the image center of the historical sample image. This is a preset constant.

[0055] When the previous test product adjacent to the current test product does not belong to the previous multiple test products produced with the same part number, the historical selection weight corresponding to each sample sub-block image can be referred to the above process for determining the current selection weight corresponding to the sample sub-block image, which will not be elaborated here.

[0056] Of course, the registration quality indicators in the embodiments of this disclosure may also include, for example, matching confidence, the proportion of matching inliers, the frequency of participation in registration, and spatial coverage contribution. For example, for sample sub-block images that consistently generate high matching confidence, high proportion of matching inliers, and stable spatial coverage contribution among multiple test products, their selection weight in subsequent test products of the same part number is increased; while for sample sub-block images that have long-term sparse matching points, low proportion of matching inliers, large average reprojection error, or are prone to mismatches, their selection weight in subsequent test products of the same part number is reduced, or they are removed from the candidate sub-block image set. In addition, for sample sub-block images with insufficient historical matching coverage, the selection probability of their adjacent sample sub-block images can be appropriately increased to maintain the spatial coverage of global matching points.

[0057] After determining the current selection weight corresponding to each sample sub-block image, sampling without replacement can be performed on all sample sub-block images based on all current selection weights to select multiple target sample sub-block images that meet the preset selection sub-block ratio. Based on each target sample sub-block image, the corresponding target template sub-block image is determined among all template sub-block images (that is, each target sub-block image has a target sample sub-block image and a corresponding target template sub-block image).

[0058] In addition, to prevent the selection weights from converging prematurely, random exploration can be performed with a preset probability each time sampling is performed. For example, random selection can be performed uniformly from all sample sub-block images, or selection can be performed from all sample sub-block images that have never been selected. As the number of products to be tested increases, the preset probability can be gradually reduced. Furthermore, after the number of products to be tested exceeds a certain number, the selection weight of low-quality sample sub-block images will approach 0. At this time, the preset selection sub-block ratio mentioned above can be adjusted to reduce the number of sample sub-block images to be selected, thereby accelerating the registration efficiency of subsequent products to be tested. This is not limited to this.

[0059] In some implementations, when determining the current selection weight corresponding to each current sub-block image, the processing terminal further determines the structural indicators corresponding to each current sub-block image based on the template image, in response to determining that the product number does not exceed a preset number threshold. Here, since the template image comes from the Gerber standard image, by utilizing the structural prior of the template image, template sub-block images containing structurally rich areas such as line edges, vias, pads, and border intersections can be preferred (i.e., given higher selection weights), while the matching priority of template sub-block images in blank areas and repetitive texture areas can be reduced (i.e., given lower selection weights).

[0060] Understandably, after determining that the current product under test belongs to the first few products under test produced from the same part number, image recognition processing can be performed on each template sub-block image divided from the template image to determine the structural indicators corresponding to each template sub-block image. These structural indicators include, for example, edge density, corner density, pad density, and via density. Among them, edge density can be the ratio between the number of edge pixels and the total number of pixels calculated by edge detection processing of the template sub-block image; corner density can be the proportion of pixels with corner response values ​​greater than the global threshold calculated by corner detection processing of the template sub-block image; pad density can be the area ratio of the pad area calculated by extracting the pad area from the template sub-block image; and via density can be the area ratio of the via area calculated by extracting the via area from the template sub-block image.

[0061] Next, the edge density, corner density, pad density, and via density corresponding to each current sub-block image are weighted and summed to obtain the comprehensive structure score. Here, the comprehensive structure score corresponding to the corresponding template sub-block image can be determined based on the edge density, corner density, pad density, and via density corresponding to each template sub-block image using the expression shown below:

[0062]

[0063] In the above formula, The comprehensive structural score is the image corresponding to the template sub-block in the i-th row and j-th column. The edge density corresponding to the template sub-block image in the i-th row and j-th column. The corner density corresponding to the template sub-block image in the i-th row and j-th column. The pad density corresponding to the template sub-block image in the i-th row and j-th column. The pore density corresponding to the template sub-block image in the i-th row and j-th column. , , and These are the preset weight parameters.

[0064] Next, based on all comprehensive structural scores, the current selection weight corresponding to each current sub-block image is determined. Here, the current selection weight corresponding to each template sub-block image (i.e., the current selection weight corresponding to each current sub-block image) can be determined based on the comprehensive structural scores corresponding to all template sub-block images using the expression shown below:

[0065]

[0066] In the above formula, The current selected weight is the template sub-block image corresponding to the i-th row and j-th column. The comprehensive structural score is the image corresponding to the template sub-block in the i-th row and j-th column. This is the preset smoothing factor parameter.

[0067] After determining the current selection weight corresponding to each template sub-block image, sampling without replacement can be performed on all template sub-block images based on all current selection weights to select multiple target template sub-block images that meet the preset selection sub-block ratio. Based on each target template sub-block image, the corresponding target sample sub-block image is determined in all sample sub-block images (that is, each target sub-block image has a target sample sub-block image and a corresponding target template sub-block image).

[0068] In some implementations, when determining the current selection weight corresponding to each current sub-block image based on all comprehensive structural scores, the processing terminal further determines the edge bias weight based on the center coordinates of each current sub-block image and the center coordinates of the image set to be registered. Here, the edge bias weight corresponding to each sample sub-block image (i.e., the edge bias weight corresponding to each current sub-block image) can be determined based on the center coordinates of each sample sub-block image and the center coordinates of the sample image, using the expression shown below:

[0069]

[0070] In the above formula, The edge bias weights are the values ​​corresponding to the sample sub-block images in the i-th row and j-th column. and Let be the center coordinates of the sample sub-block image in the i-th row and j-th column. and The center coordinates of the sample image, This is a preset constant.

[0071] Next, based on all comprehensive structural scores, the structural weights corresponding to each current sub-block image are determined. Here, the structural weights corresponding to each current sub-block image are the same as the structural weights corresponding to each sample sub-block image (or the structural weights corresponding to each template sub-block image). The determination process can be found in the above-described process for determining the currently selected weights corresponding to each template sub-block image, and will not be elaborated upon here.

[0072] Next, based on all edge bias weights and all structure weights, the current selection weight corresponding to each current sub-block image is determined. Here, the current selection weight corresponding to each sample sub-block image can be determined based on all edge bias weights and all structure weights using the expression shown below:

[0073]

[0074] In the above formula, The current selection weight is corresponding to each current sub-block image. The edge bias weights are the values ​​corresponding to the current sub-block image in the i-th row and j-th column. The structural weights are the values ​​corresponding to the current sub-block image in the i-th row and j-th column.

[0075] Please see Figure 4 The illustration shows a selection effect diagram of a target sub-block image according to some embodiments of the present disclosure, such as... Figure 4 As shown, the left side of the target sub-block image 400 displays multiple target sub-block images selected using checkerboard sampling (black and white areas and non-black and white areas form a checkerboard pattern), while the right side of the target sub-block image 400 displays multiple target sub-block images selected using the above embodiment. It can be seen that all the target sub-block images selected on the left exhibit obvious regularity and fail to reflect a sub-block selection distribution more suitable for the current part number; all the target sub-block images selected on the right do not exhibit obvious regularity, and the selection weight for the central region is relatively large, thus reflecting a sub-block selection distribution more suitable for the current part number.

[0076] Please see Figure 5 The diagram shown illustrates the matching effect of a target sub-block image according to some embodiments of the present disclosure, such as... Figure 5 As shown, the left side of the target sub-block image 500 shows any sample sub-block image in the sample image, and the right side of the target sub-block image 500 shows the template sub-block image corresponding to the sample sub-block image.

[0077] In box 106, method 100 can perform feature matching on each target sub-block image to obtain multiple matching point pairs in the local coordinate system, and perform coordinate transformation processing on all matching point pairs based on all offsets to obtain a global matching point pair set. Here, after determining multiple target sub-block images (i.e., target sample sub-block images and target template sub-block images), grayscale conversion, normalization, contrast enhancement, or other image preprocessing can be performed on each target sample sub-block image and each target template sub-block image, respectively. Then, local feature extraction processing is performed on each target sample sub-block image and each target template sub-block image by using SuperPoint, ALIKED, SIFT, ORB, or other equivalent feature extraction algorithms, respectively. Finally, LightGlue, LoFTR, or other equivalent local feature matching algorithms are used to match all feature points of each target sample sub-block image with the corresponding target. Feature matching is performed on all feature points of the template sub-block image to obtain multiple matching point pairs of the corresponding target sub-block image in the local coordinate system. This local coordinate system can be, for example, a Cartesian coordinate system with the origin at the upper left corner of the target sample sub-block image (or the target template sub-block image), and the coordinate axes parallel to the edges of the target sample sub-block image (or the target template sub-block image) (e.g., the x-axis points to the right along the horizontal direction of the image, the y-axis points downward along the vertical direction of the image, and the coordinate unit is pixels). Each matching point pair has the coordinates of any feature point of the target sample sub-block image in the local coordinate system, and the coordinates of the target template sub-block image corresponding to the feature point in the local coordinate system.

[0078] After determining all matching point pairs for each target sub-block image, the matching point pairs can be filtered based on processing methods such as matching distance, matching confidence, nearest neighbor ratio, bidirectional consistency, or geometric consistency to obtain all matching point pairs that meet the matching requirements. It is understood that if the number of matching point pairs for any target sub-block image after filtering is lower than a preset point pair threshold, that target sub-block image will be discarded; if the number of matching point pairs for any target sub-block image after filtering is not lower than the preset point pair threshold, all matching point pairs for that target sub-block image will be retained. By integrating all retained matching point pairs, a valid set of matching point pairs is obtained, and coordinate transformation is performed on this valid set of matching point pairs based on all offsets to obtain a global set of matching point pairs.

[0079] Here, when performing coordinate transformation on the set of valid matching points based on all offsets, taking the feature point coordinates of the target sample sub-block image in the local coordinate system of any matching point pair in the set of valid matching points as (x1, y1) and the offset corresponding to the target sample sub-block image as (x2, y2) as an example, the feature point coordinates after transformation can be expressed as (x1+x2, y1+y2), which is obtained through accumulation.

[0080] In box 108, method 100 can determine multiple global homography matrices based on a global set of matching point pairs, and determine the target homography matrix from all global homography matrices based on the set of interior points corresponding to each global homography matrix. Here, after determining the global set of matching point pairs, a small number of matching point pairs (e.g., 4) can be randomly selected multiple times from this set, and a robust estimation algorithm (such as RANSAC, USAC, or other equivalent robust estimation methods) can be used to solve for all the selected matching point pairs each time, thereby obtaining the global homography matrix from the sample image to the template image. It should be noted that if the solution fails, a failure flag is output and a rollback is performed to redetermine the global set of matching point pairs.

[0081] In some implementations, when the processing terminal determines the target homography matrix from all global homography matrices based on the set of inliers corresponding to each global homography matrix, it specifically determines the set of inliers whose reprojection error is less than a preset error threshold based on each global homography matrix and the global matching point pair set. Here, after determining each global homography matrix, the feature point coordinates corresponding to the target sample sub-block image of each matching point pair in the global matching point pair set are substituted into each global homography matrix to solve for the corresponding predicted feature point coordinates, and the reprojection error between the feature point coordinates corresponding to the target template sub-block image of the corresponding matching point pair and the predicted feature point coordinates is calculated (e.g., the Euclidean distance between the two is determined as the reprojection error).

[0082] Understandably, after obtaining all reprojection errors corresponding to each global homography matrix, all reprojection errors can be filtered based on a preset error threshold to filter out all reprojection errors less than the preset error threshold, and all matching point pairs corresponding to all filtered reprojection errors can be determined as the set of interior points of the corresponding global homography matrix.

[0083] Next, based on the set of all interior points corresponding to all global homography matrices, the target matching point pair set is determined, and it is determined whether the number of matching point pairs in the target matching point pair set exceeds a preset threshold. Here, based on the set of all interior points corresponding to all global homography matrices, the set of interior points with the largest number of matching point pairs can be taken as the target matching point pair set, and it is determined whether the number of matching point pairs in the target matching point pair set exceeds the preset threshold. Understandably, the target matching point set has the characteristics of having the most interior points and the smallest error. When the number of matching point pairs in the target matching point set exceeds a preset threshold, it indicates that the target matching point set is valid and reliable (i.e., registration is successful). If the number of matching point pairs in the target matching point set is much greater than the preset threshold, downsampling can be used to reduce the number of matching point pairs in the target matching point set, thereby ensuring image registration efficiency. When the number of matching point pairs in the target matching point set does not exceed the preset threshold, it indicates that the target matching point set is not valid and reliable (i.e., registration fails). A failure flag is then output and a rollback process is performed to redetermine the target matching point set.

[0084] Subsequently, in response to the determination that the number of matching point pairs exceeds a preset threshold, the target homography matrix is ​​determined based on the target matching point pair set. Here, after determining that the number of matching point pairs exceeds the preset threshold, the target matching point pair set can be solved by using a robust estimation algorithm (such as RANSAC, USAC, or other equivalent robust estimation methods), or the corresponding global homography matrix can be optimized based on the target matching point pair set to obtain the final target homography matrix.

[0085] In box 110, method 100 can perform perspective transformation on the sample image based on the target homography matrix to obtain a registered image. It can be understood that by performing perspective transformation on the sample image based on the target homography matrix, a registered image aligned with the template image is obtained, and this registered image is output, or the result of overlaying the registered image with the template image is output.

[0086] Please see Figure 6 The diagram shown illustrates the output effect of a registered image according to some embodiments of the present disclosure, such as... Figure 6 As shown, the output registration image 600 is the result of overlaying the registration image and the template image. Its blue channel corresponds to the template image, and its green channel corresponds to the registration image.

[0087] For ultra-large format sample images, the output area can be divided into multiple transformation sub-regions for block perspective transformation and stitching processing. This effectively avoids the high memory consumption required for full-image perspective transformation of the entire sample image at once, and is easier to deploy under the actual computing power and memory conditions of the production line, thereby maintaining the integrity of the output registered image.

[0088] In some implementations, when the processing terminal performs perspective transformation on the sample image based on the target homography matrix to obtain a registered image, it further determines whether the size parameters of the sample image exceed a preset size threshold. Here, the size parameters of the sample image are, for example, image width and image height, and the preset size threshold is, for example, a preset width threshold and a preset height threshold. When the image width of the sample image exceeds the preset width threshold, and / or the image height of the sample image exceeds the preset height threshold, it indicates that the sample image is an ultra-large format image, and thus it is determined that the size parameters of the sample image exceed the preset size threshold. When the image width of the sample image does not exceed the preset width threshold, and the image height of the sample image does not exceed the preset height threshold, it indicates that the sample image is not an ultra-large format image, and thus it is determined that the size parameters of the sample image do not exceed the preset size threshold.

[0089] Subsequently, in response to the determination that the size parameter exceeds a preset size threshold, the sample image is divided into multiple sub-sample images according to a preset direction. Here, based on a preset strip height, the sample image can be uniformly divided along the direction of the image width, so that the image height of each sub-sample image divided in each row of the sample image is the preset strip height, and the image width is the same. It can be understood that when the image height of the sample image is not divisible by the preset strip height, the image height of each sub-sample image divided in the last row of the sample image is the same and less than the preset strip height.

[0090] Next, perspective transformation is performed on each sub-sample image based on the target homography matrix, and all processed sub-sample images are stitched together to obtain the registered image. Here, after dividing the sample image into multiple sample sub-images, the pixel coordinates of each sample sub-image in the global coordinate system of the sample image can be determined. Then, based on the target homography matrix and all pixel coordinates corresponding to each sample sub-image, perspective transformation is performed on the corresponding sub-sample images, and all processed sub-sample images are stitched together according to their positions in the sample image to obtain the final complete registered image.

[0091] The above embodiments specifically bring about the following technical effects:

[0092] Improving the stability of homography matrix solution and registration success rate: By dividing the image into multiple sub-blocks, obtaining matching point pairs within each sub-block, and then converting local matching points into global coordinates based on sub-block offsets, the matching point pairs from multiple sub-blocks are aggregated to form a global point set, thereby robustly solving the global homography matrix in one go. Since the global point set can cover multiple regions simultaneously, even if some sub-blocks have insufficient matching points due to low texture, occlusion, or insufficient overlap, it will not prevent the entire image from failing to solve for the homography matrix. Compared to existing methods that "calculate the homography matrix for each sub-block separately," this effectively avoids local unsolvability or numerical instability caused by insufficient points within a single block, thereby improving the stability and success rate of registration.

[0093] Reducing boundary discontinuities and stitching artifacts caused by local transformation inconsistencies: Existing local homography methods typically estimate different transformations for different regions, which easily leads to transformation inconsistencies at sub-block boundaries, further introducing boundary discontinuities, local distortions, or stitching seams. In contrast, the embodiments of this disclosure employ a "global one-time homography matrix calculation" approach, enabling the same sample image to be aligned under the same transformation model. This reduces the sources of cross-regional inconsistencies at the transformation model level, thereby reducing boundary artifacts and improving the continuity and interpretability of the overlay results.

[0094] Improve processing efficiency and reduce computational overhead: Based on block processing, a target sub-block selection mechanism is introduced, performing feature extraction and matching only on a subset of sub-blocks; at the same time, an upper limit is set on the number of matching points within a single sub-block to avoid generating too many redundant matching points in local areas; since feature extraction and matching are the main time-consuming steps, reducing the number of sub-blocks involved in matching and the size of control point pairs can significantly reduce the overall computational load; and the global homography matrix is ​​solved only once, avoiding the redundant estimation overhead caused by solving the homography matrix separately for multiple sub-blocks, thereby improving processing efficiency in online detection scenarios.

[0095] Enhance engineering usability and output consistency in abnormal scenarios: Implement filtering strategies for sub-blocks with insufficient matching point pairs, and set thresholds for the number of interior points and validity checks for global homography matrix solving; perform rollback processing when there are insufficient matching points, homography matrix solving fails, or there are too few interior points. Through the above mechanisms, abnormal registration results can be avoided in scenarios with low texture, reflection pollution, incomplete camera coverage, etc., improving stable operation and result consistency under complex working conditions.

[0096] Adapting to ultra-large format images, reducing memory pressure and facilitating engineering deployment: For ultra-large format images, this proposal allows the output area to be divided into multiple transformation sub-regions and subjected to block perspective transformation and stitching. Block processing avoids the high memory consumption required for a full-image perspective transformation of the entire large image at once, making the method easier to deploy under the actual computing power and memory conditions of production lines, while maintaining the integrity of the output registered image.

[0097] Figure 7 A block diagram of an automatic optical detection image registration system based on block matching according to some embodiments of the present disclosure is shown. The various embodiments in this specification are described in a progressive manner, with reference to each other for similar or identical parts. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments. Figure 7 As shown, the automatic optical detection image registration system 700 based on block matching may include at least an offset determination module 702, configured to perform grid division processing on the image set to be registered, obtain multiple current sub-block images, and determine the offset of each current sub-block image in the global coordinate system; the image set to be registered has sample images and template images. The automatic optical detection image registration system 700 based on block matching also includes a target sub-block selection module 704, configured to determine the current selection weight based on the registration quality index of the historical sub-block image set and the historical selection weight corresponding to each current sub-block image, and determine multiple target sub-block images from all current sub-block images based on all current selection weights. The automatic optical detection image registration system 700 based on block matching also includes a matching point pair determination module 706, configured to perform feature matching on each target sub-block image, obtain multiple matching point pairs in the local coordinate system, and perform coordinate transformation processing on all matching point pairs based on all offsets to obtain a global matching point pair set. The block-matching-based automatic optical inspection image registration system 700 further includes a homography matrix determination module 708, configured to determine multiple global homography matrices based on a global matching point pair set, and to determine a target homography matrix from all global homography matrices based on the set of interior points corresponding to each global homography matrix. Furthermore, the block-matching-based automatic optical inspection image registration system 700 also includes a registration image generation module 710, configured to perform perspective transformation processing on the sample image based on the target homography matrix to obtain a registration image.

[0098] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0099] Figure 8 Block diagrams of electronic devices that can implement various embodiments of the present disclosure are shown. For example... Figure 8 As shown, the electronic device 800 includes a processor 801, which can perform various appropriate actions and processes based on computer program instructions loaded into random access memory (RAM) 803 according to computer program instructions stored in read-only memory (ROM) 802. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The processor 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0100] The various processes and procedures described above, such as method 100, can be executed by processor 801. For example, in some embodiments, method 100 may be implemented as a software program tangibly contained in a machine-readable medium. In some embodiments, part or all of the software program may be loaded into and / or installed onto electronic device 800 via ROM 802. When the software program is loaded into RAM 803 and executed by processor 801, one or more actions of method 100 described above may be performed.

[0101] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0102] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0103] This disclosure can be a method, apparatus, system, and / or program product. The program product may include a machine-readable storage medium on which machine-readable program instructions for performing various aspects of this disclosure are loaded. The machine-readable program instructions described herein can be downloaded from the machine-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the machine-readable program instructions from the network and forwards them to the machine-readable storage medium in the respective computing / processing device.

[0104] Machine program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. Machine-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the machine-readable program instructions to implement various aspects of this disclosure.

[0105] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0106] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An automatic optical detection image registration method based on block matching, characterized in that, include: Based on the set of images to be registered, the set of images to be registered is divided into grids to obtain multiple current sub-block images, and the offset of each current sub-block image in the global coordinate system is determined. The set of images to be registered includes sample images and template images; Based on the registration quality index of the historical sub-block image set and the historical selection weight corresponding to each current sub-block image, the current selection weight is determined, and based on all the current selection weights, multiple target sub-block images are determined from all the current sub-block images. Feature matching is performed on each of the target sub-block images to obtain multiple matching point pairs in the local coordinate system, and coordinate transformation is performed on all the matching point pairs based on all the offsets to obtain a global matching point pair set; Multiple global homography matrices are determined based on the global matching point pair set, and a target homography matrix is ​​determined from all the global homography matrices based on the set of interior points corresponding to each global homography matrix. as well as Based on the target homography matrix, the sample image is subjected to perspective transformation to obtain a registered image.

2. The method according to claim 1, characterized in that, The process involves dividing the image set to be registered into grids to obtain multiple current sub-block images, including: Identify invalid regions from sample images in the image set to be registered, and crop the sample images based on the invalid regions; The scaling factor is determined based on the camera calibration parameters, template resolution, and line width parameters, and the processed sample image and the template image of the image set to be registered are scaled based on the scaling factor. The processed sample image and template image are subjected to vertical projection processing respectively, and the width boundary region is determined based on the vertical projection result; and Based on the width boundary region, the processed sample image and the template image are aligned respectively, and the processed sample image and the template image are divided into grids to obtain multiple current sub-block images.

3. The method according to claim 1, characterized in that, The determination of the current selection weight based on the registration quality index of the historical sub-block image set and the historical selection weight corresponding to each current sub-block image includes: Determine whether the product number corresponding to the set of images to be registered exceeds a preset number threshold; In response to determining that the product number exceeds the preset number threshold, a registration quality score is determined for each current sub-block image based on the registration quality index of the historical sub-block image set; the registration quality index includes the number of intra-registration points, the number of matched point pairs, and the average reprojection error; and The current selection weight is determined based on the historical selection weights corresponding to each current sub-block image and the registration quality score.

4. The method according to claim 3, characterized in that, The method further includes: In response to determining that the product number does not exceed the preset number threshold, structural indicators corresponding to each current sub-block image are determined based on the template image; the structural indicators include edge density, corner density, pad density, and via density; The edge density, corner density, pad density, and via density corresponding to each current sub-block image are weighted and summed to obtain a comprehensive structural score; and Based on all the comprehensive structural scores, the current selection weight corresponding to each of the current sub-block images is determined.

5. The method according to claim 4, characterized in that, The step of determining the current selection weight corresponding to each of the current sub-block images based on all the comprehensive structural scores further includes: The edge offset weights are determined based on the center coordinates of each current sub-block image and the center coordinates of the image set to be registered; Based on all the comprehensive structural scores, determine the structural weights corresponding to each of the current sub-block images; and Based on all the edge bias weights and all the structure weights, determine the current selection weight corresponding to each of the current sub-block images.

6. The method according to claim 1, characterized in that, The step of determining the target homography matrix from all the global homography matrices based on the set of interior points corresponding to each of the global homography matrices includes: Based on the global homography matrices and the global matching point pair set, determine the set of interior points whose reprojection error is less than a preset error threshold; Based on the set of all interior points corresponding to all the global homography matrices, a target matching point pair set is determined, and it is determined whether the number of matching point pairs in the target matching point pair set exceeds a preset threshold; and In response to determining that the number of matching point pairs exceeds the preset number threshold, a target homography matrix is ​​determined based on the target matching point pair set.

7. The method according to claim 1, characterized in that, The step of performing perspective transformation processing on the sample image based on the target homography matrix to obtain a registered image includes: Determine whether the size parameter of the sample image exceeds a preset size threshold; In response to determining that the size parameter exceeds the preset size threshold, the sample image is divided into multiple sub-sample images according to a preset direction; and Based on the target homography matrix, perspective transformation is performed on each of the sub-sample images, and all the processed sub-sample images are stitched together to obtain a registered image.

8. An automatic optical detection image registration system based on block matching, characterized in that, include: The offset determination module is configured to perform grid division processing on the image set to be registered based on the image set to be registered, to obtain multiple current sub-block images, and to determine the offset of each current sub-block image in the global coordinate system; the image set to be registered has sample images and template images; The target sub-block selection module is configured to determine the current selection weight based on the registration quality index of the historical sub-block image set and the historical selection weight corresponding to each current sub-block image, and to determine multiple target sub-block images from all the current sub-block images based on all the current selection weights. The matching point pair determination module is configured to perform feature matching on each of the target sub-block images to obtain multiple matching point pairs in the local coordinate system, and perform coordinate transformation processing on all the matching point pairs based on all the offsets to obtain a global matching point pair set; The homography matrix determination module is configured to determine multiple global homography matrices based on the global matching point pair set, and to determine a target homography matrix from all the global homography matrices based on the set of interior points corresponding to each of the global homography matrices. as well as The registration image generation module is configured to perform perspective transformation processing on the sample image based on the target homography matrix to obtain the registration image.

9. A computer-readable storage medium having a computer program stored thereon, the computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the steps of the method as claimed in any one of claims 1-7.

10. An electronic device, characterized in that, include: One or more processors, and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method as described in any one of claims 1-7.

Images