An Image Registration Method for Ultra-Large-Size Ceramic Substrates

Abstract

The present invention relates to the technical field of ceramic substrate image registration, and discloses a method for image registration of super-large-sized ceramic substrates, comprising the following steps: acquiring images from multiple line scan cameras; complete image stitching; coarse registration between a template image and an image to be tested; fine registration between the template image and the image to be tested; and global application of the fine registration results. The present invention is based on two 31000×16000 images acquired by two line scan cameras, obtains their stitching matrix, performs coarse and fine registration on the template image and the image to be tested, and applies the fine registration results to the global situation to obtain an efficient robust registration matrix, thereby improving detection accuracy and reducing false detection and missed detection rates. In addition, through the imaging mode of two line scan cameras, the problem of insufficient coverage of a single line scan camera is alleviated, and the scale of detection data is expanded; the use of a coarse-to-fine registration method achieves speed while ensuring registration accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of ceramic substrate image registration, and in particular to an image registration method for an oversized ceramic substrate. Background Art

[0002] With the continuous progress of my country's industry, the technology and process of ceramic substrates have been continuously improved, and many market segments have emerged. The performance of different types of ceramic substrates is also very different. They are widely used in integrated circuits, mobile communications, automotive electronics, aviation medical and other fields, and the market is extremely broad.

[0003] Many domestic ceramic companies have placed higher demands on printing processes, urgently requiring strict quality control through quality inspections to improve product quality. Printing quality inspections rely on the difference between the template image and the defect image. The accuracy of the registration results significantly impacts the accuracy of inspection quality.

[0004] Common ceramic substrate production generally uses two sizes of raw ceramic sheets: 6 inches and 8 inches, with side lengths of 152mm and 203mm. The inspection requirements of different areas of the circuit pattern are different. The finest line width and line spacing are 0.1mm, and the minimum detectable defect size is 20μm. The main defects are short circuits, opens, debris, dents, burrs, pinholes, etc.

[0005] To address the problem of line scan cameras not being able to cover the entire area of a ceramic substrate, and to ensure detectable defect size, two line scan cameras were used to capture two 31000×16000 images, which were then stitched together to form a complete image. Since the stitched image was so large, it required registration between the template image and the image sample to be inspected. Therefore, we developed a registration method for oversized ceramic substrate images to address this registration issue. Summary of the Invention

[0006] In view of this, an object of the present invention is to provide an image registration method for an ultra-large ceramic substrate to solve the problems pointed out in the above background technology.

[0007] In order to achieve the above-mentioned purpose of the invention, the technical solutions adopted are as follows:

[0008] A method for image registration of an oversized ceramic substrate comprises the following steps:

[0009] S1. Acquire images from multiple line scan cameras

[0010] Install multiple line scan cameras side by side so that they are relatively fixed in position. Also, ensure that the scanning areas of adjacent line scan cameras overlap during scanning. This ensures sufficient overlap for stitching multiple images.

[0011] S2. Complete image stitching

[0012] Based on the position setting in step S1, a mathematical model assumes that the image transformation matrix between any two line scan cameras is an affine transformation and that its affine transformation matrix is fixed. In other words, the image transformation matrix calculated once by the present invention can be applied to both the defect image and the template image and can be reused multiple times. Based on this, the direct stitching matrix of multiple line scan cameras is calculated offline and the calculation results are stored. When used online, the offline calculated matrix is directly read for image stitching.

[0013] In order to meet the real-time requirements, the present invention adopts a coarse-to-fine registration method. The purpose of coarse registration is to reduce the registration error between the template image and the image to be measured to a certain range and narrow the solution space of the transformation matrix of the two images.

[0014] S3. Rough registration between the template image and the image to be tested

[0015] First, the image is reduced by m times to obtain a small-sized image, and then the SIFT feature is used to calculate the transformation matrix between the small-sized images. Using this calculation method, the error of the calculated rough transformation matrix will be m times that of the source image, but the benefit is that the calculation speed is increased by m times;

[0016] S4. Precise registration between the template image and the image to be tested

[0017] After the images are registered in step S3, there is a misregistration difference of 1 to 20 pixels between the two sets of images. In order to further reduce the error and provide a high-precision registered image, and to meet the real-time requirements, a local area registration method is used to achieve precise registration.

[0018] This precise registration calculation method can ensure accurate registration within the cropped local area, while the computational complexity of this step is only 1 / n of that of the source image. Note that the transformation matrix of this step is a bounded matrix, and its value range is related to the coarse registration accuracy of step S3.

[0019] The precise registration calculation method is as follows:

[0020] S4-1, selecting the source image, that is, the area with rich texture features in the ceramic substrate circuit pattern, and cropping it, with the cropping area being 1 / n of the source image;

[0021] S4-2, obtaining a precise registration matrix by calculating the transformation matrix between the two cropped images;

[0022] S5. Global application of precise registration results

[0023] The precise registration matrix obtained in step S4 cannot be directly applied to the two sets of source images. Instead, the following steps are required:

[0024] S5-1, aligning the coordinate relationship between the local cropping area and the overall source image, integrating the two sets of coordinates into the same reference system;

[0025] S5-2, using the precise registration matrix to perform sampling and obtain a certain amount of corresponding points;

[0026] S5-3. Convert the corresponding point coordinates from the local image to the global image coordinates, and use the obtained global image coordinates to calculate the final source image precise registration matrix.

[0027] In the above steps, the present invention ensures real-time performance during online calculation through offline calculation in step S2. Through coarse registration and local registration in steps S3, S4, and S5, the computational effort of the present invention is reduced to (m+n) / mn of the direct overall computational effort.

[0028] This method uses multiple line scan cameras to capture and stitch multiple images, creating a large-scale 31,000×31,000 template image and the image to be tested for easy registration. During registration, a coarse registration process is first established to align the template image and the sample to be tested within a certain error range. The results of the coarse registration are then used as input for fine registration, and a representative region of the coarse registration is further adjusted using an affine transformation. Finally, the local affine matrix is projected back onto the overall image to achieve fine registration of the entire image.

[0029] The essence of the SIFT algorithm is to find key points (feature points) in different scale spaces and calculate the direction of the key points. The key points found by SIFT are very prominent points that do not change due to factors such as lighting, affine transformation and noise, such as corner points, edge points, bright spots in dark areas and dark spots in bright areas.

[0030] The SIFT algorithm operation steps include the following steps:

[0031] (1) Image Pyramid

[0032] The image Image(x,y) is blurred and downsampled using a Gaussian function to obtain L(x,y,σ)=G(x,y,σ)×Image(x,y).

[0033] (2) Detection of spatial extreme points (i.e. key points)

[0034] Keypoints are composed of local extreme points in the DOG space. Initial keypoint detection is done by comparing two adjacent layers of each DOG image within the same group. To find the extreme points of the DOG function, each pixel is compared with all its neighbors to see if it is larger or smaller than its neighbors in the image domain and scale domain, ensuring that extreme points are detected in both scale space and 2D image space.

[0035] (3) Key point direction allocation

[0036] In order to make the descriptor rotation-invariant, it is necessary to use the local features of the image to assign a reference direction to each key point. The image gradient method is used to obtain the stable direction of the local structure.

[0037] (4) Feature point descriptor

[0038] Each key point has three pieces of information: position, scale, and orientation. A descriptor is created for each key point so that it does not change with various changes, such as lighting changes, perspective changes, etc., and the descriptor should have high uniqueness to increase the probability of correct matching of feature points.

[0039] When stitching based on the SIFT algorithm, the steps are as follows:

[0040] (1) Convert the input image into a grayscale image using the cv2.cvtColor(left,cv2.THRESH_BINARY) function;

[0041] (2) Create a SIFT object using cv2.xfeatures2d.SIFT_create(). No parameters are required. To solve feature points and feature vectors, use sift.detectAndCompute(gray1,None). The first parameter is the image to be solved for feature points and feature vectors. The second parameter determines whether to solve for a specific area. A mask is passed in. The return value kps is the feature point solved for the image. It is a one-dimensional vector, in which each element is of keypoint type. dp is the feature vector corresponding to kps, which is also a list, in which each element is a vector of length 128.

[0042] (3) Construct a BFMacher object using cv2.BFMatcher();

[0043] (4) Use the knnMatch method to perform multi-point matching of key points, use bf.knnMatch(dpA, dpB, k=2) to match, and return a list;

[0044] (5) Remove unreliable matches and traverse matches. If the Euclidean distance of the first pair of points is less than k times the Euclidean distance of the second pair of points, it means that the first pair of points is reliable. The index number of this pair of points is appended to good_matches.

[0045] (6) Find reliable matching points and convert types;

[0046] (7) Use cv2.findHomography to solve the transformation matrix;

[0047] (8) Image space transformation, using cv2.warpPerspective function;

[0048] (9) Image stitching.

[0049] The SIFT algorithm obtains feature points and their corresponding feature descriptions in the image. The KNN algorithm is generally used to match the feature points. The K-nearest neighbor algorithm finds the K closest data points in space and groups them into one category. If the ratio of the closest point to the closest point is greater than a predetermined value, the closest point is retained and the matching point is considered a good match.

[0050] The beneficial effects of the present invention are as follows: based on two 31000×16000 images acquired by two line scan cameras, the present invention obtains their stitching matrix, adopts the two-step technology proposed by the present invention to perform coarse and fine registration on the template image and the image to be detected, and applies the fine registration result to the global situation to obtain an efficient robust registration matrix, thereby improving detection accuracy and reducing the false detection rate and missed detection rate.

[0051] The use of two line scan cameras alleviates the limited coverage of a single line scan camera and enables the expansion of inspection data. A coarse-to-fine registration approach significantly accelerates registration accuracy while ensuring real-time performance for industrial mass production applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0053] Figure 1 It is a flow chart of an image registration method for an ultra-large-size ceramic substrate according to the present invention;

[0054] Figure 2 is the image captured by the line scan camera on the left side of the present invention;

[0055] Figure 3 is the image captured by the line scan camera on the right side of the present invention;

[0056] Figure 4 is a flow chart of the SIFT algorithm according to an embodiment of the present invention;

[0057] Figure 5 is a checkerboard test result of a coarse registration image according to an embodiment of the present invention;

[0058] Figure 6 is a checkerboard test result of a coarse registration image according to an embodiment of the present invention;

[0059] Figure 7 is the image registration result of the embodiment of the present invention. DETAILED DESCRIPTION

[0060] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0061] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0062] like Figure 1-7 As shown, a method for image registration of an oversized ceramic substrate includes the following steps:

[0063] S1. Acquire images from multiple line scan cameras

[0064] Install multiple line scan cameras side by side so that they are relatively fixed in position. Also, ensure that the scanning areas of adjacent line scan cameras overlap during scanning. This ensures sufficient overlap for stitching multiple images.

[0065] S2. Complete image stitching

[0066] Based on the position setting in step S1, a mathematical model assumes that the image transformation matrix between any two line scan cameras is an affine transformation and that its affine transformation matrix is fixed. In other words, the image transformation matrix calculated once by the present invention can be applied to both the defect image and the template image and can be reused multiple times. Based on this, the direct stitching matrix of multiple line scan cameras is calculated offline and the calculation results are stored. When used online, the offline calculated matrix is directly read for image stitching.

[0067] In order to meet the real-time requirements, the present invention adopts a coarse-to-fine registration method. The purpose of coarse registration is to reduce the registration error between the template image and the image to be measured to a certain range and narrow the solution space of the transformation matrix of the two images.

[0068] S3. Rough registration between the template image and the image to be tested

[0069] First, the image is reduced by m times to obtain a small-sized image, and then the SIFT feature is used to calculate the transformation matrix between the small-sized images. Using this calculation method, the error of the calculated rough transformation matrix will be m times that of the source image, but the benefit is that the calculation speed is increased by m times;

[0070] S4. Precise registration between the template image and the image to be tested

[0071] After the images are registered in step S3, there is a misregistration difference of 1 to 20 pixels between the two sets of images. In order to further reduce the error and provide a high-precision registered image, and to meet the real-time requirements, a local area registration method is used to achieve precise registration.

[0072] This precise registration calculation method can ensure accurate registration within the cropped local area, while the computational complexity of this step is only 1 / n of that of the source image. Note that the transformation matrix of this step is a bounded matrix, and its value range is related to the coarse registration accuracy of step S3.

[0073] The precise registration calculation method is as follows:

[0074] S4-1, selecting the source image, that is, the area with rich texture features in the ceramic substrate circuit pattern, and cropping it, with the cropping area being 1 / n of the source image;

[0075] S4-2, obtaining a precise registration matrix by calculating the transformation matrix between the two cropped images;

[0076] S5. Global application of precise registration results

[0077] The precise registration matrix obtained in step S4 cannot be directly applied to the two sets of source images. Instead, the following steps are required:

[0078] S5-1, aligning the coordinate relationship between the local cropping area and the overall source image, integrating the two sets of coordinates into the same reference system;

[0079] S5-2, using the precise registration matrix to perform sampling and obtain a certain amount of corresponding points;

[0080] S5-3. Convert the corresponding point coordinates from the local image to the global image coordinates, and use the obtained global image coordinates to calculate the final source image precise registration matrix.

[0081] In the above steps, the present invention ensures real-time performance during online calculation through offline calculation in step S2. Through coarse registration and local registration in steps S3, S4, and S5, the computational effort of the present invention is reduced to (m+n) / mn of the direct overall computational effort.

[0082] This method uses multiple line scan cameras to capture and stitch multiple images, creating a large-scale 31,000×31,000 template image and the image to be tested for easy registration. During registration, a coarse registration process is first established to align the template image and the sample to be tested within a certain error range. The results of the coarse registration are then used as input for fine registration, and a representative region of the coarse registration is further adjusted using an affine transformation. Finally, the local affine matrix is projected back onto the overall image to achieve fine registration of the entire image.

[0083] The essence of the SIFT algorithm is to find key points (feature points) in different scale spaces and calculate the direction of the key points. The key points found by SIFT are very prominent points that do not change due to factors such as lighting, affine transformation and noise, such as corner points, edge points, bright spots in dark areas and dark spots in bright areas.

[0084] The SIFT algorithm operation steps include the following steps:

[0085] (1) Image Pyramid

[0086] The image Image(x,y) is blurred and downsampled using a Gaussian function to obtain L(x,y,σ)=G(x,y,σ)×Image(x,y).

[0087] (2) Detection of spatial extreme points (i.e. key points)

[0088] Keypoints are composed of local extreme points in the DOG space. Initial keypoint detection is done by comparing two adjacent layers of each DOG image within the same group. To find the extreme points of the DOG function, each pixel is compared with all its neighbors to see if it is larger or smaller than its neighbors in the image domain and scale domain, ensuring that extreme points are detected in both scale space and 2D image space.

[0089] (3) Key point direction allocation

[0090] In order to make the descriptor rotation-invariant, it is necessary to use the local features of the image to assign a reference direction to each key point. The image gradient method is used to obtain the stable direction of the local structure.

[0091] (4) Feature point descriptor

[0092] Each key point has three pieces of information: position, scale, and orientation. A descriptor is created for each key point so that it does not change with various changes, such as lighting changes, perspective changes, etc., and the descriptor should have high uniqueness to increase the probability of correct matching of feature points.

[0093] When stitching based on the SIFT algorithm, the steps are as follows:

[0094] (1) Convert the input image into a grayscale image using the cv2.cvtColor(left,cv2.THRESH_BINARY) function;

[0095] (2) Create a SIFT object using cv2.xfeatures2d.SIFT_create(). No parameters are required. To solve feature points and feature vectors, use sift.detectAndCompute(gray1,None). The first parameter is the image to be solved for feature points and feature vectors. The second parameter determines whether to solve for a specific area. A mask is passed in. The return value kps is the feature point solved for the image. It is a one-dimensional vector, in which each element is of keypoint type. dp is the feature vector corresponding to kps, which is also a list, in which each element is a vector of length 128.

[0096] (3) Construct a BFMacher object using cv2.BFMatcher();

[0097] (4) Use the knnMatch method to perform multi-point matching of key points, use bf.knnMatch(dpA, dpB, k=2) to match, and return a list;

[0098] (5) Remove unreliable matches and traverse matches. If the Euclidean distance of the first pair of points is less than k times the Euclidean distance of the second pair of points, it means that the first pair of points is reliable. The index number of this pair of points is appended to good_matches.

[0099] (6) Find reliable matching points and convert types;

[0100] (7) Use cv2.findHomography to solve the transformation matrix;

[0101] (8) Image space transformation, using cv2.warpPerspective function;

[0102] (9) Image stitching.

[0103] The SIFT algorithm obtains feature points and their corresponding feature descriptions in the image. The KNN algorithm is generally used to match the feature points. The K-nearest neighbor algorithm finds the K closest data points in space and groups them into one category. If the ratio of the closest point to the closest point is greater than a predetermined value, the closest point is retained and the matching point is considered a good match.

[0104] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, improvements, component splits, or combinations, etc., that fall within the spirit and principles of the present invention, shall be included within the scope of protection of the present invention.

Claims

1. A method for image registration of an oversized ceramic substrate, characterized in that: The following steps are involved: S1. Acquire images from multiple line scan cameras Install multiple line scan cameras side by side so that their positions are relatively fixed. During the scanning process, ensure that there is overlap between the scanning areas of adjacent line scan cameras. S2. Complete image stitching Based on the position setting in step S1, a mathematical model is established assuming that the image transformation matrix between any two line scan cameras is an affine transformation and that the affine transformation matrix is fixed. Based on this, the stitching matrix of multiple line scan cameras is calculated offline and the calculation result is stored. When used online, the offline calculated matrix is directly read for image stitching. S3. Rough registration between the template image and the image to be tested First, the image is reduced by m times to obtain a small-sized image, and then the SIFT feature is used to calculate the transformation matrix between the small-sized images; S4. Precise registration between the template image and the image to be tested After the images are registered in step S3, there is a misregistration difference of 1 to 20 pixels between the two groups of images. Local area registration is used to achieve precise registration. The precise registration calculation method is as follows: S4-1, selecting the source image, that is, the area with rich texture features in the ceramic substrate circuit pattern, and cropping it, with the cropping area being 1 / n of the source image; S4-2, obtaining a precise registration matrix by calculating the transformation matrix between the two cropped images; S5. Global application of precise registration results The precise registration matrix obtained in step S4 cannot be directly applied to the two sets of source images. Instead, the following steps are required: S5-1, aligning the coordinate relationship between the local cropping area and the overall source image, integrating the two sets of coordinates into the same reference system; S5-2, using the precise registration matrix to perform sampling and obtain a certain amount of corresponding points; S5-3. Convert the corresponding point coordinates from the local image to the global image coordinates, and use the obtained global image coordinates to calculate the final source image precise registration matrix.

Images