Image stitching method and defect detection method

Abstract

The application discloses an image splicing method and a defect detection method, comprising collecting a plurality of detection images of a to-be-detected object located on a moving platform moving along a preset path, and a platform actual coordinate of the moving platform corresponding to each detection image; inputting each detection image and the corresponding platform actual coordinate into a geometric mapping model, performing geometric transformation on each detection image to obtain a plurality of to-be-spliced images, wherein the geometric mapping model is obtained by training according to the platform actual coordinates of a plurality of moving platforms, the corresponding detection images and the positions of feature points in the corresponding detection images in a global splicing coordinate system; and splicing each to-be-spliced image to obtain an overall splicing image of the to-be-detected object, wherein the method does not need to input or identify the positions of feature points of the corresponding detection images during use, can not rely on image feature matching for main positioning, and is suitable for stable splicing of transparent, weak-texture, low-contrast and non-pattern samples.

Description

Technical Field

[0001] This invention relates to the field of defect detection, and specifically to an image stitching method and a defect detection method. Background Technology

[0002] In existing large-size workpiece visual inspection, when the size of the object being inspected exceeds the single field of view of the camera, a motion platform is typically used to move the camera or workpiece along a preset path, acquiring multiple frames of local images at different positions. These multiple frames are then stitched together into a complete image according to theoretical step distance or image feature matching. Common solutions include: 1. Tiling adjacent images according to a fixed pixel offset based on the theoretical step distance of the motion platform; 2. Correcting the position of adjacent images by using feature points, grayscale correlation, or template matching results in overlapping areas; 3. Obtaining pixel equivalents through camera calibration, and then calculating the stitching position of the images in the larger image based on the platform's movement distance; 4. For line scanning or TDI scanning systems, the scanned images are usually arranged according to the scanning direction based on encoder pulses or theoretical line frequency. The aforementioned methods can achieve basic mosaicking in scenarios with rich textures and low requirements for stitching accuracy. However, in scenarios such as telecentric high-precision inspection, transparent substrates, samples with weak textures, patternless wafers, glass wafers, and TGV substrates, there are fewer natural features in the images that can be used for matching, and random defects such as contamination, bubbles, scratches, and dust can easily cause mismatches. Therefore, traditional stitching methods that rely on image feature matching lack stability. Summary of the Invention

[0003] The purpose of this invention is to overcome the aforementioned defects or problems existing in the prior art or to provide a material basis for overcoming the aforementioned defects or problems existing in the prior art, and to provide an image stitching method and a defect detection method.

[0004] To achieve the above objectives, the present invention and its preferred embodiments employ the following technical solutions, but the embodiments are not limited to the following solutions: Option 1, an image stitching method, including... Acquire several detection images of the object under test located on a motion platform moving along a preset path, as well as the actual platform coordinates of the motion platform corresponding to each detection image acquisition. Each detection image and its corresponding actual platform coordinates are input into the geometric mapping model. Geometric transformation is performed on each detection image to obtain several images to be stitched together. The geometric mapping model is trained based on the actual platform coordinates of multiple motion platforms, the detection images of their corresponding standard workpieces, and the positions of feature points in the corresponding detection images in the global stitching coordinate system. The individual images to be stitched together are then stitched together to obtain the overall stitched image of the object under test.

[0005] Option 2, based on Option 1, involves latching the X / Y coordinates of the motion platform at each exposure trigger to obtain the actual platform coordinates for area scan camera flying shots; and for line scan or TDI camera scanning, using an encoder to trigger line acquisition at equal intervals and recording the actual platform position corresponding to the scan start point, scan direction, and line coordinates to obtain the actual platform coordinates.

[0006] Option 3, based on Option 1, involves periodic checks. A specific calibration board with shooting intervals matching the camera's field of view is used to score the stitching effect. If the score is below a threshold, recalibration is required to update the geometric mapping model.

[0007] Option 4, based on Option 1, adopts an adaptive fusion strategy based on position weight, image quality evaluation and residual optimization for overlapping areas of adjacent images to be stitched during the stitching process, so as to achieve seamless stitching of overlapping areas.

[0008] Option 5, based on Option 1, includes the following steps for training the geometric mapping model: Construct a telecentric imaging coordinate system, a motion platform coordinate system, and a global stitching coordinate system; The standard workpiece is placed on the motion platform, and the motion platform is controlled to move to multiple preset positions. At each position, a telecentric camera image is acquired as a standard pattern calibration image, and the actual platform coordinates of the motion platform are recorded at each image acquisition. The acquired standard pattern calibration image is processed to extract feature points; Based on the actual platform coordinates of multiple motion platforms and the global positions of feature points in the corresponding images, a geometric mapping model from the actual platform coordinates to the global coordinates of the puzzle is fitted.

[0009] Option 6, based on Option 5, considers one or more of the following compensations during fitting: X-direction pixel equivalent; Y-direction pixel equivalent; rotation error of the image coordinate system relative to the platform coordinate system; camera mounting skew error; XY motion axis non-orthogonality error; scanning direction error; trigger delay or position latching error; local residual compensation parameters; The basic attribute model of the geometric mapping model is:

[0010] Where X and Y are the actual coordinates of the platform; K x It is the pixel equivalent in the column direction / X direction, K y is the pixel equivalent in the row / Y direction; a and b represent the compensation coefficients in the row and column directions after the coordinate coupling errors of the motion platform orthogonal error, camera skew error, and rotation error of the image coordinate system relative to the platform coordinate system are superimposed; b1 and b2 are the offsets from mechanical coordinates to image coordinates; Cg and Rg are the global stitching coordinates.

[0011] Option 7, based on Option 5, uses a sub-pixel algorithm to obtain the feature point positions.

[0012] Option 8, based on Option 1, allows the geometric mapping model to be an affine model, a second-order model, a partitioned lookup table model, or a composite model combining an affine model and a residual LUT.

[0013] Option 9, a defect detection method, based on an image stitching method described in any one of Options 1 to 8, further includes... The overall stitched image is analyzed, and the defect detection results are output.

[0014] Option 10, based on Option 9, analyzes the overall stitched image and outputs one or more of the following: calibration residual, overlapping area residual, cumulative error in the X / Y direction, image sharpness distribution, effective image coverage, stitching seam location, and abnormal image alarm results.

[0015] As can be seen from the above description of the present invention and its preferred embodiments, compared with the prior art, the technical solution of the present invention and its preferred embodiments have the following beneficial effects due to the adoption of the following technical means: An image stitching method includes acquiring several detection images of an object under test located on a motion platform moving along a preset path, and the actual platform coordinates corresponding to the motion platform at the time of acquisition of each detection image; inputting each detection image and its corresponding actual platform coordinates into a geometric mapping model, performing geometric transformation on each detection image to obtain several images to be stitched, the geometric mapping model being trained based on the actual platform coordinates of multiple motion platforms, the detection images of corresponding standard workpieces, and the positions of feature points in the corresponding detection images in a global stitching coordinate system; stitching the images to be stitched together to obtain a complete stitched image of the object under test. This method eliminates the need to input or identify the feature point positions of the corresponding detection images; only the actual platform coordinates are recorded to output the images to be stitched for stitching. It can perform primary localization without relying on image feature matching and is suitable for stable stitching of transparent, weakly textured, low-contrast, and patternless samples. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments are briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the steps of an image stitching method in Embodiment 1; Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are preferred embodiments of the present invention and should not be considered as excluding other embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0018] Unless otherwise expressly defined, the use of terms such as "first," "second," or "third" in the claims, description, and accompanying drawings of this invention is for distinguishing different objects and not for describing a specific order.

[0019] Unless otherwise expressly defined, in the claims, description, and accompanying drawings of this invention, the use of directional terms such as "center," "lateral," "longitudinal," "horizontal," "vertical," "top," "bottom," "inner," "outer," "upper," "lower," "front," "rear," "left," "right," "clockwise," and "counterclockwise" to indicate orientation or positional relationships is based on the orientation and positional relationships shown in the accompanying drawings and is only for the convenience of describing the invention and simplifying the description, and is not intended to indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the specific scope of protection of this invention.

[0020] Unless otherwise expressly defined, the terms "fixed connection" or "fixed connection" used in the claims, description and drawings of this invention should be interpreted broadly to refer to any connection in which there is no displacement or relative rotation relationship between the two parties, including non-removable fixed connection, detachable fixed connection, integral connection and fixed connection by other means or components.

[0021] In the claims, description and accompanying drawings of this invention, the terms "comprising," "having," and variations thereof are used to mean "including but not limited to."

[0022] refer to Figure 1 An image stitching method, including S1: Perform position-synchronized image acquisition: Acquire several detection images of the object under test located on a motion platform moving along a preset path, as well as the actual platform coordinates corresponding to the acquisition of each detection image. Specifically, for the corresponding detection system, control the motion platform (such as an assembly line) to move along a preset path, and the telecentric camera (to achieve high-precision measurement) acquires images at a preset position or under preset trigger conditions. Preferably, for area scan camera shooting, the X / Y coordinates of the motion platform are latched at each exposure trigger to obtain the actual platform coordinates; for line scan or TDI camera scanning, the encoder triggers line acquisition at equal intervals, and records the actual platform position corresponding to the scanning start point, scanning direction, and line coordinates to obtain the actual platform coordinates. Obtaining the actual platform coordinates in the above manner reduces the mosaic position error caused by speed fluctuations, trigger delays, and acceleration / deceleration processes. By being compatible with area scan, line scan, and TDI scanning, the applicability is expanded to be suitable for different types of telecentric vision inspection equipment, so that each frame of detection image or each line of detection image establishes a correspondence with an actual platform position (i.e., the actual platform coordinates).

[0023] S2: Perform image geometric compensation - image correction: Input each detected image and its corresponding actual platform coordinates into the geometric mapping model, perform geometric transformation on each detected image to obtain several images to be stitched. The geometric mapping model is trained based on the actual platform coordinates of multiple motion platforms, the detected images of their corresponding standard workpieces, and the positions of feature points in the corresponding detected images in the global stitching coordinate system (the specific training process is explained below). By considering corresponding compensation when establishing the geometric mapping model from the actual platform coordinates to the global coordinates of the stitching, the platform step distance error, XY orthogonality error, camera installation angle error, and pixel equivalent difference can be compensated. Specifically, based on the actual platform coordinates and geometric mapping model corresponding to each frame of the detected image, the position of the detected image in the global coordinate system of the jigsaw puzzle is calculated. Translation, shearing, or full affine transformation is performed on the detected image to align its coordinates with the global coordinate system of the jigsaw puzzle, resulting in several images to be stitched together. This step achieves sub-pixel-level compensation, avoiding the systematic errors caused by traditional integer pixel tile stitching.

[0024] S3: Perform image fusion and large image generation: Stitch the images to be stitched together to obtain the overall stitched image of the object under test. Specifically, ① the images to be stitched are written into the global coordinate system of the stitching; ② for the overlapping areas of adjacent images to be stitched, an adaptive fusion strategy based on position weight, image quality evaluation, and residual optimization is adopted to achieve seamless stitching of the overlapping areas. In the image fusion process of this system, no interpolation processing is actually performed on the overlapping images. The main reason is that although the image models of this system overlap, no seams are generated. During stitching, only radial changes such as cropping and translation are performed according to the overlap distance set by the compensation model and the actual setting. No additional processing through interpolation is required to prevent interpolation from introducing new measurement errors, and secondly, to save performance overhead. ② Finally, the entire stitched image is generated.

[0025] A defect detection method, S4: Based on the image stitching method described above, output the stitching quality evaluation result of the entire stitched image: analyze the overall stitched image and output the defect detection result. The system outputs the entire stitched image and simultaneously outputs stitching quality evaluation data, which may include: ① calibration residual; ② overlapping area residual; ③ cumulative error in the X / Y directions; ④ image sharpness distribution; ⑤ effective image coverage; ⑥ stitching seam location; ⑦ abnormal image alarm result. By introducing the overlapping area residual and calibration residual evaluation mechanism, the stitching quality can be quantified and used for geometric model iterative optimization.

[0026] During use, the stitching effect is scored by a specific calibration board with shooting spacing that matches the camera's field of view through regular checks. If the score is lower than the threshold, recalibration is required to update the geometric mapping model.

[0027] This method enables product inspection without relying on image feature matching for primary localization, and is suitable for stable splicing of transparent, weakly textured, low-contrast, and patternless samples.

[0028] The principle of training the geometric mapping model is to construct a multi-dimensional collaborative spatial geometric coordinate mapping model: This scheme no longer relies on the traditional multi-camera calibration plate feature point matching, but instead jointly models the motion platform coordinate system of the motion platform and the telecentric imaging coordinate system of the telecentric imaging system. After coupling the two coordinate systems, the global row and column coordinates of the mosaic are obtained. Furthermore, the row and column coordinates after the sub-pixel level affine transformation matching correction of the image are established, and the relationship between the coordinates and the actual sample coordinates is restored with high precision, realizing the physical space coordinate binding mechanism.

[0029] The steps for training a geometric mapping model include: ① Construct the system coordinate system: including the telecentric imaging coordinate system, the motion platform coordinate system, and the global stitching coordinate system; The telecentric imaging coordinate system is based on the inherent geometric characteristics of the telecentric lens's object-side equal magnification and orthophoto projection. Through an intrinsic parameter model obtained from telecentric lens calibration, and by analyzing calibration parameters and performing coordinate transformations, a quantitative mapping relationship between image pixel coordinates and physical space coordinates is constructed. In other words, a linear mapping relationship between image space and object space is established.

[0030] The motion platform coordinate system achieves nanosecond-level hard synchronization between the image acquisition trigger pulse and the actual physical coordinates of the platform by introducing hardware instantaneous position latching technology, thereby obtaining the absolute reference during the motion process; Global stitching coordinate system: A spatial geometric error compensation lookup table is constructed by establishing a spatial geometric error compensation model between the motion platform coordinates, telecentric imaging coordinates, and global image coordinates of the stitching. Pre-calibrated row and column compensation coefficients are used to correct the mechanical coordinates, and the global stitching coordinate mapping is completed based on the corrected coordinates. Each single-frame image is mapped to the global stitching coordinate system in real time through nonlinear spatial geometric transformation, completely decoupling the systematic deviations caused by mechanical non-orthogonality, installation oblique cutting, and motion triggering delay, ensuring stitching accuracy reaches the sub-pixel level, and exhibiting extremely high stitching robustness to weak textures, transparent, and patternless targets. The global coordinate system uses the actual platform coordinates corresponding to the first frame image as the reference origin. The platform's actual coordinates after error compensation for each image are converted into global pixel coordinates, and the stitching canvas size is dynamically calculated based on the coordinate range of all images. Each image is then written onto this canvas, ultimately generating the global stitched image.

[0031] ② Acquire standard pattern calibration images: Place the standard workpiece on the motion platform and control the platform to move to multiple preset positions. At each position, acquire a telecentric camera image as the standard pattern calibration image, and record the actual platform coordinates corresponding to each image acquisition. For example, place a checkerboard calibration plate, a dot array calibration plate, or other standard workpieces with known geometric features on the motion platform. The actual platform coordinates can be obtained from encoder, grating ruler, motion controller position feedback, or position latching module. Extract sub-pixel feature points from the standard pattern to establish a geometric model, improving the calibration accuracy and long-term repeatability of the jigsaw puzzle model.

[0032] ③ Extracting sub-pixel feature points: The acquired standard pattern calibration image is processed to extract feature points; specifically, corner points, circle centers, line segment intersections, edge points, or other geometric feature points can be extracted. The location of feature points is obtained using a sub-pixel algorithm to improve the fitting accuracy of the geometric mapping model.

[0033] ④ Establish a geometric mapping model from the actual platform coordinates to the global coordinates of the puzzle: Based on the actual platform coordinates of multiple motion platforms and the global positions of feature points in the corresponding images, fit a geometric mapping model from the actual platform coordinates to the global coordinates of the puzzle. During fitting, consider one or more of the following compensations: ① X-direction pixel equivalent; ② Y-direction pixel equivalent; ③ Rotation error of the telecentric imaging coordinate system relative to the motion platform coordinate system; ④ Camera mounting oblique error; ⑤ XY motion axis non-orthogonality error; ⑥ Scanning direction error; ⑦ Trigger delay or position latching error; ⑧ Local residual compensation parameters. The geometric mapping model can be an affine model, a second-order model, a partition lookup table model, or a composite model combining an affine model and a residual LUT.

[0034] The basic attribute model of the geometric mapping model is:

[0035] Where X and Y are the actual coordinates of the platform; K x It is the pixel equivalent in the column direction / X direction, which can characterize ⑥ / ① in the above compensation, K y is the pixel equivalent in the row / Y direction, which can represent ⑥ / ② in the above compensation; a and b represent the compensation coefficients in the row and column directions after the coordinate coupling errors of the motion platform orthogonal error, camera skew error and the rotation error of the image coordinate system relative to the platform coordinate system are superimposed, which can represent ③④⑤ in the above compensation; b1 and b2 are the offsets from mechanical coordinates to image coordinates, which can represent ⑦⑧ in the above compensation; Cg and Rg are global stitching coordinates; finally, a geometric mapping model with corresponding compensation is obtained through training.

[0036] Compared to existing technologies, it has the following beneficial effects: An image stitching method includes acquiring several detection images of an object under test located on a motion platform moving along a preset path, and the actual platform coordinates corresponding to the motion platform at the time of acquisition of each detection image; inputting each detection image and its corresponding actual platform coordinates into a geometric mapping model, performing geometric transformation on each detection image to obtain several images to be stitched together, the geometric mapping model being trained based on the actual platform coordinates of multiple motion platforms, the detection images of corresponding standard workpieces, and the positions of feature points in the corresponding detection images in the global stitching coordinate system; stitching the images to be stitched together to obtain a complete stitched image of the object under test. This method allows for primary localization without relying on image feature matching and is suitable for stable stitching of transparent, weakly textured, low-contrast, and patternless samples.

[0037] The descriptions of the foregoing specification and embodiments are used to explain the scope of protection of the present invention, but do not constitute a limitation on the scope of protection of the present invention. Modifications, equivalent substitutions, or other improvements to the embodiments of the present invention or some of its technical features that can be obtained by those skilled in the art through logical analysis, reasoning, or limited experimentation, based on the teachings of the present invention or the foregoing embodiments, in conjunction with common general knowledge, ordinary technical knowledge in the art, and / or existing technology, should all be included within the scope of protection of the present invention.

Claims

1. An image stitching method, characterized in that: include Acquire several detection images of the object under test located on a motion platform moving along a preset path, as well as the actual platform coordinates of the motion platform corresponding to each detection image acquisition. Each detection image and its corresponding actual platform coordinates are input into the geometric mapping model. Geometric transformation is performed on each detection image to obtain several images to be stitched together. The geometric mapping model is trained based on the actual platform coordinates of multiple motion platforms, the detection images of their corresponding standard workpieces, and the positions of feature points in the corresponding detection images in the global stitching coordinate system. The individual images to be stitched together are then stitched together to obtain the overall stitched image of the object under test.

2. The image stitching method as described in claim 1, characterized in that: For area scan camera shooting, the X / Y coordinates of the motion platform are latched at each exposure trigger to obtain the actual platform coordinates; for line scan or TDI camera scanning, the encoder triggers line acquisition at equal intervals and records the actual platform position corresponding to the scan start point, scan direction and line coordinates to obtain the actual platform coordinates.

3. The image stitching method as described in claim 1, characterized in that: Regular checks are performed, and a specific calibration board with shooting spacing matching the camera's field of view is used to score the stitching effect. If the score is lower than the threshold, recalibration is required to update the geometric mapping model.

4. The image stitching method as described in claim 1, characterized in that: During the stitching process, for the overlapping areas of adjacent images to be stitched, an adaptive fusion strategy based on position weights, image quality evaluation, and residual optimization is adopted to achieve seamless stitching of the overlapping areas.

5. The image stitching method as described in claim 1, characterized in that: The steps for training a geometric mapping model include: Construct a telecentric imaging coordinate system, a motion platform coordinate system, and a global stitching coordinate system; The standard workpiece is placed on the motion platform, and the motion platform is controlled to move to multiple preset positions. At each position, a telecentric camera image is acquired as a standard pattern calibration image, and the actual platform coordinates of the motion platform are recorded at each image acquisition. The acquired standard pattern calibration image is processed to extract feature points; Based on the actual platform coordinates of multiple motion platforms and the global positions of feature points in the corresponding images, a geometric mapping model from the actual platform coordinates to the global coordinates of the puzzle is fitted.

6. The image stitching method as described in claim 5, characterized in that: During fitting, consider one or more of the following compensations: pixel equivalent in the X direction; pixel equivalent in the Y direction; rotation error of the telecentric imaging coordinate system relative to the motion platform coordinate system; camera mounting oblique error; XY motion axis non-orthogonality error; scan direction error; trigger delay or position latching error; local residual compensation parameters; The basic attribute model of the geometric mapping model is: Where X and Y are the actual coordinates of the platform; K x It is the pixel equivalent in the column direction / X direction, K y is the pixel equivalent in the row / Y direction; a and b represent the compensation coefficients in the row and column directions after the coordinate coupling errors of the motion platform orthogonal error, camera skew error, and image coordinate system rotation error relative to platform coordinate system are superimposed; b1 and b2 are the offsets from mechanical coordinates to image coordinates; Cg and Rg are the global stitching coordinates.

7. The image stitching method as described in claim 5, characterized in that: The location of feature points is obtained using a sub-pixel algorithm.

8. The image stitching method as described in claim 1, characterized in that: The geometric mapping model can be an affine model, a second-order model, a partition lookup table model, or a composite model combining an affine model and a residual LUT.

9. A defect detection method, characterized in that: Based on the image stitching method according to any one of claims 1-8, it further includes The overall stitched image is analyzed, and the defect detection results are output.

10. A defect detection method as described in claim 9, characterized in that: The system analyzes the overall stitched image and outputs one or more of the following: calibration residual, overlapping area residual, cumulative error in the X / Y direction, image sharpness distribution, effective image coverage, stitching seam location, and abnormal image alarm results.

Images