Light and dark field image automatic alignment method and system for WireBond AOI aerial photography, electronic equipment and computer readable medium
By acquiring reference images of bright and dark fields during the Wire Bond process, constructing feature templates, and calculating affine transformation matrices, the image inconsistency problem during the AOI equipment's aerial imaging process was solved, achieving stable alignment of bright and dark field images and a unified coordinate system, thus reducing maintenance costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MATFRON (SHANGHAI) SEMICON TECH CO LTD
- Filing Date
- 2026-03-06
- Publication Date
- 2026-05-08
AI Technical Summary
In the Wire Bond process, the AOI equipment suffers from issues such as camera exposure and platform movement timing shifts during aerial imaging, inconsistent imaging between bright and dark light sources, and the inability of a unified compensation model based on mechanical position to resolve pixel shifts caused by differences in light sources, resulting in inconsistent images.
Acquire bright and dark field reference images, establish a common reference coordinate system, construct bright and dark field feature templates, and achieve automatic alignment of bright and dark field images through gradient feature matching and affine transformation matrix calculation.
Stable alignment of bright and dark field images during high-speed aerial photography reduces maintenance costs, provides a unified coordinate basis, and offers a reliable basis for subsequent detection.
Smart Images
Figure CN121998968A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of machine vision and semiconductor packaging inspection (AOI) technology, and particularly to an automatic alignment method and system for bright and dark field images for WireBond AOI aerial photography, as well as electronic devices and computer-readable media. Background Technology
[0002] In the wire bonding process, AOI equipment needs to simultaneously detect different types of defects such as gold wire curvature, wire collapse, short circuit, contamination, scratches, and edge chipping. To improve production cycle time, the camera typically adopts a high-speed flying imaging method that moves synchronously with the motion platform, continuously acquiring multiple frames of images without interruption.
[0003] However, the following problems exist in the aerial imaging process: 1. There is a time shift between camera exposure and platform movement. Due to factors such as motion speed, trigger delay, and vibration, the imaging center of different frames of images may have unpredictable slight shifts, directly leading to image inconsistencies.
[0004] 2. Light sources in bright and dark fields cannot be imaged synchronously, resulting in completely different image characteristics. Bright-field images are suitable for gold wire geometry; Dark field images are suitable for surface scratches, chipped edges, and other defects.
[0005] The two images differ greatly in structural features such as texture, brightness, and gradient direction, making it difficult to directly match the bright field image with the dark field image.
[0006] 3. A unified compensation model based on mechanical position cannot solve the pixel shift caused by differences in light source. Differences in light source angle, reflection path, and exposure time can cause additional displacement, which mechanical encoders cannot reflect.
[0007] Traditional image registration methods (such as gray-level cross-correlation, optical flow, and single-template matching) cannot simultaneously align bright and dark field images under aerial photography conditions. Therefore, there is an urgent need for a precise method that can align the bright and dark fields separately while ensuring that the aligned images share the same coordinate system. Summary of the Invention
[0008] According to a first aspect of the present invention, an automatic alignment method for bright and dark field images for WireBond AOI aerial photography is provided, comprising the following steps: Before the aerial photography begins, acquire bright-field reference images and dark-field reference images, and establish a shared common light field reference coordinate system for both; Based on the bright-field reference image and the dark-field reference image, a bright-field feature template and a dark-field feature template are constructed respectively. During the aerial photography process, bright-field and dark-field aerial photography images are acquired, and their gradient features are extracted to obtain the corresponding set of edge points. The feature point sets of the bright-field and dark-field aerial images are matched with the corresponding templates, the orientation consistency and spatial consistency scores are calculated, and the optimal affine transformation matrix for each is solved. Based on the fixed light field offset matrix between reference images, combined with the optimal affine transformation matrix, the final transformation matrix for aligning the dark field aerial image to the bright field coordinate system is calculated. The final transformation matrix is applied to the bright-field and dark-field images respectively, and the aligned bright-field and dark-field image pairs are output.
[0009] Furthermore, constructing the bright-field feature template and the dark-field feature template involves the following steps: Extract the training region and the shielded region to determine the effective template region; Calculate the gradient magnitude and direction within the effective template region, and extract edge points; Construct an image pyramid to generate bright-field and dark-field feature templates.
[0010] Furthermore, the directional consistency score is calculated based on the gradient direction angle between the template point and the flying point; the spatial consistency score is calculated based on the spatial error after affine transformation mapping; and the comprehensive score is a weighted sum of the two.
[0011] Furthermore, the fixed light field offset matrix is pre-calibrated based on the spatial transformation relationship between the bright field reference image and the dark field reference image.
[0012] According to a second aspect of the present invention, an automatic image alignment system for bright and dark fields in WireBond AOI aerial photography is provided, comprising: A reference module is established to acquire bright-field and dark-field reference images before the aerial photography begins, and to establish a common light field reference coordinate system shared by the two. The template construction module is used to construct bright-field feature templates and dark-field feature templates based on bright-field reference images and dark-field reference images, respectively. The acquisition and extraction module is used to acquire bright-field and dark-field aerial images during the aerial photography process, and extract their gradient features to obtain the corresponding edge point set. The calculation and solution module is used to match the feature point sets of bright-field and dark-field aerial images with the corresponding templates, calculate the directional consistency and spatial consistency scores, and solve for the optimal affine transformation matrix for each. The matrix calculation module is used to calculate the final transformation matrix for aligning a dark field image to the bright field coordinate system based on a fixed light field offset matrix between reference images and the optimal affine transformation matrix. The image output module applies the final transformation matrix to the bright-field and dark-field images respectively, and outputs aligned bright-field and dark-field image pairs.
[0013] Furthermore, constructing the bright-field feature template and the dark-field feature template involves the following steps: Extract the training region and the shielded region to determine the effective template region; Calculate the gradient magnitude and direction within the effective template region, and extract edge points; Construct an image pyramid to generate bright-field and dark-field feature templates.
[0014] Furthermore, the directional consistency score is calculated based on the gradient direction angle between the template point and the flying point; the spatial consistency score is calculated based on the spatial error after affine transformation mapping; and the comprehensive score is a weighted sum of the two.
[0015] Furthermore, the fixed light field offset matrix is pre-calibrated based on the spatial transformation relationship between the bright field reference image and the dark field reference image.
[0016] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and the processor executes the computer program to perform an automatic alignment method for bright and dark field images for WireBond AOI aerial photography, as described in the first aspect.
[0017] According to a fourth aspect of the present invention, a computer-readable medium having processor-executable non-volatile program code is provided, characterized in that the program code causes the processor to run a method for automatic alignment of bright and dark field images for WireBondAOI aerial photography according to the first aspect.
[0018] An automatic alignment method for bright and dark field images for WireBond AOI aerial photography according to an embodiment of the present invention has the following beneficial effects: 1. Separate templates are constructed for bright field and dark field to prevent mutual interference and effectively solve the problem of inconsistent features caused by differences in light field.
[0019] 2. It can maintain stable alignment during high-speed aerial photography, and the pyramid template structure enables stable recognition of blurred images.
[0020] 3. The geometric relationship between bright and dark fields can be calibrated once and used for a lifetime, significantly reducing maintenance costs.
[0021] 4. The final output bright and dark field images are perfectly aligned, providing a unified coordinate basis for subsequent gold line height detection, scratch detection, and contamination detection.
[0022] 5. It does not rely on any specific hardware or software interface, can be implemented across platforms, conforms to general machine vision principles, and has greater value for patent protection.
[0023] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description
[0024] Figure 1 This is a flowchart of an automatic alignment method for bright and dark field images for WireBond AOI aerial photography according to an embodiment of the present invention.
[0025] Figure 2 This is a structural diagram of an automatic image alignment system for WireBond AOI aerial photography according to an embodiment of the present invention.
[0026] Figure 3 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0027] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, further illustrating the present invention.
[0028] First, combine Figure 1 This invention describes an automatic alignment method for bright and dark field images for WireBond AOI aerial photography, which has a wide range of applications.
[0029] like Figure 1 As shown in the figure, an automatic alignment method for bright and dark field images for WireBond AOI aerial photography according to an embodiment of the present invention includes the following steps: like Figure 1 As shown, in S1, before the aerial photography begins, a bright-field reference image and a dark-field reference image are acquired, and a shared light field reference coordinate system is established between the two. The bright-field reference image is: Dark field reference image: Common reference coordinate system for light field: All subsequent images must be aligned to this coordinate system.
[0030] like Figure 1 As shown, in S2, a bright-field feature template and a dark-field feature template are constructed based on the bright-field reference image and the dark-field reference image, respectively. The bright-field feature template is as follows: Dark field feature template: Constructing bright-field and dark-field feature templates involves the following steps: Extract the training region and the masked region to determine the effective template region; the training region is denoted as... The shielded area is denoted as The valid template area is: .
[0031] Calculate the gradient magnitude and direction within the effective template region, and extract edge points; specifically: within the effective template region... Internally calculate the gradient for each pixel: ; in, , For Sobel or other edge operators, use the convolution kernel.
[0032] The gradient magnitude and direction are: ; Take the set of edge points: ; Construct an image pyramid to generate bright-field and dark-field feature templates. Specifically: Using image pyramids: ; The corresponding bright-field feature template is: ; Similarly, the dark field feature template is obtained as follows: ; like Figure 1 As shown in S3, during the aerial photography process, bright-field and dark-field aerial images are acquired, and their gradient features are extracted to obtain the corresponding edge point sets. The bright-field aerial image is denoted as: Dark-field aerial photographs are denoted as: Perform the same gradient feature extraction as in step S2 on both the bright-field and dark-field images to obtain the set of edge points: .
[0033] like Figure 1 As shown, in S4, the feature point sets of the bright-field and dark-field aerial images are matched with their corresponding templates, and directional consistency and spatial consistency scores are calculated to solve for their respective optimal affine transformation matrices. The directional consistency score is calculated based on the gradient direction angle between the template point and the aerial image point; the spatial consistency score is calculated based on the spatial error after the affine transformation mapping; the comprehensive score is a weighted sum of the two. Specifically: Set of feature points for aerial photography With template collection Perform a match.
[0034] Directional consistency score: For any template point =( ) and flying camera points =( ), Direction matching score: Spatial consistency score: Suppose there exists an affine transformation matrix H such that the template point is mapped to the flying point: Spatial consistency score: Overall rating: Take the highest score among all candidate transformations .
[0035] Brightfield optimal transformation matrix: Optimal transformation matrix for dark field: like Figure 1 As shown, in S5, based on the fixed light field offset matrix between the reference images and combined with the optimal affine transformation matrix, the final transformation matrix for aligning the dark-field image to the bright-field coordinate system is calculated. The fixed light field offset matrix is pre-calibrated based on the spatial transformation relationship between the bright-field and dark-field reference images. Specifically: Since the bright field and dark field reference images are in the same coordinate system, there is a reference transformation: in "Fixed light field offset matrix" for two light source imaging systems.
[0036] Therefore, the final transformation for aligning a dark-field image to a bright-field coordinate system is: The alignment transformation of the bright-field image taken by aerial photography is as follows: After the two are aligned, they will share the same coordinate system. .
[0037] like Figure 1 As shown, in S6, the final transformation matrix is applied to both the bright-field and dark-field images, outputting aligned bright-field and dark-field image pairs. Specifically: Perform the following on the aerial photograph: The final result is an automatically aligned pair of light and dark field images: Used for subsequent gold thread detection, edge chipping detection, scratch detection, etc.
[0038] As described above, the automatic alignment method for bright and dark field images for WireBond AOI aerial photography according to an embodiment of the present invention has the following beneficial effects: 1. Separate templates are constructed for bright field and dark field to prevent mutual interference and effectively solve the problem of inconsistent features caused by differences in light field.
[0039] 2. It can maintain stable alignment during high-speed aerial photography, and the pyramid template structure enables stable recognition of blurred images.
[0040] 3. The geometric relationship between bright and dark fields can be calibrated once and used for a lifetime, significantly reducing maintenance costs.
[0041] 4. The final output bright and dark field images are perfectly aligned, providing a unified coordinate basis for subsequent gold line height detection, scratch detection, and contamination detection.
[0042] 5. It does not rely on any specific hardware or software interface, can be implemented across platforms, conforms to general machine vision principles, and has greater value for patent protection.
[0043] The above combined with the appendix Figure 1 An automatic bright-dark field image alignment method for WireBond AOI aerial photography according to an embodiment of the present invention is described. Furthermore, the present invention can also be applied to an automatic bright-dark field image alignment system for WireBond AOI aerial photography.
[0044] like Figure 2 As shown, according to a second aspect of the present invention, an automatic image alignment system for bright and dark fields in WireBond AOI aerial photography is provided, comprising: A reference module 100 is established to acquire bright-field reference images and dark-field reference images before the aerial photography begins, and to establish a common light field reference coordinate system shared by the two.
[0045] The template construction module 200 is used to construct a bright-field feature template and a dark-field feature template based on a bright-field reference image and a dark-field reference image, respectively. The construction of the bright-field feature template and the dark-field feature template includes the following steps: extracting the training region and the masking region, and determining the effective template region; calculating the gradient magnitude and direction within the effective template region, and extracting edge points; constructing an image pyramid, and generating the bright-field feature template and the dark-field feature template.
[0046] The acquisition and extraction module 300 is used to acquire bright-field and dark-field aerial images during the aerial photography process, and extract their gradient features to obtain the corresponding edge point set.
[0047] The calculation and solution module 400 is used to match the feature point sets of bright-field and dark-field aerial images with their corresponding templates, calculate directional consistency and spatial consistency scores, and solve for the optimal affine transformation matrix for each. The directional consistency score is calculated based on the gradient direction angle between the template point and the aerial image point; the spatial consistency score is calculated based on the spatial error after the affine transformation mapping; the comprehensive score is a weighted sum of the two.
[0048] The matrix calculation module 500 is used to calculate the final transformation matrix for aligning a dark-field image to the bright-field coordinate system, based on a fixed light field offset matrix between reference images and combined with the optimal affine transformation matrix. The fixed light field offset matrix is pre-calibrated based on the spatial transformation relationship between the bright-field and dark-field reference images.
[0049] The image output module 600 is used to apply the final transformation matrix to the bright-field and dark-field images respectively, and output the aligned bright-field and dark-field image pairs.
[0050] The above combined with the appendix Figure 2 This paper describes an automatic bright-dark field image alignment system for WireBond AOI aerial photography according to an embodiment of the present invention. It is a system expression corresponding to the first aspect of the present invention, an automatic bright-dark field image alignment method for WireBond AOI aerial photography; therefore, the details therein are not repeated, as they are the same as the method. Furthermore, the present invention can also be applied to an electronic device.
[0051] like Figure 3 As shown, according to a third aspect of the present invention, an electronic device is provided, comprising: a memory 1, a processor 2, and a computer program 3, wherein the computer program 3 is stored in the memory 1, and the processor 2 executes the computer program 3 to perform an automatic alignment method for bright and dark field images for WireBond AOI aerial photography according to the first aspect.
[0052] According to a fourth aspect of the present invention, a computer-readable medium having processor-executable non-volatile program code is provided, characterized in that the program code causes the processor to run a method for automatic alignment of bright and dark field images for WireBondAOI aerial photography according to the first aspect.
[0053] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of a computer program from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the ASIC can reside within a device. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. The present invention also provides a program product comprising executable instructions stored in the readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the automatic alignment method for bright and dark field images for WireBondAOI aerial photography provided in the various embodiments described above. In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0054] It should be noted that, in this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0055] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.
Claims
1. A method for automatic alignment of bright and dark field images for WireBond AOI aerial photography, characterized in that, It includes the following steps: Before the aerial photography begins, acquire bright-field reference images and dark-field reference images, and establish a shared common light field reference coordinate system for both; Based on the bright-field reference image and the dark-field reference image, a bright-field feature template and a dark-field feature template are constructed respectively; During the aerial photography process, bright-field and dark-field aerial photography images are acquired, and their gradient features are extracted to obtain the corresponding set of edge points. The feature point sets of the bright-field and dark-field aerial images are matched with the corresponding templates, the orientation consistency and spatial consistency scores are calculated, and the optimal affine transformation matrix for each is solved. Based on the fixed light field offset matrix between reference images, and combined with the optimal affine transformation matrix, the final transformation matrix for aligning the dark field aerial image to the bright field coordinate system is calculated. The final transformation matrix is applied to the bright-field and dark-field images respectively, and the aligned bright-field and dark-field image pairs are output.
2. The method for automatic alignment of bright and dark field images for WireBond AOI aerial photography as described in claim 1, characterized in that, The construction of the bright-field feature template and the dark-field feature template includes the following steps: Extract the training region and the shielded region to determine the effective template region; Calculate the gradient magnitude and direction within the effective template region, and extract edge points; Construct an image pyramid to generate bright-field and dark-field feature templates.
3. The automatic alignment method for bright and dark field images for WireBond AOI aerial photography as described in claim 1, characterized in that, The directional consistency score is calculated based on the gradient direction angle between the template point and the flying point; the spatial consistency score is calculated based on the spatial error after affine transformation mapping; the comprehensive score is a weighted sum of the two.
4. The automatic alignment method for bright and dark field images for WireBond AOI aerial photography as described in claim 1, characterized in that, The fixed light field offset matrix is pre-calibrated based on the spatial transformation relationship between the bright field reference image and the dark field reference image.
5. An automatic image alignment system for bright and dark fields in WireBond AOI aerial photography, characterized in that, Include: A reference module is established to acquire bright-field and dark-field reference images before the aerial photography begins, and to establish a common light field reference coordinate system shared by the two. The template construction module is used to construct a bright-field feature template and a dark-field feature template based on the bright-field reference image and the dark-field reference image, respectively. The acquisition and extraction module is used to acquire bright-field and dark-field aerial images during the aerial photography process, and extract their gradient features to obtain the corresponding edge point set. The calculation and solution module is used to match the feature point sets of bright-field and dark-field aerial images with the corresponding templates, calculate the directional consistency and spatial consistency scores, and solve for the optimal affine transformation matrix for each. The matrix calculation module is used to calculate the final transformation matrix for aligning the dark field image to the bright field coordinate system based on the fixed light field offset matrix between reference images and the optimal affine transformation matrix. The image output module is used to apply the final transformation matrix to the bright-field and dark-field images respectively, and output aligned bright-field and dark-field image pairs.
6. The automatic image alignment system for bright and dark fields used in WireBond AOI aerial photography as described in claim 5, characterized in that, The construction of the bright-field feature template and the dark-field feature template includes the following steps: Extract the training region and the shielded region to determine the effective template region; Calculate the gradient magnitude and direction within the effective template region, and extract edge points; Construct an image pyramid to generate bright-field and dark-field feature templates.
7. The automatic image alignment system for bright and dark fields used in WireBond AOI aerial photography as described in claim 5, characterized in that, The directional consistency score is calculated based on the gradient direction angle between the template point and the flying point; the spatial consistency score is calculated based on the spatial error after affine transformation mapping; the comprehensive score is a weighted sum of the two.
8. The automatic image alignment system for bright and dark fields used in WireBond AOI aerial photography as described in claim 5, characterized in that, The fixed light field offset matrix is pre-calibrated based on the spatial transformation relationship between the bright field reference image and the dark field reference image.
9. An electronic device, characterized in that, include: The device includes a memory, a processor, and a computer program, wherein the computer program is stored in the memory and the processor executes the computer program to perform the automatic alignment method for bright and dark field images for WireBond AOI aerial photography as described in any one of claims 1 to 4.
10. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the automatic alignment method for bright and dark field images for WireBond AOI aerial photography as described in any one of claims 1-4.