Mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging

By using a polarization-splitting pseudo-TDI imaging system to achieve dual positioning of components and pads, the problems of mounting deviation and insufficient image quality caused by pad position deviation are solved, thereby improving positioning accuracy and production efficiency, and reducing costs and light source burden.

CN123118772APending Publication Date: 2026-11-17HARBIN INST OF TECH +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611512139.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-28
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In traditional surface mount technology (SMT) processes, pad position deviations can lead to mounting offsets, bridging, and cold solder joint defects. Existing imaging systems and positioning algorithms cannot achieve high-precision positioning in complex backgrounds, and are either costly or inefficient.

Method used

A polarization-splitting pseudo-TDI imaging system is adopted, including a polarization-splitting dual-array pseudo-TDI imaging module, a positioning module, and a pose calculation and mounting compensation module. By combining polarization illumination, a beam splitting component, and an array camera, dual positioning of components and pads is achieved. Combined with synchronous control and robust processing, the image signal-to-noise ratio and positioning accuracy are improved.

Benefits of technology

It reduces system costs and light source heat dissipation burden, improves positioning efficiency and accuracy, enhances robustness to complex backgrounds, ensures image quality and placement accuracy, and improves the automation level and yield of placement production.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118772A_ABST
    Figure CN123118772A_ABST
Patent Text Reader

Abstract

This application relates to a secondary positioning system for component placement based on polarization-splitting pseudo-TDI imaging, and pertains to the fields of electronic assembly and semiconductor packaging. The aim of this application is to address the problems of placement deviations caused by preset pad positions in traditional processes, insufficient image quality from aerial photography of moving components, and poor robustness of single-positioning to complex backgrounds and pad defects. This application employs a polarization-splitting dual-array pseudo-TDI imaging subsystem to perform aerial photography of components moving with the carrier mechanism, and a pad imaging module to photograph stationary pads. A positioning module performs coarse and fine positioning on the component and pad images. Coarse positioning determines the effective recognition area for the component and pad, while fine positioning determines the positioning information of the component and pad within the effective recognition area. A pose calculation and placement compensation module calculates the relative pose deviation between the component and pad based on the positioning information, and drives the placement execution mechanism to perform compensated placement based on the relative pose deviation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of electronic assembly and semiconductor packaging technology. Background Technology

[0002] In surface mount technology (SMT) and semiconductor die bonding packaging processes, after components are picked up by the mounting actuator, they need to be precisely aligned with target pads on the printed circuit board (PCB) substrate before they can be mounted. Traditional processes typically use preset pad positions, which are given in advance based on design coordinates or teach coordinates. Only one visual positioning is performed on the picked-up component, and then the deviation is calculated according to the preset pad position before mounting. However, due to substrate expansion and contraction, warping, incoming material deviations, and clamping errors, the actual position of the pads often deviates from the preset position, resulting in defects such as mounting misalignment, bridging, and cold solder joints. Moreover, these deviations fluctuate with substrate batches and operating conditions and cannot be eliminated by a single teach point.

[0003] To eliminate deviations in the actual position of the solder pads, actual imaging and positioning of the pads is required. This involves introducing pad positioning in addition to component positioning, forming a secondary positioning process that separately positions both the component and the pad. This secondary positioning places dual demands on the imaging system and the positioning algorithm. On the one hand, because components move at high speeds with the mounting head and other transport mechanisms, aerial imaging requires the imaging system to obtain images with high signal-to-noise ratios and no motion blur within extremely short line exposure times. On the other hand, due to the varied shapes and complex surface conditions of the pads and components, and the influence of adverse factors such as material properties, lighting conditions, and image quality, images may exhibit blurred boundaries, multiple protrusions and burrs, contamination, and other defects. This necessitates that the positioning algorithm possess robust general processing capabilities for both components and pads.

[0004] Existing motion imaging solutions mostly employ high-frequency linear array cameras or time-delay integration (TDI) linear array cameras. These cameras and their associated linear array light sources are expensive, and the stringent requirements for brightness and uniformity of the light source result in high power consumption and heat dissipation burdens. If ordinary low-frame-rate area array cameras are used for flight photography, the limited frame rate and exposure time lead to insufficient image signal-to-noise ratio and sharpness, directly affecting positioning accuracy. Existing dual-camera alternating exposure solutions (such as Chinese patent CN106060354A) use alternating superposition of entire frames, which is pure time-division multiplexing, and ordinary beam splitting causes approximately 50% light energy loss, actually increasing the burden on the light source. Existing encoder synchronization solutions (such as Chinese patent CN121357408A) are only designed for single-line array cameras and cannot solve the problems of constant line spacing and dual-path image registration under speed fluctuations in dual-area array systems. This leads to unstable line spacing and dual-path image registration errors in dual-area array systems when the speed of the carrier mechanism fluctuates. Existing pad positioning methods mostly employ a single-shot positioning strategy. This strategy struggles to balance rapid searching across a large field of view with precise positioning in small areas, and lacks robustness against complex backgrounds and defects. Consequently, positioning accuracy and reliability decrease under complex background conditions and with pad defects. Furthermore, high-speed component flight photography requires high signal-to-noise ratio, motion-blur-free images within extremely short exposure times. However, existing low-cost area scan camera solutions are limited by frame rate and exposure time, resulting in insufficient image quality for flight photography of moving components. Due to the variable shapes and complex surface conditions of pads and components, and the influence of material properties, lighting conditions, and image quality, images are prone to blurred boundaries, protrusions, burrs, and damage. Existing positioning algorithms lack general robustness, leading to frequent failures or significant errors in precise component and pad positioning. Summary of the Invention

[0005] This application aims to address the issues of mounting deviation caused by preset pad positions in traditional processes, insufficient image quality of moving components during flight, and poor robustness of single positioning against complex backgrounds and pad defects. It provides a secondary mounting positioning system based on polarization-splitting pseudo-TDI imaging.

[0006] The first aspect of this application provides a secondary positioning system for mounting based on polarization-splitting pseudo-TDI imaging, including: a component imaging module, a pad imaging module, a positioning module, and a pose calculation and mounting compensation module.

[0007] The component imaging module is a polarization-splitting dual-array pseudo-TDI imaging subsystem, used to take aerial images of components that move with the carrier mechanism.

[0008] The pad imaging module is used to photograph stationary pads to obtain pad images;

[0009] The positioning module is used to perform coarse positioning and fine positioning on the component image and the pad image. The coarse positioning is used to determine the effective identification area where the component and the pad are located. The fine positioning is used to determine the positioning information of the component and the pad within the effective identification area. The positioning information includes center coordinates and geometric information.

[0010] The pose calculation and placement compensation module is used to calculate the relative pose deviation between the component and the pad based on the positioning information of the component and the pad, and drive the placement execution mechanism to perform compensation placement based on the relative pose deviation.

[0011] The polarization-splitting dual-array pseudo-TDI imaging subsystem includes:

[0012] An illumination assembly for providing the element with a light source having two orthogonal polarization directions;

[0013] A beam splitter is used to separate the imaging light returned by the element into two beams according to their polarization states, which are then received by two area array cameras respectively.

[0014] The image processing unit is used to align the images output by the two area array cameras along the row direction, and to integrate multiple exposures corresponding to the same spatial pixel column position by row to generate a TDI line array image.

[0015] The synchronization control unit is used to alternately trigger the two area array cameras to expose line by line according to the pulse signal of the encoder, so that the same spatial position on the element is exposed multiple times.

[0016] In one possible design, the lighting assembly includes: a lighting source and an array of polarizers covering its light-emitting surface;

[0017] The polarizer array includes two different polarizer halves spliced ​​side by side, so that the light emitted by the illumination source can form two mutually orthogonal polarization light sources. The dividing line of the two different polarizer halves is perpendicular to the movement direction of the transport mechanism. The two area array cameras respectively receive the imaging light of the field of view, and the two images are spliced ​​to obtain a complete element image.

[0018] The beam-splitting assembly includes: an imaging objective lens and a polarizing beam-splitting prism;

[0019] The imaging objective lens converges the imaging light returned by the element and then directs it onto the polarizing beam splitter.

[0020] The polarizing beam splitter reflects a polarized light source to one imaging arm, which is then received by an area array camera. The polarizing beam splitter transmits another polarized light source to another imaging arm, which is then received by another area array camera after passing through a wedge prism.

[0021] In one possible design, determining the effective identification area where the component and the pad are located includes:

[0022] Calculate the global grayscale segmentation threshold of the target image, and separate the target from the background based on the global grayscale segmentation threshold;

[0023] Remove the burrs and noise from the segmented target image and fill the holes to obtain the repaired image;

[0024] The boundary set of all targets in the repaired image is extracted using a boundary contour extraction algorithm;

[0025] Based on preset geometric feature parameters, the boundary that is closest to the target geometric feature is selected from the boundary set, and the area where it is located is taken as the effective identification area.

[0026] The target image is either the component image or the pad image.

[0027] In one possible design, determining the positioning information of the component and the pad within the effective identification area includes:

[0028] Within the effective identification area, a preset template is retrieved. Using the preset reference position coordinates on the target as the center, the preset template is translated by at least one unit in the vertical and / or horizontal direction to splice it, resulting in a spliced ​​template, the center of which is located on the target.

[0029] The image located within the stitching template is binarized to extract the target boundary set;

[0030] Calculate the center coordinates of the target boundary set, and use them as the center coordinates of the target;

[0031] Calculate the matching score between the target boundary set and the preset template, and confirm the geometric information of the target based on the matching score;

[0032] The target is the component or the pad.

[0033] In one possible design, calculating the matching score between the target boundary set and the preset template, and confirming the geometric information of the target based on the matching score, includes:

[0034] Obtain the grayscale value of each pixel in the overlapping region of the target boundary set and the preset template;

[0035] Calculate the matching score based on the grayscale value of each pixel in the overlapping region;

[0036] The geometric shape corresponding to the preset template with the highest matching score is determined as the geometric shape of the target.

[0037] The rotation angle of the target is calculated based on the coordinates of each pixel in the target boundary set;

[0038] The geometry, the boundary length of the target boundary set, and the rotation angle are used as the geometric information of the target.

[0039] In one possible design, the matching score is a normalized grayscale correlation coefficient, a normalized grayscale error sum of squares, or a weighted fusion value of the normalized grayscale correlation coefficient and the normalized grayscale error sum of squares.

[0040] In one possible design, the mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging further includes a closed-loop feedback module, which is used to calculate the deviation between the center coordinates of the target and the center of the effective identification area determined by the coarse positioning after the center coordinates of the target are output by the fine positioning. If the deviation exceeds a preset threshold, the effective identification area is corrected according to the deviation, and the coarse positioning and the fine positioning are re-executed until the deviation is less than the preset threshold or the maximum number of iterations is reached. The target is the component or the pad.

[0041] In one possible design, the mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging further includes a robustness processing module for detecting and repairing defects in the target and adaptively adjusting the matching score threshold according to the severity of the defects.

[0042] In one possible design, the mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging further includes an image preprocessing module for sequentially performing grayscale conversion, noise reduction, and enhancement processing on the component image and the pad image.

[0043] A second aspect of this application provides a computer device, including a memory, a processor, and a network interface that are communicatively connected to each other via a system bus; the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the above-described mounting secondary positioning method.

[0044] A third aspect of this application provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the above-described mounting secondary positioning method.

[0045] The beneficial effects of this application are:

[0046] 1. The imaging module of this application adopts a polarization-splitting dual-array pseudo-TDI imaging subsystem. The polarization illumination component provides the element with first polarization state illumination light and second polarization state illumination light with mutually orthogonal polarization directions. The polarization beam splitting component guides the first polarization state component and the second polarization state component in the imaging light returned by the element to the first imaging arm and the second imaging arm, respectively. The first array camera and the second array camera receive the two polarized lights respectively. Therefore, there is no beam splitting energy loss in the two imaging lights under their respective polarization states. The total power of the illumination source can be equal to that of the single-camera imaging scheme. At the same time, two low-cost array cameras are used to equivalently replace the high-frequency linear array camera or TDI linear array camera, thereby obtaining a high signal-to-noise ratio element image while reducing system cost and heat dissipation burden of the light source.

[0047] 2. The positioning module of this application adopts the same coarse and fine positioning processing flow for component images and pad images. The coarse positioning determines the effective recognition area by calculating the global grayscale segmentation threshold, removing burrs and noise, filling holes, extracting the boundary set, and filtering the boundary set according to preset geometric feature parameters. The fine positioning confirms the geometric information within the effective recognition area by retrieving the preset template, splicing it to form a splicing template centered on the target, binarizing and extracting the target boundary set, calculating the center coordinates of the target boundary set, and obtaining the matching degree score. Therefore, this positioning module is applicable to both component and pad targets. It can reduce the recognition area to save recognition time and ensure that the splicing template covers the target, thereby improving positioning efficiency and positioning accuracy.

[0048] 3. Since the synchronous control unit of this application is connected to the encoder of the carrier mechanism, the N times frequency of the encoder pulse is used as the row clock. The first array camera is triggered to expose on even-numbered row clock pulses and the second array camera is triggered to expose on odd-numbered row clock pulses, forming a checkerboard row-level time-slot time-division exposure. The image processing unit integrates the M exposures corresponding to the same spatial pixel column position by row. Therefore, the readout noise of each exposure is independent of each other. The useful signal is linearly accumulated by M while the noise is accumulated by the square root of M, thereby outputting an equivalent M-level TDI linear array image, which improves the image signal-to-noise ratio by the square root of M times for a single exposure.

[0049] 4. The FPGA synchronous control unit of this application calculates the instantaneous speed of the carrier mechanism in real time and dynamically adjusts the sub-row clock frequency division coefficient and the two-way trigger phase delay according to the instantaneous speed. Before stitching, the image processing unit performs sub-pixel translation pre-compensation on the image of the array camera B according to the speed and time difference. Therefore, a closed-loop control of speed feedback, frequency division adjustment, phase adjustment and pre-compensation is formed. Under the acceleration, deceleration and jitter conditions of the carrier mechanism, the imaging row spacing is kept constant and the dual-channel images are accurately registered, avoiding motion blur and image misalignment, and ensuring the image quality of flight shooting.

[0050] 5. Since the output module outputs the center coordinates and geometric information of the component and pad, the relative pose deviation, and the mounting result, which serve as teaching points and compensation data for the mounting equipment, this application can be directly embedded into existing mounting equipment such as pick-and-place machines and die bonders, thereby improving the automation level and yield of mounting production. Attached Figure Description

[0051] Figure 1 The above is a diagram showing the overall structure and production process of the mounting secondary positioning system described in the embodiment.

[0052] Figure 2 This is a schematic diagram of the optical path structure of the polarization-splitting dual-array pseudo-TDI imaging subsystem described in the embodiment.

[0053] Figure 3 This is a flowchart of the coarse localization (image segmentation) process described in the embodiment;

[0054] Figure 4 The flowchart for precise positioning (template matching) described in the embodiment is shown below;

[0055] Figure 5 This is a schematic diagram of the preset template splicing described in the embodiment;

[0056] Figure 6 This is a flowchart illustrating the pose calculation and mounting compensation process described in the embodiment.

[0057] Figure 7 This is a schematic diagram of the computer device structure described in the embodiment;

[0058] Reference numerals: 1. Target image; 2. Target; 21. Center coordinates; 3. Non-target; 4. Preset template; 41. Reference position coordinates; 5. Computer equipment; 51. Memory; 52. Processor; 53. Network interface; 100. Polarization-splitting double-sided array pseudo-TDI imaging subsystem; 101. Illumination source; 102. Polarizer array; 103. Component under test; 104. Carrier mechanism; 105. Encoder; 106. Imaging objective lens; 107. Polarization-splitting prism; 108. First Imaging arm; 109, Second imaging arm; 110, Wedge prism; 111, Area scan camera A; 112, Area scan camera B; 113, FPGA synchronization control unit; 114, Image processing unit; 120, Ordinary area scan camera; 130, Mounting actuator; 210, Image preprocessing module; 220, Coarse positioning module; 230, Fine positioning module; 240, Output module; 250, Closed-loop feedback module; 260, Robustness processing module; 270, Pose calculation and mounting compensation module. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0060] Specific Implementation Method 1: The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging described in this implementation method is as follows:

[0061] I. System Overall Architecture

[0062] like Figure 1 As shown, the mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging described in this application includes: a polarization-splitting dual-array pseudo-TDI imaging subsystem 100, a conventional area array camera 120, an image preprocessing module 210, a coarse positioning module 220, a fine positioning module 230, an output module 240, a closed-loop feedback module 250, a robustness processing module 260, a pose calculation and mounting compensation module 270, and a mounting execution mechanism 130.

[0063] The production process of the mounting secondary positioning system is as follows: the polarization-splitting dual-array pseudo-TDI imaging subsystem 100 takes aerial pictures of the tested component 103 that moves with the carrier mechanism 104, outputs component images, and completes the first positioning (component positioning) through the image preprocessing module 210, coarse positioning module 220 and fine positioning module 230; the ordinary area array camera 120 takes pictures of the stationary PCB pads, outputs pad images, and completes the second positioning (pad positioning) through the same set of modules (image preprocessing module 210, coarse positioning module 220 and fine positioning module 230); the pose calculation and mounting compensation module 270 calculates the relative pose deviation between the two objects and drives the mounting execution mechanism 130 to compensate for the mounting; the output module 240 outputs positioning and mounting data.

[0064] II. Polarization-Spectral Dual-Face Array Pseudo-TDI Imaging Subsystem

[0065] like Figure 2 As shown, the polarization beam splitting dual-array pseudo-TDI imaging subsystem 100 includes: an illumination source 101, a polarizer array 102, an imaging objective lens 106, a polarization beam splitting prism (PBS) 107, a first imaging arm 108, a second imaging arm 109, a wedge prism 110, an area array camera A111, an area array camera B112, an FPGA synchronization control unit 113, and an image processing unit 114.

[0066] The component under test 103 is placed on the transport mechanism 104 (the transport platform of the feeder or the mounting head) and moves in the direction shown in the figure. An encoder 105 is installed on the drive shaft of the transport mechanism 104, and the pulse signal output by the encoder 105 is connected to the FPGA synchronous control unit 113.

[0067] The illumination source 101 is an LED surface light source, with a polarizer array 102 covering its light-emitting surface. The polarizer array 102 is composed of a left half of S-polarizer and a right half of P-polarizer, so that the emitted illumination light is S-polarized in the left half of the field of view and P-polarized in the right half. The boundary between the left and right halves is perpendicular to the movement direction of the transport mechanism 104. The area array camera A111 receives the imaging light from the S-polarized field of view, and the area array camera B112 receives the imaging light from the P-polarized field of view. The image processing unit 114 stitches the two images together to obtain a complete image of the component. The imaging light returned by the component under test 103 is converged by the imaging objective lens 106 and then incident on the PBS 107. The PBS 107 reflects the S-polarized component into the first imaging arm 108, which is received by the area array camera A111; and transmits the P-polarized component into the second imaging arm 109, which is received by the area array camera B112 after passing through the wedge prism 110. Since neither the S nor P component experiences energy loss during polarization beam splitting, the total power of the illumination source 101 is comparable to that of a single-camera imaging scheme. In other embodiments, the polarizer array 102 can also be replaced by a timing polarization modulation device (such as a liquid crystal polarization rotator or an alternately lit S / P dual light source), which alternately emits S / P polarized light within adjacent illumination time slots, avoiding the decrease in field of view utilization caused by spatial segmentation.

[0068] The wedge prism 110 is inserted into the second imaging arm 109 (or the first imaging arm 108), and is made of K9 optical glass (refractive index 110). wedge angle Pick The deflection angle produced by the wedge prism The focal length of the imaging objective lens 106 After being converted to the image plane, the image captured by the area scan camera B112 is shifted relative to the area scan camera A111. (in pixels), during design according to ( Select a wedge angle (for the pixel size) so that fall into The preferred range of pixels. More generally, it can be found in... Within a pixel, selection is performed. The wedge direction of the wedge prism 110 is positioned along the movement direction of the transport mechanism 104, so that the misalignment direction is parallel to the movement direction. , , For example, , Pixel. This misalignment enables the two images to form sub-pixel interleaved sampling on the same object plane column, upgrading pure time-division multiplexing to "time-division + space-division" hybrid sampling; the image processing unit 114 can use this misalignment to perform dual-image sub-pixel super-resolution reconstruction, further improving the positioning accuracy of the element edge.

[0069] By inserting a wedge prism with a wedge angle of 0.5 to 2 degrees into any imaging arm, the images formed by the first array camera and the second array camera are offset by 0.1 to 0.9 pixels relative to each other along the direction of movement of the transport mechanism. Therefore, the two images form sub-pixel interleaved sampling on the same object plane column, upgrading pure time-division multiplexing to time-division and space-division hybrid sampling. The image processing unit can use this misalignment to perform dual-image sub-pixel super-resolution reconstruction, thereby improving the positioning accuracy of the element edge.

[0070] The FPGA synchronization control unit 113 multiplies the pulses output by the encoder 105 by N (in this embodiment). The line clock is obtained. The line clock pulses are grouped by parity: the first... traveling pulse The A111 area array camera was exposed, the first The horizontal pulse triggers the exposure of the area array camera B112. Both area array cameras operate in horizontal ROI readout mode, exposing and reading out only a few rows corresponding to the virtual rows with each trigger. The two virtual rows, A and B, are staggered along the motion direction on the object surface, forming a checkerboard-style row-level time-slot sampling. The image processing unit 114 aligns the two images in the horizontal direction (taking fixed misalignment into account). After that, the corresponding spatial pixel column positions will be... Subsequent exposures are scored row-wise: Output equivalent Level TDI linear array image. Because the readout noise of each exposure is independent, the signal is... Linear accumulation while noise is calculated according to... Accumulation improves the signal-to-noise ratio of the output image compared to a single exposure. times. It can be configured to 2, 4, 8 or 16 according to the positioning accuracy requirements.

[0071] The FPGA synchronous control unit 113 contains a frequency multiplication phase-locked loop, a speed estimation module, a frequency division coefficient lookup table, a dual-path trigger phase delay register, and a sub-pixel displacement calculation module. The speed estimation module performs a sliding window averaging (window length is 32 pulse intervals) on the encoder pulse intervals and combines it with a first-order low-pass filter to output the instantaneous speed of the transport mechanism in real time. The frequency division coefficient lookup table is based on... Output frequency division coefficient This ensures that the spatial spacing between adjacent virtual rows remains constant despite velocity fluctuations; the phase delay register is based on... Adjust the phase delay of trigger signals A and B. Before image stitching, image processing unit 114 presses... The subpixel displacement of the image from the B112 area array camera is calculated, and bicubic interpolation is used for subpixel translation pre-compensation. The compensation accuracy is better than 0.1 pixels, thus forming a complete closed loop of "velocity feedback → frequency division adjustment → phase adjustment → pre-compensation" to ensure that the images captured during flight under acceleration, deceleration and shaking conditions of the carrier mechanism are free from motion blur and misalignment.

[0072] III. Image Preprocessing

[0073] Retrieve the preset template 4, and, with reference position coordinates 41 as the center, translate it at least one unit in the vertical and / or horizontal directions to splice it, obtaining a spliced ​​template, wherein the center of the spliced ​​template is located on target 2. For example... Figure 5 As shown, with reference position coordinate 41 as the center, a preset template 4 is set at the first, second, third, and fourth quadrants of the cross coordinate system, respectively. The boundaries of the four preset templates 4 are spliced ​​together to form a spliced ​​template. One unit is the size of the preset template 4. The preset template 4 is selected to be a template type with a shape similar to that of the target 2. In this embodiment, the adaptive template splicing module dynamically adjusts the number and arrangement of preset templates 4 according to the effective recognition area size obtained from coarse positioning.

[0074] The image preprocessing module 210 is used to perform grayscale conversion, denoising, and enhancement processing on component images and pad images. Specifically, the image is linearly transformed to convert it to the YCrCb color space, and the luminance component is used as the grayscale information used in image processing; median filtering or Gaussian filtering is used to remove image noise; and histogram equalization is used to enhance image contrast.

[0075] IV. Coarse Positioning

[0076] like Figure 3 As shown, the coarse positioning module 220 is used to perform image segmentation on the target image to determine the effective recognition area where the target is located. The target is a component or pad, and the target image corresponds to a component image or pad image. Specifically, it includes the following steps:

[0077] Step S21: Calculate the global grayscale segmentation threshold of the target image, and process the target image using this threshold to separate the target from the background. In this embodiment, the threshold required for binarization is set using the Otsu automatic acquisition method, marking the pixels representing the target as 255 and the pixels representing the background as 0.

[0078] Step S22: Remove burr boundaries and noise from the target image and fill the holes in the target image. Specifically, perform morphological opening operations on the binarized image to remove burr boundaries and noise, further clearing to obtain smoother boundaries; finally, fill the holes in the image to obtain the repaired target area.

[0079] Step S23: Use a boundary contour extraction algorithm to extract the boundary set of all targets in the target image. In this embodiment, the boundary contour extraction algorithm used is the Zhang-Suen thinning algorithm. It distinguishes between edges and internal points by traversing the foreground points and their corresponding image neighborhoods to determine whether they meet specific judgment rules provided by the algorithm. After a certain number of iterations and convergence, it is thinned to a centerline formed by a single pixel width. Pixels within the same boundary set must all be at least 4-connected.

[0080] Step S24: Compare all boundary sets with the preset template, and select the boundary set whose geometric feature parameters are closest to the preset template as the target boundary set, and determine the region where it is located as the effective recognition region. The geometric feature parameters are selected from at least one of the following: the length of the boundary set, the central moments of each order, or the distances of each type of moment.

[0081] V. Precise Positioning

[0082] like Figure 4 As shown, the precision positioning module 230 is used to accurately identify and locate the target image within the effective recognition area. Specifically, it includes the following steps:

[0083] Step S31: Obtain reference position coordinates 41, which are located on target 2. In this embodiment, the reference position coordinates 41 are cross coordinates, pre-input by the user. Alternatively, the reference position coordinates 41 can be automatically generated by the center of the effective recognition area determined by the coarse positioning module 220.

[0084] Step S32: Binarize the image located within the stitching template and extract the target boundary set. Specifically, use a boundary contour extraction algorithm to extract the boundary set of all targets in the image within the stitching template; compare all boundary sets with the preset template 4, and take the boundary set whose geometric feature parameters are closest to the preset template 4 as the target boundary set.

[0085] Step S33: Extract the center coordinates of the target boundary set, which will be used as the center coordinates 21 of target 2. Specifically, obtain the coordinates of each pixel in the target boundary set; calculate the center coordinates of the target boundary set using the following formula:

[0086] ,

[0087] ,

[0088] in, The center coordinates of the target boundary set, These are the x-coordinates of each pixel. These are the ordinates of each pixel. This represents the number of pixels in the target boundary set. In this embodiment, Take the boundary length (zero-order moment) of the target boundary set.

[0089] Step S34: Obtain the matching score between the target boundary set and the preset template 4 to confirm the geometric information of target 2. Specifically, obtain the center coordinates of the preset template 4 and make the center coordinates of the target boundary set coincide with the center coordinates of the preset template 4; obtain the corresponding grayscale of each pixel in the overlapping area of ​​the target boundary set and the preset template 4; obtain the matching score based on the corresponding grayscale of each pixel and output the geometric information of target 2.

[0090] In this embodiment, the matching score is the normalized gray-scale correlation coefficient, calculated using the following formula:

[0091] ,

[0092] in, This is the normalized gray-level correlation coefficient. These represent the grayscale values ​​of each pixel in the preset template. This represents the average grayscale value of each pixel in the preset template. These are the grayscale values ​​of each pixel within the target boundary set. This represents the average grayscale value of each pixel within the target boundary set.

[0093] In other implementations, the matching score can also be the normalized sum of squared grayscale errors, calculated using the following formula:

[0094] ,

[0095] in, To normalize the sum of squared grayscale errors, These represent the grayscale values ​​of each pixel in the preset template. These are the grayscale values ​​of each pixel in the target boundary set.

[0096] In this embodiment, the normalized grayscale correlation coefficient and the normalized grayscale error sum of squares are weighted and fused to form a comprehensive matching score, which is calculated using the following formula:

[0097] ,

[0098] in, The overall matching score is calculated as follows: and All are weighted coefficients, and In this embodiment, Take 0.6, Take 0.4.

[0099] In this embodiment, the multi-scale template matching module uses multiple preset templates 4 of different scales in parallel to match the target boundary set, and takes the template corresponding to the one with the highest matching degree score as the best matching template, thereby automatically determining the optimal matching scale of the target.

[0100] Step S36: Output the geometric information of target 2. Specifically, based on the coordinates of each pixel in the target boundary set, the center coordinates, and the boundary length, obtain the rotation angle of the target boundary set; based on the matching score between the target boundary set and the preset template 4, obtain the geometric shape of the preset template 4 corresponding to target 2; output the geometric shape, boundary length, and rotation angle of the target boundary set as the geometric information of target 2. The rotation angle is calculated using the following formula:

[0101] ,

[0102] in, Let the rotation angle be the set of target boundaries. , , Let be the second-order normalized central moments.

[0103] In this embodiment, if the user pre-inputs multiple preset templates 4 of different shapes and sizes in step S32, and fails to specify a preset template 4 in advance according to the target 2, then according to step S35 above, the matching degree scores between the target boundary set and the multiple preset templates 4 can be obtained respectively, and the preset template 4 corresponding to the highest matching degree score can be taken as the preset template 4 that best matches the target 2, thereby determining the geometric shape of the target 2 and initially determining the size of the target 2. At the same time, a voting mechanism is used to determine the final geometric shape, that is, the number of times each template is judged as "best match" is counted, and the template with the highest number of votes is selected as the final result.

[0104] VI. Secondary Positioning Production Process and Pose Calculation and Mounting

[0105] The secondary positioning production process of this application is as follows: The first positioning is component positioning. The polarization-splitting dual-array pseudo-TDI imaging subsystem 100 performs aerial photography of the tested component 103, which moves with the carrier mechanism 104, and outputs a high signal-to-noise ratio component image. After image preprocessing, coarse positioning, and fine positioning, the center coordinates and geometric information of the component are obtained. The second positioning is pad positioning. The ordinary area array camera 120 photographs the stationary PCB pads and outputs pad images. After the same set of image preprocessing, coarse positioning, and fine positioning algorithms, the center coordinates and geometric information of the pads are obtained. Unlike the traditional process of pre-setting pad positions and only performing one positioning of the component, this application performs actual imaging positioning of the pads. The two positioning operations target two objects, the component and the pad, respectively, hence the name secondary positioning.

[0106] like Figure 6 As shown, the workflow of the pose calculation and mounting compensation module 270 includes the following steps:

[0107] Step S61: Obtain the component center coordinates and geometric information from the first positioning output;

[0108] Step S62: Obtain the center coordinates and geometric information of the pads from the second positioning output;

[0109] Step S63: After calibration transformation, align the component coordinate system and the pad coordinate system to the mounting coordinate system, and calculate the relative pose deviation between the component and the pad. Specifically, the external parameters of the polarization-splitting dual-array pseudo-TDI imaging subsystem 100 and the ordinary area array camera 120 are calibrated in advance to establish the transformation relationship between the image coordinate systems of the two cameras and the mounting coordinate system (motion platform coordinate system). During the calculation, the component center coordinates and the pad center coordinates are transformed to the mounting coordinate system respectively, and the translational deviation is calculated. , And calculate the angular deviation based on the rotation angle of the component and the pad respectively. ;

[0110] Step S64: Calculate the placement compensation amount based on the relative pose deviation, drive the motion platform to drive the placement execution mechanism 130 to complete translation and rotation compensation, and accurately place the picked-up components onto the target pads.

[0111] Step S65: Output module 240 outputs the center coordinates and geometric information of the component and pad, relative pose deviation, and placement result, which serve as teaching points and compensation data for the placement equipment and can be transmitted back to the production management system for process traceability.

[0112] This embodiment uses a common area array camera as the pad imaging module. Before the placement execution mechanism places the image of the stationary target pad area, it completes the first component positioning and the second pad positioning respectively with the component imaging module. The pose calculation and placement compensation module unifies the center coordinates and geometric information of the component and pad to the placement coordinate system through calibration transformation. It calculates the relative pose deviation, including translational deviation and angular deviation, and drives the placement execution mechanism to compensate. Therefore, it replaces the method of preset pad position in the traditional process, eliminates the placement deviation caused by substrate expansion and contraction, warping, incoming material deviation and clamping error, and achieves precise alignment between the component and the target pad.

[0113] VII. Execution Sequence and Closed-Loop Feedback

[0114] In one embodiment of this application, the execution order of the coarse positioning module 220 and the fine positioning module 230 is as follows: first, the coarse positioning module 220 performs image segmentation to determine the effective recognition area, and then the fine positioning module 230 performs accurate recognition and positioning. This order is suitable for situations where the contrast between the target and the background is high and segmentation is relatively easy, and can quickly narrow down the search range and improve the overall positioning efficiency.

[0115] In this embodiment, the specific workflow of the closed-loop feedback module 250 is as follows: After the fine positioning module 230 outputs the precise center coordinates of the target, the closed-loop feedback module 250 calculates the deviation between the precise center coordinates and the center of the effective recognition area determined by the coarse positioning module 220. If the deviation exceeds a preset threshold, the position and size of the effective recognition area are corrected according to the deviation, and the coarse positioning module 220 and the fine positioning module 230 are triggered to re-execute the positioning process. If the deviation exceeds the preset threshold, the camera is moved again, and the camera is moved relative to the previous position based on the deviation between the coarse and fine positioning. The closed-loop feedback module calculates the deviation between the precise center coordinates of the target and the center of the effective recognition area determined by the coarse positioning after the fine positioning outputs the precise center coordinates of the target. When the deviation exceeds the preset threshold, the position and size of the effective recognition area are corrected according to the deviation, and the coarse and fine positioning are triggered to re-execute. Therefore, an adaptive iterative closed loop is formed, which improves the accuracy of coarse positioning and enhances the stability of the positioning process.

[0116] In this embodiment, the robustness processing module 260 is used to detect and repair defects in the target. Specifically, in the image segmentation stage of coarse localization, in addition to removing burrs and noise, a convex / concave detection algorithm based on contour curvature analysis is added, and morphological repair or marking is performed on the detected defect areas. The matching score determination threshold is adaptively adjusted according to the severity of the defect. For example, when the area of ​​the defect area exceeds a preset proportion, the matching score threshold is reduced to avoid overall matching failure due to local defects. The robustness processing module detects and repairs defects in the target and adaptively adjusts the matching score determination threshold according to the severity of the defect. At the same time, the fine localization adopts multi-scale template matching, matching score fusion, and multi-template voting mechanism, thus effectively dealing with complex situations such as blurred boundaries, multiple protrusions and burrs, and contamination in component and pad images, improving the reliability of the matching results.

[0117] Specific Implementation Method Two: Refer to Figure 7 In detail, this application also provides a computer device 5, including a memory 51, a processor 52, and a network interface 53 that are interconnected via a system bus.

[0118] The memory 51 stores computer-readable instructions, and the processor 52 executes these computer-readable instructions to implement the steps of the above-described mounting secondary positioning method. The computer device 5 can be a desktop computer, laptop, handheld computer, or cloud server, etc. The memory 51 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory, random access memory, static random access memory, read-only memory, electrically erasable programmable read-only memory, programmable read-only memory, magnetic memory, disk, optical disk, etc. The processor 52 can be a central processing unit, controller, microcontroller, microprocessor, or other data processing chip. The network interface 53 can include a wireless network interface or a wired network interface.

[0119] Specific implementation method three: This application also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the above-mentioned mounting secondary positioning method.

[0120] In summary, this application utilizes a polarization-splitting dual-array pseudo-TDI imaging subsystem to perform high signal-to-noise ratio flight imaging of moving components, while using a conventional area array camera for actual imaging and positioning of the pads. The same "coarse positioning + fine positioning" algorithm is used to obtain the precise poses of the two objects, replacing the traditional method of pre-setting pad positions. Simultaneously, a pre-set template is stitched together to form a stitching template, effectively reducing the size of the recognition area during target identification, thus saving recognition time. The center of the stitching template is placed on the target to ensure complete coverage and positioning, thereby improving recognition accuracy. The secondary positioning system provided in this application can enhance the automation level of placement equipment such as pick-and-place machines and die bonders, reduce the probability of human error, eliminate placement deviations caused by substrate expansion and contraction and clamping errors, and further improve production efficiency and yield.

[0121] While specific embodiments of this application have been described herein with reference to them, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging, characterized in that, include: Component imaging module, pad imaging module, positioning module, and pose calculation and placement compensation module; The component imaging module is a polarization-splitting dual-array pseudo-TDI imaging subsystem, used to take aerial images of components that move with the carrier mechanism. The pad imaging module is used to photograph stationary pads to obtain pad images; The positioning module is used to perform coarse positioning and fine positioning on the component image and the pad image. The coarse positioning is used to determine the effective identification area where the component and the pad are located. The fine positioning is used to determine the positioning information of the component and the pad within the effective identification area. The positioning information includes center coordinates and geometric information. The pose calculation and placement compensation module is used to calculate the relative pose deviation between the component and the pad based on the positioning information of the component and the pad, and drive the placement execution mechanism to perform compensation placement based on the relative pose deviation. The polarization-splitting dual-array pseudo-TDI imaging subsystem includes: An illumination assembly for providing the element with a light source having two orthogonal polarization directions; A beam splitter is used to separate the imaging light returned by the element into two beams according to their polarization states, which are then received by two area array cameras respectively. The image processing unit is used to align the images output by the two area array cameras along the row direction, and to integrate multiple exposures corresponding to the same spatial pixel column position by row to generate a TDI line array image. The synchronization control unit is used to alternately trigger the two area array cameras to expose line by line according to the pulse signal of the encoder, so that the same spatial position on the element is exposed multiple times.

2. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 1, characterized in that, The lighting assembly includes: a lighting source and a polarizer array covering its light-emitting surface; The polarizer array comprises two different polarizer halves joined side by side, enabling the light emitted by the illumination source to form two mutually orthogonal polarization light sources; The beam-splitting assembly includes: an imaging objective lens and a polarizing beam-splitting prism; The imaging objective lens converges the imaging light returned by the element and then directs it onto the polarizing beam splitter. The polarizing beam splitter reflects a polarized light source to one imaging arm, which is then received by an area array camera. The polarizing beam splitter transmits another polarized light source to another imaging arm, which is then received by another area array camera after passing through a wedge prism.

3. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 1, characterized in that, The dividing line between the two different polarizer halves is perpendicular to the direction of movement of the transport mechanism; The two area array cameras receive imaging light from the field of view respectively, and the two images are stitched together to obtain a complete component image.

4. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 1, characterized in that, Determining the effective identification area where the component and the pad are located includes: Calculate the global grayscale segmentation threshold of the target image, and separate the target object from the background based on the global grayscale segmentation threshold; Remove the burrs and noise from the segmented target image and fill the holes to obtain the repaired image; The boundary set of all target objects in the restored image is extracted using a boundary contour extraction algorithm; Based on preset geometric feature parameters, the boundary that is closest to the geometric features of the target object is selected from the boundary set, and the area where it is located is taken as the effective identification area. The target object is the component or the pad, and the target image is the image of the component or the image of the pad.

5. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 1, characterized in that, Determining the positioning information of the component and the pad within the effective identification area includes: Within the effective recognition area, a preset template is retrieved. Using the preset reference position coordinates on the target object as the center, the preset template is translated by at least one unit in the vertical and / or horizontal direction to splice the template, thereby obtaining a spliced ​​template. The center of the spliced ​​template is located on the target object. The image located within the stitching template is binarized to extract the target boundary set; Calculate the center coordinates of the target boundary set, and use them as the center coordinates of the target object; Calculate the matching score between the target boundary set and the preset template, and confirm the geometric information of the target object based on the matching score; The target object is the component or the pad.

6. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 5, characterized in that, The step of calculating the matching score between the target boundary set and the preset template, and confirming the geometric information of the target object based on the matching score, includes: Obtain the grayscale value of each pixel in the overlapping region of the target boundary set and the preset template; Calculate the matching score based on the grayscale value of each pixel in the overlapping region; The geometric shape corresponding to the preset template with the highest matching score is determined as the geometric shape of the target object; Calculate the rotation angle of the target object based on the coordinates of each pixel in the target boundary set; The geometric shape, the boundary length of the target boundary set, and the rotation angle are used as the geometric information of the target object.

7. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 6, characterized in that, The matching score is the normalized grayscale correlation coefficient, the normalized grayscale error sum of squares, or the weighted fusion value of the normalized grayscale correlation coefficient and the normalized grayscale error sum of squares.

8. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claims 1 to 7, characterized in that, It also includes a closed-loop feedback module, which is used to calculate the deviation between the center coordinates and the center of the effective identification area determined by the coarse positioning after the center coordinates of the target object are output by the fine positioning. If the deviation exceeds a preset threshold, the effective identification area is corrected according to the deviation, and the coarse positioning and the fine positioning are re-executed until the deviation is less than the preset threshold or the maximum number of iterations is reached. The target object is the component or the pad.

9. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 8, characterized in that, It also includes a robustness processing module, which is used to detect and repair defects in the target object and adaptively adjust the matching score threshold according to the severity of the defects.

10. The mounting secondary positioning system based on polarization-splitting pseudo-TDI imaging according to claim 9, characterized in that, It also includes an image preprocessing module, which is used to sequentially perform grayscale conversion, noise reduction and enhancement processing on the component image and the pad image.

Citation Information

Patent Citations

  • Image pick-up device with two large-format high-frame digital CCDs (Charge Coupled Device)

    CN106060354A

  • Asynchronous exposure double-view-field imaging system and method

    CN121357408A