A visual recognition-based circuit board solder joint defect detection method

Abstract

The present application relates to the technical field of machine vision, and specifically relates to a circuit board solder joint defect detection method based on visual recognition, which comprises reading pad design coordinates and extracting auxiliary pads outside the BGA packaging device boundary; visible metal pads or auxiliary pads are identified in SPI images and AOI images, corresponding rack coordinates are obtained by combining pixel resolution, distortion correction and device rack position conversion; the rack coordinates and the design coordinates constitute a control point set, and coordinate mapping functions from SPI rack coordinates to design coordinates and from AOI rack coordinates to design coordinates are established; the SPI detection results of target pads and the AOI detection results of target solder joints are mapped to the same design coordinate system, whether it is the same solder joint is judged according to the cross-process correlation distance, and the defect source is determined in combination with the solder paste coverage state and the solder coverage state. The present application reduces the cross-process coordinate correlation error of the printed circuit board in the BGA peripheral area of the BGA packaging device and reduces the error correlation of adjacent pads.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically a method for detecting defects in circuit board solder joints based on visual recognition. Background Technology

[0002] Solder paste inspection equipment and automated optical inspection equipment each use independent racks. The rack coordinates recorded for the same solder pad are not the same in the two equipments, and both deviate from the design coordinates defined in the design file (Gerber file). The existing method for cross-process coordinate association involves reserving 2 to 3 circular bare copper positioning marks at the corners or edges of the printed circuit board. The two equipments identify the pixel coordinates of these marks in their respective images, and fit affine transformation parameters based on the design coordinates of the marks, covering three types of linear transformations: translation, rotation, and uniform scaling. During cross-process association, the rack coordinates of each solder joint in the two equipments are mapped to the design coordinates through their respective affine transformations. The distance between the mapped coordinates is used as the criterion, and the one with the smallest distance is identified as the same solder joint. On printed circuit boards without BGA packaged devices, the copper layer is relatively uniformly distributed on the board surface, and the association error of the affine transformation method is usually between several micrometers and more than ten micrometers, which can meet the accuracy requirements of most package specifications and is widely used in the industry.

[0003] However, printed circuit boards (PCBs) containing BGA packages exhibit inspection defects. The copper coverage of the copper plating area beneath the BGA package mounting area is typically between 60% and 80%, while the copper coverage of the signal traces surrounding the device is only between 10% and 20%. This abrupt change in copper coverage occurs in a transition zone approximately 2mm to 5mm wide near the BGA package boundary. The in-plane coefficient of thermal expansion of the copper layer is approximately 17ppm / ℃, while that of the PCB substrate is approximately 14ppm / ℃ to 15ppm / ℃. After solder paste inspection, the PCB must undergo a reflow soldering process, with a peak temperature of approximately 240℃ to 260℃, a temperature difference of approximately 220℃ from room temperature. Due to the difference in thermal expansion coefficients between the copper layer and the substrate, the local effective in-plane shrinkage strain generated by the BGA copper plating area with higher copper coverage after reflow thermal cycling is greater than that of the surrounding signal trace area with lower copper coverage. When the copper coverage difference between the two areas is approximately 55%, the local effective in-plane strain difference between the two areas is approximately 2.4 × 10⁻⁶. -4The strain is concentrated in a transition region approximately 2mm to 5mm wide near the BGA package boundary, where a significant strain gradient occurs and the strain changes nonlinearly with position. The affine transformation establishes a uniform linear mapping across the entire board using corner positioning marks as anchor points, essentially assuming a uniform strain distribution across the board. For a 400mm × 300mm printed circuit board, the corner positioning marks are approximately 150mm to 250mm from the BGA package boundary. The affine transformation uses this distance to linearly extrapolate the region near the BGA package boundary. The extrapolation residual is approximately the product of the local effective strain difference and the extrapolation distance, ranging in size from approximately 36μm to 60μm, increasing with the size of the printed circuit board, reaching the 60μm level in cases with larger board sizes.

[0004] The center-to-center spacing of the 0201 package pads is approximately 0.3 mm, and the safety tolerance for cross-process correlation is about 1 / 4 of the pad spacing, or approximately 80 μm. The approximately 60 μm residual introduced by the affine transformation near the BGA package boundary is on the same order of magnitude as this 80 μm tolerance. In actual production lines, there have been instances where the affine transformation residual exceeds the tolerance: the distance between the affine mapping coordinates of two adjacent pads is less than the mapping error of a certain pad itself, causing solder paste inspection data for that pad to be incorrectly correlated with the automatic optical inspection solder joint data of adjacent pads. The 0201 package components are densely distributed around the BGA package device, specifically in the area where copper coverage abruptly transitions, making traditional correlation methods highly prone to errors. Once an incorrect association occurs, the subsequent defect source judgment will be directly wrong: if a solder pad is underprinted but its associated data comes from a defect-free solder joint of an adjacent solder pad, the defect will be missed; if a solder joint has insufficient solder but its associated solder paste detection data comes from an adjacent solder pad with sufficient solder paste, the defect source will be judged as poor reflow. The process engineer will adjust the reflow oven parameters accordingly, but the actual reason is insufficient printing. The adjustment direction is wrong, and the defect rate will naturally not be improved. Summary of the Invention

[0005] (1) Technical problem to be solved: The purpose of this invention is to provide a method for detecting solder joint defects on circuit boards based on visual recognition, so as to solve the cross-process coordinate correlation error of printed circuit boards of BGA packaged devices in the BGA peripheral area.

[0006] (2) Technical solution: To achieve the above objectives, on the one hand, the present invention provides a method for detecting solder joint defects on circuit boards based on visual recognition, including: The design coordinates of each pad are read from the PCB design file. The design coordinates of auxiliary pads that have no openings in the solder paste layer but have openings in the solder mask layer are extracted. The auxiliary pads are distributed outside the boundary of the BGA package device, and there is at least one auxiliary pad in each of the four directions of the BGA package device boundary. Pads in the bare copper state are detected in the SPI image. The SPI image coordinates of each bare copper pad are converted by pixel resolution and distortion correction to obtain the SPI rig coordinates. The SPI rig coordinates are paired with the design coordinates of each bare copper pad to form the SPI control point set. The coordinate mapping from the SPI rig coordinates to the design coordinates is established using the SPI control point set as the interpolation node to obtain the SPI coordinate mapping function. In the AOI image, auxiliary pads are detected. The AOI image coordinates of each auxiliary pad are converted by pixel resolution and distortion correction to obtain the AOI bench coordinates. The AOI bench coordinates are paired with the design coordinates of each auxiliary pad to form an AOI control point set. The AOI control point set is used as the interpolation node to establish a coordinate mapping from the AOI bench coordinates to the design coordinates, and the AOI coordinate mapping function is obtained. Extract the centroid coordinates of the solder paste outline of the target pad from the SPI image. After pixel resolution conversion and distortion correction, input the results into the SPI coordinate mapping function to obtain the SPI mapping design coordinates of the target pad. Extract the centroid coordinates of the solder outline of the target solder joint from the AOI image. After pixel resolution conversion and distortion correction, input the results into the AOI coordinate mapping function to obtain the AOI mapping design coordinates of the target solder joint. Calculate the cross-process association distance based on the SPI mapping design coordinates and the AOI mapping design coordinates. When the cross-process association distance does not exceed the association distance threshold, it is determined that the detection results of the SPI image and the AOI image belong to the same solder joint. Based on the solder paste coverage state of the target pad in the SPI image and the solder coverage state of the target solder joint in the AOI image, the solder joint defect source determination result of the target solder joint is obtained.

[0007] Furthermore, the method for detecting pads in a bare copper state in the SPI image includes: An initial affine transformation of the SPI image is established by using the image coordinates of the corner positioning marks on the printed circuit board surface and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of each pad, the image sub-region of the i-th pad is determined in the SPI image. The SPI image is binarized to obtain a binary image of the SPI coverage. The number of solder paste pixels in the i-th pad's image sub-region and the total number of pixels in the SPI coverage binary image are counted to obtain the solder paste coverage of the i-th pad. When the solder paste coverage of the i-th pad is lower than the bare copper determination threshold, the i-th pad is marked as a bare copper pad, and auxiliary pads are also included in the bare copper pad category. Based on the initial affine transformation and the design coordinates and dimensions of each bare copper pad, the image sub-region of each bare copper pad is determined in the SPI image. Contour extraction is performed on the image sub-region of each bare copper pad to obtain the SPI image coordinates of each bare copper pad.

[0008] Furthermore, the method for extracting the contour of each bare copper pad in the image sub-region to obtain the SPI image coordinates of each bare copper pad includes: The bare copper pads are subjected to median filtering in the image sub-region to obtain a filtered bare copper image. The filtered bare copper image is then binarized using the Otsu thresholding method, with the Otsu threshold automatically calculated from the grayscale histogram of the filtered bare copper image, resulting in a binary bare copper image. Morphological closing operations are performed on the binary bare copper image to fill the internal holes of the contour, resulting in a closed bare copper image. Canny edge detection is applied to the closed bare copper image to extract the pixel set of the outer contour of the pads. The geometric center is calculated based on the coordinates of each pixel in the pixel set of the outer contour of the pads, resulting in the SPI image coordinates of the bare copper pads.

[0009] Furthermore, the method for establishing a coordinate mapping from SPI bench coordinates to design coordinates using the SPI control point set as interpolation nodes to obtain the SPI coordinate mapping function, and for establishing a coordinate mapping from AOI bench coordinates to design coordinates using the AOI control point set as interpolation nodes to obtain the AOI coordinate mapping function, includes: Using the SPI platform coordinates of each control point in the SPI control point set as interpolation nodes, thin-plate spline interpolation functions are established for the horizontal and vertical directions respectively; through the first... The control point and the first Substituting the Euclidean distance between the SPI platform coordinates of each control point into the basis function, the resulting basis function value is used as the kernel matrix. Line 1 The elements of the column are used to construct the kernel matrix, when equal The corresponding elements are set to 0; an affine matrix is ​​constructed using the horizontal and vertical components of the SPI bench coordinates of each control point, and a constant term with a value of one; the kernel matrix, affine matrix, transpose of the affine matrix, and zero matrix are combined to obtain the coefficient matrix; the horizontal component of the pad design coordinates of each control point is used as the first segment of the right-hand vector, and the remaining components in the right-hand vector are set to 0, forming a system of block linear equations with the coefficient matrix, and the weight coefficients and affine parameters of each control point in the horizontal direction are obtained by solving the system; the first segment of the right-hand vector is replaced with the vertical component of the pad design coordinates of each control point, and the weight coefficients and affine parameters of each control point in the vertical direction are obtained by solving the system with the same coefficient matrix; the Euclidean distance between the bench coordinates to be mapped and the bench coordinates of each control point is substituted into the basis function, and the corresponding affine parameters are superimposed after weighted summation with the horizontal and vertical weight coefficients respectively, to obtain the mapped design coordinates in the horizontal and vertical directions, which together constitute the SPI coordinate mapping function; the AOI coordinate mapping function is obtained through the AOI control point set using the same method.

[0010] Furthermore, the above is achieved through the first The control point and the first Methods for substituting the Euclidean distance between the SPI platform coordinates of each control point into the basis function include: Let the first The SPI bench coordinates of each control point are: , No. The SPI bench coordinates of each control point are: Euclidean distance between two points for: ; when When it is greater than 0, by substituting the basis function Obtain the basis function values; when When the value is 0, the basis function value is 0.

[0011] Furthermore, the method for extracting the centroid coordinates of the solder contour of the target solder joint in the AOI image includes: An initial affine transformation for AOI is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the AOI image and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of the target solder joint, the image sub-region of the target solder joint in the AOI image is determined. Median filtering is applied to the image sub-region of the target solder joint to suppress the noise of speckled spots on the solder spherical surface, resulting in a solder-filtered image. The solder-filtered image is binarized using the Otsu thresholding method to separate the solder area from the printed circuit board substrate area. The Otsu threshold is automatically calculated from the grayscale histogram of the solder-filtered image, resulting in a solder binary image. Morphological closing operations are performed on the solder binary image to obtain a solder closing operation image. Canny edge detection is applied to the solder closing operation image to extract the pixel set of the solder outer contour. The geometric center is calculated based on the coordinates of each pixel in the solder outer contour pixel set, resulting in the centroid coordinates of the solder contour of the target solder joint.

[0012] Furthermore, the method for determining the source of solder joint defects of the target solder joint based on the solder paste coverage state of the target pad in the SPI image and the solder coverage state of the target solder joint in the AOI image includes: The measured solder paste coverage area of ​​the target pad is obtained from the SPI image. The designed area of ​​the target pad is read from the design file. The solder paste area coverage ratio is obtained from the measured solder paste coverage area and the designed pad area. The measured solder joint coverage area of ​​the target solder joint is obtained from the AOI image. The measured solder joint coverage area is compared with the solder joint area qualification threshold. When the measured solder joint coverage area is lower than the solder joint area qualification threshold, it is determined that the target solder joint has a solder shortage defect. The solder paste area coverage ratio is further compared with the solder paste quantity qualification ratio threshold. When the solder paste area coverage ratio is lower than the solder paste quantity qualification ratio threshold, the defect source is determined to be insufficient printing. When the solder paste area coverage ratio is not lower than the solder paste quantity qualification ratio threshold, the defect source is determined to be poor reflow.

[0013] Furthermore, the method for obtaining the measured solder paste coverage area of ​​the target pad based on the SPI image includes: An initial affine transformation of the SPI image is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the SPI image and their design coordinates. Based on the initial affine transformation of the SPI image and the design coordinates and dimensions of the target pad, the image sub-region of the target pad in the SPI image is determined. Median filtering is performed on the target pad image sub-region to obtain a solder paste filtered image. The solder paste filtered image is binarized using the Otsu thresholding method, where the Otsu threshold is automatically calculated from the grayscale histogram of the solder paste filtered image to obtain a solder paste binary image. Morphological closing operation is performed on the solder paste binary image to obtain a solder paste closed image. The number of pixels belonging to the solder paste region in the solder paste closed image is counted to obtain the number of pixels in the solder paste region. The measured coverage area of ​​the solder paste is obtained by dividing the number of pixels in the solder paste region by the square of the SPI pixel resolution.

[0014] Furthermore, the method for obtaining the measured coverage area of ​​the target solder joint based on the AOI image includes: An initial affine transformation for the AOI (Automated Optical Inspection) is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the AOI image and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of the target solder joint, the image sub-region of the target solder joint in the AOI image is determined. Median filtering is applied to the target solder joint image sub-region to obtain a solder area filtered image. The solder area filtered image is binarized using the Otsu thresholding method to separate the solder area from the printed circuit board substrate area. The Otsu threshold is automatically calculated from the grayscale histogram of the solder area filtered image to obtain a solder area binary image. Morphological closing operation is performed on the solder area binary image to obtain a solder area closed operation image. The number of pixels belonging to the solder area in the solder area closed operation image is counted to obtain the number of solder area pixels. The measured coverage area of ​​the solder joint is obtained by dividing the number of solder area pixels by the square of the AOI pixel resolution.

[0015] (3) Beneficial effects: Compared with the prior art, the present invention significantly reduces the cross-process coordinate correlation error of 0201 solder joints around BGA on printed circuit boards containing BGA packaged devices compared with the traditional affine transformation method, and eliminates the erroneous correlation between adjacent solder pads; this method only uses the existing solder paste inspection images and automatic optical inspection images on the production line, as well as the solder pad coordinate information already existing in the printed circuit board design file, to arrange auxiliary solder pads near the BGA package boundary, without the need to introduce additional sensors. By comparing the ratio of the solder paste coverage area measured by solder paste inspection to the solder pad design area and the preset threshold, the sources of insufficient solder defects determined by automatic optical inspection can be divided into two categories: insufficient printing and poor reflow, providing process engineers with reliable process improvement directions. Attached Figure Description

[0016] The above and / or other aspects of this application will become more apparent from the description of certain embodiments with reference to the accompanying drawings, in which: Figure 1 This is a flowchart of a circuit board solder joint defect detection method based on visual recognition according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the printed circuit board component layout according to Embodiment 1 of the present invention; Figure 3 This is a comparison diagram of cross-process coordinate correlation errors around the BGA in Embodiment 1 of the present invention; Figure 4 This is a diagram illustrating the SPI bare copper pad contour extraction process in Embodiment 3 of the present invention. Figure 5 This is a diagram illustrating the AOI solder contour extraction process in Embodiment 6 of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Before providing examples, it is necessary to describe the application scenario of this invention. A surface mount production line for printed circuit boards consists of multiple fixed machines arranged in sequence. The printed circuit boards pass through each machine sequentially under the drive of a conveyor mechanism. A typical production sequence is: solder paste printer, pick and place machine, reflow oven, and optical inspection equipment located at the exit of each key process.

[0019] The solder paste printer uses a metal stencil to apply solder paste to the surface of each pad on a printed circuit board. The coverage area and thickness of the solder paste on each pad determine the volume of the solder joint after reflow soldering. After the solder paste printing is completed, the printed circuit board is conveyed out of the printer by a conveyor belt. At this point, the board only has solder paste on it and no components yet, and then it enters the solder paste inspection process.

[0020] The solder paste inspection equipment is fixedly installed at the exit of the solder paste printer. Its main structure consists of a set of guide rails mounted above the printed circuit board (PCB). An industrial camera is mounted on an electrically controlled frame that can move independently in two mutually perpendicular directions (hereinafter referred to as the X-axis and Y-axis) in the horizontal plane. After the PCB is conveyed into the equipment and fixed by a clamping mechanism, the frame drives the camera to move continuously above the PCB along a partitioned path. At each position, a top-down view image is captured, eventually covering the entire PCB surface. At the moment each frame is captured, the equipment automatically records the actual position of the frame in the X-axis and Y-axis directions; these two values ​​together are called the frame coordinates for that frame. The solder paste inspection equipment records the coordinates of the geometric center of the solder paste coverage area on each pad under the frame coordinates, forming the solder paste inspection data.

[0021] After the solder paste inspection is completed, the printed circuit board enters the surface mount process, where the pick-and-place machine places each surface mount component onto its corresponding pad. The printed circuit board then enters the reflow oven, where it undergoes a thermal cycle of heating, peak holding, and cooling. The peak temperature is approximately 240°C to 260°C. During the peak temperature, the solder paste on the pads completely melts and wets the component leads. After cooling and solidifying, it forms a solder joint (solder joint).

[0022] The automated optical inspection equipment is fixedly installed at the reflow oven outlet. Its main structure consists of an industrial camera mounted on an electrically controlled frame that can move along the X and Y axes. After the printed circuit board enters the equipment, the frame drives the camera to scan the entire board and capture images frame by frame, recording the corresponding frame coordinates for each frame. The frame coordinate systems of the automated optical inspection equipment and the solder paste inspection equipment are independent of each other, and the equipment records the coordinates of the geometric center of the solder contour at each solder joint under the machine's frame coordinate system.

[0023] Because there are two processes—surface mounting and reflow soldering—between solder paste inspection and automated optical inspection, printed circuit boards (PCBs) must be re-transferred and repositioned separately when entering the two machines. Each time, the actual position of the PCB relative to the machine platform has a random deviation, determined by the repeatability of the conveying and clamping mechanisms, typically on the order of 0.1mm to 0.5mm. Furthermore, the zero points of the two machine platforms are inherently different; the platform coordinates recorded for the same physical pad in the solder paste inspection equipment can differ by several millimeters or even tens of millimeters from those recorded in the automated optical inspection equipment, making direct comparison impossible.

[0024] Printed circuit board (PCB) design files (Gerber files) are created in advance by PCB design engineers, recording the geometry and center coordinates of all pads, traces, and markings on the board. Solder paste inspection equipment and automated optical inspection equipment can read Gerber files to obtain the design coordinates of each pad, which is used for target area positioning during inspection.

[0025] BGA (Ball Grid Array) packaged devices have densely packed pins on the bottom of the chip, with pin pitch typically between 0.5mm and 1mm. The device size varies from a few millimeters to tens of millimeters, and there is a large copper overlay beneath the mounting area. 0201 packaged surface-mount resistors and capacitors are usually densely arranged around the BGA package, with the center-to-center spacing between adjacent 0201 package pads approximately 0.3mm. For example, a communication printed circuit board with dimensions of approximately 400mm × 300mm contains a 25mm × 25mm BGA packaged device, surrounded by densely packed 0201 packaged resistors and capacitors. During production line operation, when automatic optical inspection detects insufficient solder at a 0201 solder joint around the BGA, the process engineer needs to determine whether the defect is caused by insufficient solder paste printing or inadequate reflow soldering: insufficient printing should be addressed by adjusting the stencil pressure or squeegee parameters of the printer; inadequate reflow should be addressed by adjusting the reflow oven temperature settings or conveyor speed. The two types of processing have different directions, and the basis for judgment is the cross-process correlation result of the solder paste inspection data and the automatic optical inspection data of the same pad.

[0026] Example 1: As Figure 1 As shown, this embodiment provides a method for detecting solder joint defects on circuit boards based on visual recognition, including: The design coordinates of each pad are read from the PCB design file. The design coordinates of auxiliary pads that have no openings in the solder paste layer but have openings in the solder mask layer are extracted. The auxiliary pads are distributed outside the boundary of the BGA package device, and there is at least one auxiliary pad in each of the four directions of the BGA package device boundary. Pads in the bare copper state are detected in the SPI image. The SPI image coordinates of each bare copper pad are converted by pixel resolution and distortion correction to obtain the SPI rig coordinates. The SPI rig coordinates are paired with the design coordinates of each bare copper pad to form the SPI control point set. The coordinate mapping from the SPI rig coordinates to the design coordinates is established using the SPI control point set as the interpolation node to obtain the SPI coordinate mapping function. In the AOI image, auxiliary pads are detected. The AOI image coordinates of each auxiliary pad are converted by pixel resolution and distortion correction to obtain the AOI bench coordinates. The AOI bench coordinates are paired with the design coordinates of each auxiliary pad to form an AOI control point set. The AOI control point set is used as the interpolation node to establish a coordinate mapping from the AOI bench coordinates to the design coordinates, and the AOI coordinate mapping function is obtained. Extract the centroid coordinates of the solder paste outline of the target pad from the SPI image. After pixel resolution conversion and distortion correction, input the results into the SPI coordinate mapping function to obtain the SPI mapping design coordinates of the target pad. Extract the centroid coordinates of the solder outline of the target solder joint from the AOI image. After pixel resolution conversion and distortion correction, input the results into the AOI coordinate mapping function to obtain the AOI mapping design coordinates of the target solder joint. Calculate the cross-process association distance based on the SPI mapping design coordinates and the AOI mapping design coordinates. When the cross-process association distance does not exceed the association distance threshold, it is determined that the detection results of the SPI image and the AOI image belong to the same solder joint. Based on the solder paste coverage state of the target pad in the SPI image and the solder coverage state of the target solder joint in the AOI image, the solder joint defect source determination result of the target solder joint is obtained.

[0027] To better understand the specific implementation of this embodiment, exemplarily, as follows: Figure 2As shown, the printed circuit board (PCB) includes BGA packages located on the PCB surface, circular auxiliary pads distributed in four directions along the outer edge of the BGA package boundaries, target pads (represented by rectangles) distributed around the BGA packages, and positioning marks at the corners of the PCB. The first step involves reading the design file, extracting the coordinates of the auxiliary pads, detecting the bare copper pads in the solder paste inspection image, and establishing a solder paste inspection coordinate mapping function. Specifically, the PCB design file (Gerber file) containing BGA packages is used as input. The design coordinates and geometric dimensions of all pads on the PCB are read from the design file, with the origin defined at a fixed reference point on the PCB. Auxiliary pads that simultaneously meet the following conditions are selected: no opening at the corresponding position in the solder paste layer file, and an opening at the corresponding position in the solder mask layer file. The set of all auxiliary pad design coordinates is denoted as […]. , To indicate the total number of pads, the subscript is used. For numbering.

[0028] It should be noted that auxiliary pads are non-assembly pads reserved during the PCB design phase. They do not correspond to surface mount component leads and do not participate in soldering connections. Auxiliary pads can be isolated copper foil patterns or connected to test networks that do not affect circuit functionality, while meeting electrical clearance requirements with surrounding pads and traces. Because auxiliary pads have no openings in the solder paste layer, no solder paste is deposited during stencil printing, and no components are placed during the surface mount process, so no solder joints or solder bridges will form after reflow soldering. The term "bare copper state" or "exposed metal state" as used in this article refers to a state where the solder mask opening area of ​​the pad is not obscured by solder paste, solder, or components and can be recognized by the image. Its surface can be copper or a surface treatment layer such as immersion gold, immersion tin, or OSP.

[0029] In this specific example, the printed circuit board measures 400mm × 300mm and includes a 25mm × 25mm BGA package device. The BGA package boundary ranges approximately [187.5mm, 212.5mm] × [137.5mm, 162.5mm]. Three 0.5mm diameter circular auxiliary pads are arranged 3.5mm outside the BGA package boundary along the top, bottom, left, and right directions, for a total of... =12: Vertical coordinates of the upper auxiliary pads =166.0mm, x-axis values ​​are respectively taken as 166.0mm. =190.0mm, 200.0mm, 210.0mm; Vertical coordinates of the lower auxiliary pads =134.0mm, x-axis values ​​are respectively taken as 134.0mm. =190.0mm, 200.0mm, 210.0mm; Horizontal coordinates of the left auxiliary pads =184.0mm, the ordinate is respectively taken as 184.0mm. =144.0mm, 150.0mm, 156.0mm; x-coordinate of the auxiliary pad on the right. =216.0mm, the ordinate is respectively taken as =144.0mm, 150.0mm, 156.0mm. Upper auxiliary pads. =166.0mm>162.5mm, below =134.0mm<137.5mm, left side =184.0mm<187.5mm, right side =216.0mm>212.5mm, there is at least one auxiliary pad in each of the four directions, and the distribution conditions are met.

[0030] The solder paste inspection camera uses the Zhang Zhengyou calibration method (using a precision checkerboard calibration target to capture calibration images at multiple positions on the test bench) to solve for intrinsic parameters, including focal length, principal point coordinates, and lens radial distortion coefficient. , ) and tangential distortion coefficient ( , Pixel resolution is tested using solder paste. =0.010mm / pixel, for any pixel coordinate in the solder paste inspection image. First, the Brown-Conrady distortion correction model is executed using the calibrated distortion coefficients (calculating the distortion-free pixel coordinates based on the radial and tangential distortion coefficients) to remove the radial and tangential distortion of the lens and obtain the distortion-free pixel coordinates. Then multiply by the pixel resolution to convert to platform coordinates. The automated optical inspection equipment uses the same calibration and correction process, with the same pixel resolution. =0.008mm / pixel, corresponding to the distortion-free result multiplied by Converted to bench coordinates. The phrase "obtained bench coordinates after pixel resolution conversion and distortion correction" mentioned later in this manual refers to the operation of multiplying the Brown-Conrady distortion-free result by the corresponding device pixel resolution.

[0031] It should be noted that when converting the image coordinates to gantry coordinates, if the image coordinates are pixel coordinates within a single frame, the conversion is performed in conjunction with the gantry position recorded by the device at the moment the frame was captured. Specifically, for a frame of image from the SPI device, the device records the gantry position at the moment the frame was captured. After distortion correction, the pixel coordinates are obtained as deformed pixel coordinates. Based on the offset of the deformed pixel coordinates relative to the image principal point, the SPI pixel resolution, the gantry position corresponding to the frame, and the camera mounting offset, the SPI gantry coordinates of the pixel in the SPI gantry coordinate system are obtained. The AOI gantry coordinates are obtained in the same way, from the gantry position of the corresponding frame of the AOI image, the AOI pixel resolution, the AOI image principal point, and the AOI camera mounting offset. If the image coordinates output by the detection device are already the full-board image coordinates after device stitching and gantry position compensation, then the gantry position compensation is included in the image coordinates output by the device, and only distortion correction and scaling are needed subsequently.

[0032] In solder paste inspection images, pads that appear as bare copper are detected. The specific procedure is described later in this manual as a section on bare copper pad detection and coordinate extraction. The detected pads are... Each bare copper pad (including auxiliary pads and occasional bare copper ordinary pads) has its solder paste detection image coordinates extracted, and the table coordinates are obtained after pixel resolution conversion and distortion correction. Retrieve the corresponding pad design coordinates from the design file. This constitutes the set of solder paste inspection control points: ; In this specific example, all 12 auxiliary pads were included in the control points, and 3 additional ordinary pads were used to compensate for occasional underprinting, totaling... =15 control points, with at least one control point in each of the four directions of the BGA package boundary. Requirement ≥9 control points are required, covering all four directions of the BGA package boundary. This ensures that the control point set forms a convex hull surrounding the BGA copper pour boundary, allowing thin-board spline mapping to be calculated using interpolation for the target pad area around the BGA package. When the number of control points is extremely small (e.g., only 4 points), the interpolation quality between control points is poor, and the curvature of local areas near adjacent control points cannot be constrained by a single control point, resulting in large interpolation residuals. Experiments show that when the number of control points... When the number of control points is not less than 9, the interpolation residual in the target pad area around the BGA can be stably lower than 35μm; when the number of control points is less than 9, interpolation oscillations may occur due to insufficient local control point density.

[0033] It should be noted that there must be at least one auxiliary pad in each of the four directions of the BGA package boundary. This is the minimum spatial distribution condition to avoid the mapping extrapolation caused by all control points being located on one side of the BGA package. When actually establishing coordinate mapping, the number of control points is determined based on the target pad density, the target area, and the correlation distance threshold. When the number of auxiliary pads is small but the target pad area is small, the spacing between adjacent pads is large, or the correlation distance threshold is wide, the auxiliary pads in the four directions can be used to establish local coordinate mapping. When the target area is a densely packed 0201 package area around the BGA, it is preferable to have at least 9 control points to reduce local interpolation residuals. If the current number of auxiliary pads is insufficient to meet the preset correlation distance threshold, the board is marked as having uncertain correlation or a prompt to supplement auxiliary pads is made, and no automatic defect source determination result is output.

[0034] With control point set Establish interpolation nodes based on the coordinates of the solder paste inspection bench. To design coordinates The coordinate mapping is used to obtain the solder paste detection coordinate mapping function. and This specific example uses a thin-plate spline interpolation function, the construction method of which is described later in this specification. It should be noted that the coordinate mapping only needs to meet the following requirements: accurate interpolation at control points (mapped error is zero), continuous and smooth calculation within the area enclosed by the control points, and the ability to capture different local nonlinear deformations at various spatial locations. The thin-plate spline meets these requirements, reducing the cross-process coordinate correlation error of the 0201 solder joints around the BGA package from approximately 60μm based solely on the affine transformation method using board corner positioning marks to approximately 10μm to 30μm, meeting the accuracy requirement of approximately 80μm tolerance for the 0201 package.

[0035] It should be noted that after forming the control point set, residual checks are performed on the control points. By temporarily removing one control point and establishing a coordinate mapping using the remaining control points, the mapping residual of the removed control point is calculated. When the mapping residual exceeds the control point residual threshold (e.g., significantly higher than the upper limit of the normal residual by 50 μm), the control point is removed as an abnormal control point, and a new coordinate mapping is established.

[0036] In addition, some equivalent implementations include natural neighborhood interpolation, which is a coordinate mapping substitution that does not require user-defined parameters. Its principle is as follows: using the coordinates of all control points on the test bench as input, a Voronoi partition is constructed for the control point set; the points to be mapped... After temporarily inserting the control point set, the Voronoi partition is reconstructed. The proportion of the area occupied by the Voronoi element of each control point in the new partition and the area occupied by the element to be mapped are used as weights. The horizontal and vertical components of the design coordinates of each control point are then weighted and summed separately to obtain the result. The mapping design coordinates are used. Natural neighborhood interpolation assigns weights of 1 (the weight of that control point) and 0 (the weights of other control points) at each control point, thus achieving accurate interpolation at those points. Within the area enclosed by the control points, the mapping smoothly transitions using Voronoi area weights, possessing the ability to describe local nonlinear deformations in space. Under the condition of uniformly distributed control points around the BGA package, the correlation error achieved is comparable to that of thin-plate splines. Natural neighborhood interpolation does not require solving the thin-plate spline equations. When the number of control points does not exceed a preset number, conventional Voronoi partitioning or Delaunay triangulation algorithms can be used to complete the calculation on an industrial computer, meeting production line cycle time requirements. The set of control points for automatic optical inspection is used. Using the same natural neighborhood interpolation method as input, an automatic optical detection coordinate mapping function is constructed to obtain... and .

[0037] The second step involves further detecting auxiliary pads in the AOI image and establishing an AOI coordinate mapping function. Specifically, all auxiliary pads are detected in the AOI image. After reflow soldering, auxiliary pads are not printed with solder paste or have components mounted, leaving their copper surfaces exposed, appearing as bright circular areas in the AOI image. However, auxiliary pad identification is not limited to brightness alone. In actual detection, candidate image sub-regions for auxiliary pads are first determined based on the initial affine transformation of the AOI and the design coordinates of the auxiliary pads. Then, within these candidate image sub-regions, grayscale differences, edge closure, contour size, contour shape, and deviation from the design position are considered to identify the auxiliary pads. When the surface treatment or lighting conditions of the auxiliary pad result in lower brightness than the surrounding area, contour size and design position constraints are used for identification. If the identification result of an auxiliary pad does not meet the size or position deviation conditions, that auxiliary pad is not included in the AOI control point set. The initial affine transformation of the AOI is used... ( The construction method is described in the subsequent description of target solder joint coordinate extraction. It is constructed only once in the actual process and is used simultaneously for auxiliary pad detection and target solder joint sub-region positioning (it is directly reused here). This is combined with the auxiliary pad design coordinates. Based on the geometric dimensions, the image sub-region of each auxiliary pad is determined in the automatically optically inspected image. Median filtering (3×3 pixel kernel), Otsu threshold binarization (threshold automatically calculated from the sub-region grayscale histogram), morphological closing operation (3×3 square kernel), and Canny edge detection are then performed sequentially on the sub-region to extract the pixel set of the pad's outer contour. The geometric center is then calculated to obtain the image coordinates. The platform coordinates were obtained after pixel resolution conversion and distortion correction. Paired with the auxiliary pad design coordinates, they form a set of automatic optical inspection control points: ; In this specific example =12; After reflow soldering, the BGA packaged device body covers all the pin pads below it, and this area is not visible in the top view image; the surface of the ordinary pads has formed a solder joint and is no longer in a bare copper state, so it cannot be used as a control point. Therefore, the control points in the automatic optical inspection stage only come from the auxiliary pads, which is fundamentally different from the situation in the solder paste inspection stage where the control points include both auxiliary pads and occasional bare copper ordinary pads.

[0038] With control point set For interpolation nodes, used with Establishing an automatic optical detection coordinate mapping function using the exact same thin-plate spline method. and The construction method is described in the thin-plate spline construction section of this specification. To ensure that the number of control points in the AOI stage is not less than 9 (AOI stage control points are derived only from auxiliary pads), the total number of auxiliary pads is required to be... ≥9, meaning that when auxiliary pads are arranged in each of the four directions of the BGA package boundary, it is recommended to have at least 3 in each direction (e.g., 3 in each direction in this specific example, for a total of 12). When the total number of auxiliary pads is less than 9, the interpolation residual of AOI coordinate mapping will increase significantly due to insufficient control point density, and the cross-process correlation error may exceed [the limit]. Tolerance should be considered, and the number of auxiliary pads should be increased.

[0039] It should be noted that the k-th target pad and k-th target solder joint do not indicate that they have been confirmed as the same solder joint before cross-process association, but rather describe the calculation process for a single candidate matching pair. In actual execution, the SPI detection result set and AOI detection result set are first obtained. Each target pad in the SPI detection result set is mapped to the design coordinate system, and each target solder joint in the AOI detection result set is mapped to the design coordinate system. For an AOI target solder joint, the distance between its AOI-mapped design coordinates and the SPI-mapped design coordinates of multiple SPI target pads is calculated, and the SPI target solder joint with the smallest distance is selected as the candidate matching object. When the minimum distance does not exceed the association distance threshold, the AOI target solder joint and the candidate matching object are determined to be the same solder joint; when the minimum distance exceeds the association distance threshold, it is marked as association uncertain. When there are two or more candidate SPI target solder joints whose distances do not exceed the association distance threshold, the minimum distance and the second smallest distance are compared; if the difference between the two is less than a preset information interval, it is marked as association uncertain to avoid erroneous association in densely packed pad areas.

[0040] The third and final step involves extracting the target coordinates, calculating the cross-process correlation distance, and determining the source of the same weld point and defects. Specifically, for the first... Target pads ( ), Given the total number of target solder pads to be inspected in this batch, extract the centroid coordinates of the solder paste contour from the solder paste inspection image. Utilize the initial affine transformation for solder paste inspection. (The coordinates of the board corner positioning marks in the solder paste inspection image are fitted with their design coordinates using the least squares method, and the construction method is the same as...) (Similarly) The image sub-region is determined by combining the design coordinates and size of the target pad. In the SPI image, the target pad area is a bright area due to the scattering of illumination light by the printed solder paste particles, while the printed circuit board substrate (solder resist surface) is a low-brightness area. The gray-level histogram of the sub-region also shows a bimodal distribution of high and low brightness, which is different from the bimodal distribution of the bare copper pad area (the latter is due to the high reflectivity of the copper surface corresponding to the high brightness peak), but the bimodal characteristics are the same. The Otsu method can automatically determine the optimal threshold to separate the two peaks in both scenarios. Median filtering (3×3 pixel kernel), Otsu threshold binarization (the threshold is automatically calculated by the gray-level histogram of the sub-region to separate the bright area of ​​the solder paste from the substrate area), morphological closing operation (3×3 square kernel to fill the internal holes of the solder paste outline), and Canny edge detection are performed sequentially on the sub-region to extract the pixel set of the outer contour of the solder paste and calculate the geometric center of the pixel coordinates to obtain the image coordinates. The platform coordinates were obtained after pixel resolution conversion and distortion correction. Substitute into the solder paste detection coordinate mapping function ; Extracting the first from an automated optical inspection image The centroid coordinates of the solder contour of each target solder joint are extracted using a method described later in this manual, which provides the bench coordinates. Substitute into the automatic optical detection coordinate mapping function ; Calculate the distance between processes .

[0041] Association distance threshold Based on the target pad package specifications: for the 0201 package, the center-to-center distance between adjacent pads is approximately 0.3 mm, and the thin-plate spline mapping residual is approximately 10 μm to 30 μm. =0.08mm; at this point, the associated distance of a correctly matched pad is usually no more than 40μm, while when mismatched to an adjacent pad, the minimum possible distance between the two mapped design coordinates is approximately 0.3mm minus the 30μm residual on each side, which is approximately 240μm, far exceeding... =0.08mm, which can reliably distinguish between correct and incorrect associations. For 0402 packages (interval between adjacent pads is approximately 0.5mm), it can be taken as... =0.12mm, when the spacing is greater than that of a 0402 package, it is calculated as approximately 26% to 30% of the pad spacing. For example... Figure 3As shown in the figure, this diagram illustrates the relationship between the cross-process coordinate correlation error and the distance from the BGA boundary when using traditional affine transformation and the thin-plate spline mapping of this invention in the area surrounding the BGA. The error of traditional affine transformation increases significantly with increasing distance, reaching levels close to or exceeding the safety tolerance around the BGA; the error of the thin-plate spline mapping of this invention remains consistently below the safety tolerance.

[0042] when At that time, the judgment of the first If the solder paste inspection result and the corresponding automatic optical inspection result belong to the same solder joint, the correlation is established, and the process proceeds to the defect source determination procedure. The determination method is described later in this manual in the defect source determination description. If the solder joint is marked as having uncertain association, it will be subject to manual review.

[0043] In this specific example, cross-process correlation verification was performed on 50 0201 solder joints around the BGA: the average correlation coordinate deviation of the thin plate spline mapping method was approximately 21 μm, with a maximum of approximately 28 μm, both lower than... =0.08mm, no error association; the average coordinate deviation of the affine transformation method based solely on the board corner positioning mark is about 58μm, and the maximum is about 72μm, with a risk that the correlation deviation of the three pads is close to the tolerance.

[0044] Example 2: Based on Example 1, this example elaborates on the method for detecting pads in a bare copper state in an SPI image, including: An initial affine transformation of the SPI image is established by using the image coordinates of the corner positioning marks on the printed circuit board surface and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of each pad, the image sub-region of the i-th pad is determined in the SPI image. The SPI image is binarized to obtain a binary image of the SPI coverage. The number of solder paste pixels in the i-th pad's image sub-region and the total number of pixels in the SPI coverage binary image are counted to obtain the solder paste coverage of the i-th pad. When the solder paste coverage of the i-th pad is lower than the bare copper determination threshold, the i-th pad is marked as a bare copper pad, and auxiliary pads are also included in the bare copper pad category. Based on the initial affine transformation and the design coordinates and dimensions of each bare copper pad, the image sub-region of each bare copper pad is determined in the SPI image. Contour extraction is performed on the image sub-region of each bare copper pad to obtain the SPI image coordinates of each bare copper pad.

[0045] To better understand the specific implementation of this embodiment, exemplarily, an initial affine transformation for solder paste detection is established using the coordinates of the corner positioning marks on the printed circuit board surface in the solder paste inspection image and their design coordinates. The positioning markers are circular bare copper pads, typically located at the corners of the printed circuit board, and their design coordinates are read from the design file. Median filtering, Otsu threshold binarization, and Canny edge detection are sequentially performed on the solder paste inspection image to extract the circular contour of each positioning marker. The geometric center of the contour is calculated to obtain the image coordinates of each marker, and the bench coordinates are obtained after pixel resolution conversion and distortion correction. The bench coordinates of each positioning marker are paired one-to-one with their design coordinates, and the affine transformation matrix parameters are fitted using the least squares method, covering three types of linear transformations: translation, rotation, and uniform scaling. Using three or more positioning markers can reduce the impact of individual marker extraction errors using least squares fitting. It is only used for coarse localization of subsequent image sub-regions, and not for final cross-process association.

[0046] For the first part of the design document Each pad ( , (Total number of pads), using Pad design coordinates Mapping to the coordinates of the solder paste inspection image, and combining this with the pad geometry, determines the first... Image sub-region of each pad in the solder paste inspection image (A rectangular frame centered on the mapped coordinates and with a side length of 120% of the pad's outer diameter, with an expansion margin to accommodate the initial affine transformation estimation error).

[0047] The entire solder paste inspection image is globally binarized using the Otsu thresholding method, and the threshold is... The binary image of solder paste detection coverage is automatically calculated from the grayscale histogram of the entire image. (The pixel value for the solder paste highlight area is 1, and the pixel value for the bare copper and substrate areas is 0). The purpose of global binarization is to calculate the coverage of each pad in batches, and the binarization accuracy requirement is lower than that of the subsequent contour extraction step.

[0048] exist The Middle Each solder pad area Within, count the number of pixels in the solder paste area. (Number of pixels with a value of 1 within the sub-region) and the total number of pixels in the sub-region Calculate the first Solder paste coverage of each pad ; The value range is [0,1].

[0049] when ( When the threshold for bare copper determination is set, the first... Each pad is marked as a bare copper pad. This specific example uses... =0.10, based on: when the solder paste coverage is less than 10%, the main copper surface of the solder pad is exposed, and the error of the center coordinates of the solder pad obtained by subsequent precise contour extraction is less than 5μm, which meets the control point accuracy requirements; when When the value is ≥0.10, the solder paste obscures the edges of the pads, reducing the accuracy of contour extraction, and it is not suitable for use as a control point. It can be adjusted within the range of 0.05 to 0.15 according to the actual printing quality distribution on the production line. When the value is below 0.05, some pads with a very small amount of solder paste splatter particles (the copper surface is not truly completely exposed) may be incorrectly included in the control points. The contour extraction results of such pads are affected by the solder paste particles, resulting in a larger error in the center coordinates and affecting the accuracy of the control points. Therefore, It should not be lower than 0.05.

[0050] It should be noted that for ordinary pads marked as bare copper pads, before being included in the SPI control point set, the extracted exposed metal contour is further matched with the geometry of the corresponding pad in the design file. This matching includes deviations in the contour area from the designed pad area, deviations in the contour's circumscribed dimension from the designed pad size, and deviations in the contour's roundness or aspect ratio from the designed shape. Only when all of the above deviations are within a preset allowable range is the ordinary pad included as an SPI control point; otherwise, the ordinary pad is removed from the SPI control point set. Auxiliary pads, because they are designed not to have solder paste printed on them and not to mount components, are given priority as SPI control points.

[0051] Simultaneously, all auxiliary pads selected from the design documents will be directly incorporated into the bare copper pad set, and the auxiliary pads will not have solder paste printed on them. The solder paste is added directly to the design file without any opening marks on the solder paste layer, without the need for coverage judgment, ensuring stable inclusion of auxiliary pads.

[0052] For each pad in the bare copper pad set, use Based on the pad design coordinates and geometric dimensions, the image sub-regions containing each bare copper pad are determined in the solder paste inspection image (the outward expansion allowance is taken as 10% of the pad's outer diameter). Contour extraction is then performed on these sub-regions to obtain the solder paste inspection image coordinates for each bare copper pad. The extraction method is described in the subsequent description of bare copper pad contour extraction in this manual.

[0053] Example 3: Based on Example 2, such as Figure 4As shown, this figure illustrates the process of SPI image processing, sequentially displaying the original sub-region grayscale image, the image after median filtering and denoising, the adaptive binarized image, the image after morphological closing operation to fill internal holes, the pad outer contour obtained from edge detection, and the final result of superimposing the contour and geometric centroid (red cross). This demonstrates how the SPI stage extracts the center coordinates of bare copper pads / auxiliary pads from the image, serving as control points for subsequent thin-plate spline mapping. This embodiment details the method for extracting the contours of each bare copper pad's image sub-region to obtain the SPI image coordinates of each bare copper pad, including: The bare copper pads are subjected to median filtering in the image sub-region to obtain a filtered bare copper image. The filtered bare copper image is then binarized using the Otsu thresholding method, with the Otsu threshold automatically calculated from the grayscale histogram of the filtered bare copper image, resulting in a binary bare copper image. Morphological closing operations are performed on the binary bare copper image to fill the internal holes of the contour, resulting in a closed bare copper image. Canny edge detection is applied to the closed bare copper image to extract the pixel set of the outer contour of the pads. The geometric center is calculated based on the coordinates of each pixel in the pixel set of the outer contour of the pads, resulting in the SPI image coordinates of the bare copper pads.

[0054] To better understand the specific implementation of this embodiment, by way of example, the first... The following steps are performed sequentially on each sub-region of the image containing the bare copper pads. Specifically, median filtering (3×3 pixels for the filter kernel) is applied to the sub-region image to obtain the filtered bare copper image. Median filtering replaces each pixel with the median of the gray values ​​of all pixels in its 3×3 neighborhood, effectively suppressing salt-and-pepper noise and sensor thermal noise in image acquisition while preserving gray-level abrupt changes at pad edges.

[0055] Using Otsu thresholding method Perform adaptive binarization and thresholding. Depend on The grayscale histogram is automatically calculated using the Otsu method (which determines the optimal segmentation threshold by maximizing the inter-class variance) to obtain a binary image of bare copper. (The pixel value of the bare copper pad area is 1, and the pixel value of the substrate area is 0).

[0056] right Perform morphological closing operations (structuring element is a 3×3 pixel square) to obtain the bare copper closing operation image. Morphological closing operations sequentially perform dilation and erosion to fill grayscale voids within the pad area caused by uneven local lighting or copper oxidation, making the pad area appear as a connected solid region in the binary image. For auxiliary pads with a diameter exceeding 0.8mm (exceeding 80 pixels at 10μm / pixel resolution), the structuring element can be enlarged to a 5×5 pixel square to ensure complete filling of internal voids. Another method for filling internal voids: After inversion, the flood fill algorithm is used to mark all reachable connected background regions from the image boundary inwards, and then inverted again. This marks the background regions in the image that cannot be reached from the boundary (i.e., internal holes) as foreground, which is equivalent to the filling effect of morphological closing operation.

[0057] right The Canny edge detection operator (low threshold is 10% of the maximum grayscale value of the image, high threshold is 25%) is applied to extract the pixel set. We select the outer contour pixel subset of the connected region with the largest area, excluding noisy connected regions with smaller areas. We calculate the arithmetic mean of the coordinates of each pixel in the outer contour pixel set, transform it to the coordinate system of the complete solder paste detection image, and obtain the first... Coordinates of solder paste inspection image for each bare copper pad The reason for using the above thresholds for binary images (pixel values ​​of 0 or 255) is that the Canny operator first performs Gaussian smoothing on the input image. After Gaussian smoothing, the gradient magnitude in the edge region is usually between 20 and 200. The low threshold of 25.5 is below the lower limit of this range, ensuring that all edge candidates enter the non-maximum suppression stage; the high threshold of 63.75 is much lower than the gradient peak value, ensuring that the edge connected chains are completely preserved. The foreground region (pad or solder outline) has been connected by morphological closing operation, and the edge connectivity chain is complete. The Canny output is the set of boundary pixels of the pad or solder outline.

[0058] Example 4: Based on Example 1, this example elaborates on the methods for establishing a coordinate mapping from SPI bench coordinates to design coordinates using the SPI control point set as interpolation nodes to obtain the SPI coordinate mapping function, and for establishing a coordinate mapping from AOI bench coordinates to design coordinates using the AOI control point set as interpolation nodes to obtain the AOI coordinate mapping function, including: Using the SPI platform coordinates of each control point in the SPI control point set as interpolation nodes, thin-plate spline interpolation functions are established for the horizontal and vertical directions respectively; through the first... The control point and the first Substituting the Euclidean distance between the SPI platform coordinates of each control point into the basis function, the resulting basis function value is used as the kernel matrix. Line 1 The elements of the column are used to construct the kernel matrix, when equal The corresponding elements are set to 0; an affine matrix is ​​constructed using the horizontal and vertical components of the SPI bench coordinates of each control point, and a constant term with a value of one; the kernel matrix, affine matrix, transpose of the affine matrix, and zero matrix are combined to obtain the coefficient matrix; the horizontal component of the pad design coordinates of each control point is used as the first segment of the right-hand vector, and the remaining components in the right-hand vector are set to 0, forming a system of block linear equations with the coefficient matrix, and the weight coefficients and affine parameters of each control point in the horizontal direction are obtained by solving the system; the first segment of the right-hand vector is replaced with the vertical component of the pad design coordinates of each control point, and the weight coefficients and affine parameters of each control point in the vertical direction are obtained by solving the system with the same coefficient matrix; the Euclidean distance between the bench coordinates to be mapped and the bench coordinates of each control point is substituted into the basis function, and the corresponding affine parameters are superimposed after weighted summation with the horizontal and vertical weight coefficients respectively, to obtain the mapped design coordinates in the horizontal and vertical directions, which together constitute the SPI coordinate mapping function; the AOI coordinate mapping function is obtained through the AOI control point set using the same method.

[0059] To better understand the specific implementation of this embodiment, exemplarily, a set of solder paste inspection control points is used. As interpolation nodes, respectively for horizontal Direction and vertical The direction establishes the thin-plate spline interpolation function. Specifically, it constructs... Kernel matrix, number The elements are: ， ; These are the basis functions for thin plate splines (see the subsequent description of basis functions in this specification). ， when hour ， make .

[0060] structure Affine matrix, the first Behavior The three columns correspond to the constant term, the horizontal component of the test bench coordinates, and the vertical component of the test bench coordinates, respectively, and are used to constrain the linear (affine) components in the mapping function. , , ( (transpose) and Zero matrix Combined into Coefficient matrix: ; The former Before departure Columns are composed of kernel matrices The structure corresponds to the interpolation constraints of the control points; the last 3 rows and last 3 columns are composed of affine matrices. Its transpose constitutes the corresponding thin-plate spline affine energy minimization constraint (requiring the weight coefficients to satisfy zero sum and first moment conditions, so that the mapping degenerates into a linear affine transformation in the region without data constraints).

[0061] Construct the right-hand vector for the horizontal coordinate direction. :forward The first component is taken from the design coordinate horizontal value of each control point, and the last three components (corresponding to...) Zhongyu The corresponding three rows (i.e., the affine energy constraint rows) are set to 0: ; Solve the system of linear equations The solution vector is obtained. ,in ( () represents the horizontal weighting coefficient. , , For the affine parameters in the horizontal direction.

[0062] Construct the right-hand vector for the vertical coordinate direction. With the same coefficient matrix Solve ,have to , The vertical weighting coefficient. , , For the vertical affine parameters. It depends only on the control point platform coordinates, and is shared in both horizontal and vertical directions, allowing for... After performing an LU decomposition, and Substitute back into the original code separately to improve computational efficiency.

[0063] When not all control points are collinear It is reversible, and the system of equations has a unique solution; requiring control points to cover the four directions of the BGA package boundary ensures that the set of control points forms a non-degenerate convex hull, avoiding collinearity. When When ≤200, The matrix size does not exceed 203×203, and the LU decomposition and back-substitution solution time on an industrial control computer with a main frequency of not less than 2GHz is less than 1 second, which meets the production line cycle time requirements.

[0064] The coordinate mapping function for solder paste inspection is defined as: for any coordinate system of the solder paste inspection bench The horizontal and vertical mapping results are as follows: ; .

[0065] For any control point ,have and The mapping error at the control points is 0. Within the region enclosed by the control points, the mapping is smoothly interpolated using the principle of minimum bending energy, which can effectively describe the transition region of the BGA copper plating boundary. Local nonlinear in-plane residual strain of magnitude.

[0066] Using the same construction method, with an automated optical detection control point set For input, use the platform coordinates. Design coordinates Number of control points Weighting coefficients Affine parameters Replace with , , , , Kernel matrix elements ,in The automatic optical detection coordinate mapping function is obtained. and .

[0067] Example 5: Based on Example 4, this example elaborates on the method described in the first example. The control point and the first Methods for substituting the Euclidean distance between the SPI platform coordinates of each control point into the basis function include: Let the first The SPI bench coordinates of each control point are: , No. The SPI bench coordinates of each control point are: Euclidean distance between two points for: ; when When it is greater than 0, by substituting the basis function Obtain the basis function values; when When the value is 0, the basis function value is 0.

[0068] To better understand the specific implementation of this embodiment, an exemplary set of solder paste inspection control points is provided. The Middle Control point platform coordinates With the Control point platform coordinates Calculate the Euclidean distance between two points: .

[0069] when When substituting the thin-plate spline basis function... ;when (Right now When considering the distance between the same control point and itself, L'Hôpital's rule applies: ; Therefore, the definition That is, the kernel matrix diagonal elements This definition ensures that the kernel matrix has good conditional properties in numerical computation and does not affect the solvability of linear equation systems.

[0070] Set of automatic optical inspection control points The Middle Control point platform coordinates With the Control point platform coordinates The Euclidean distance is Substitute the same basis functions (when (when), or take 0 (when) (Time), construct an automated optical detection kernel matrix elements .

[0071] It should be noted that the distance substituted into the basis function is a normalized distance. Specifically, the actual Euclidean distance between control points is divided by a preset length reference to obtain a dimensionless distance, which is then substituted into the thin-plate spline basis function. The preset length reference can be 1 mm, the average spacing between control points, or the average spacing of auxiliary pads on the outer side of the BGA package boundary. When using the same length reference to process the SPI control point set and the AOI control point set, the precise interpolation relationship at the control points is not changed; it is only used to make the input of the logarithmic term dimensionless.

[0072] Example 6: Based on Example 1, as follows Figure 5 As shown, this figure illustrates the process of AOI image processing. It sequentially presents the original sub-region grayscale image (including highlight spots on the solder spherical cap surface), the image after median filtering to suppress the highlight spots, the adaptive binarized image, the image after morphological closing operation, the solder outer contour obtained from edge detection, and the final result of superimposing the contour and geometric centroid (red cross). This demonstrates how the AOI stage extracts the solder contour centroid coordinates from the image for cross-process correlation with the SPI results in the design coordinate system. This embodiment details the method for extracting the solder contour centroid coordinates of the target solder joint from the AOI image, including: An initial affine transformation for AOI is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the AOI image and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of the target solder joint, the image sub-region of the target solder joint in the AOI image is determined. Median filtering is applied to the image sub-region of the target solder joint to suppress the noise of speckled spots on the solder spherical surface, resulting in a solder-filtered image. The solder-filtered image is binarized using the Otsu thresholding method to separate the solder area from the printed circuit board substrate area. The Otsu threshold is automatically calculated from the grayscale histogram of the solder-filtered image, resulting in a solder binary image. Morphological closing operations are performed on the solder binary image to obtain a solder closing operation image. Canny edge detection is applied to the solder closing operation image to extract the pixel set of the solder outer contour. The geometric center is calculated based on the coordinates of each pixel in the solder outer contour pixel set, resulting in the centroid coordinates of the solder contour of the target solder joint.

[0073] To better understand the specific implementation of this embodiment, for example, the initial affine transformation for automatic optical inspection is established using the image coordinates of the corner positioning marks on the printed circuit board surface in the automatic optical inspection image and their design coordinates. The fitting method is the same as the initial affine transformation of solder paste detection. The construction method is the same. When performing automated optical inspection on a printed circuit board, the image is constructed only once. It is used both to determine the target solder joint image sub-region in this step and to determine the auxiliary solder pad image sub-region mentioned above. The same image is reused in both places. Example. It is only used for coarse localization of image sub-regions and not for final cross-process association.

[0074] use Combined with the The design coordinates and pad dimensions of each target solder joint are used to determine the image sub-region of the target solder joint in the automated optical inspection image. (A rectangle centered on the mapped coordinates, with sides equal to 120% of the pad design size). Sub-regions Perform the following steps in sequence: Median filtering (3×3 pixel kernel) is applied to the sub-region image to obtain the solder-filtered image. After reflow, the solder joints form a spherical shape, with the top exhibiting strong specular reflection from the illumination source, creating isolated bright spots in the image. If the original image is directly binarized, these bright spots are classified as part of the solder joint area, causing the center of gravity of the contour to shift towards the bright spot, introducing a positional error. Median filtering replaces the isolated bright spots with the median gray level of the neighborhood, effectively removing isolated spots while preserving the gray-level transitions at the outer contour edge of the solder joint.

[0075] Using Otsu thresholding method Perform adaptive binarization and thresholding. Depend on The grayscale histogram is automatically calculated using the Otsu method, separating the solder area (high brightness) from the printed circuit board substrate area (low brightness) to obtain a binary image of the solder. (Pixel value of solder area is 1, pixel value of substrate area is 0). Under the ring LED illumination or coaxial illumination conditions of automatic optical inspection equipment, the brightness of solder joints is significantly higher than that of the substrate, and the image grayscale histogram shows a bimodal distribution. The Otsu method can automatically determine the optimal threshold for separating the two peaks.

[0076] right Perform morphological closing operations (3×3 pixel square kernel) to fill isolated dark areas that may still remain inside the solder outline after the bright spots have been smoothed by median filtering, thus obtaining the solder closing image. .

[0077] right The Canny edge detection operator (low threshold is 10% of the maximum grayscale value of the image, high threshold is 25%) is applied to extract the pixel set of the outer contour of the solder. Take the outer contour of the connected region with the largest area, and exclude noisy connected regions with smaller areas (such as residual solder paste in neighboring areas). Calculate The arithmetic mean of the coordinates of each pixel in the image is converted to the coordinate system of the complete automatic optical inspection image to obtain the centroid image coordinates of the solder contour of the target solder joint. The platform coordinates were obtained after pixel resolution conversion and distortion correction: .

[0078] Example 7: Based on Example 1, this example elaborates on the method for determining the source of solder joint defects of a target solder joint based on the solder paste coverage state of the target pad in the SPI image and the solder coverage state of the target solder joint in the AOI image, including: The measured solder paste coverage area of ​​the target pad is obtained from the SPI image. The designed area of ​​the target pad is read from the design file. The solder paste area coverage ratio is obtained from the measured solder paste coverage area and the designed pad area. The measured solder joint coverage area of ​​the target solder joint is obtained from the AOI image. The measured solder joint coverage area is compared with the solder joint area qualification threshold. When the measured solder joint coverage area is lower than the solder joint area qualification threshold, it is determined that the target solder joint has a solder shortage defect. The solder paste area coverage ratio is further compared with the solder paste quantity qualification ratio threshold. When the solder paste area coverage ratio is lower than the solder paste quantity qualification ratio threshold, the defect source is determined to be insufficient printing. When the solder paste area coverage ratio is not lower than the solder paste quantity qualification ratio threshold, the defect source is determined to be poor reflow.

[0079] To better understand the specific implementation of this embodiment, by way of example, the first step is to obtain the solder paste inspection image. Measured solder paste coverage area of ​​each target pad The calculation method is described in the subsequent description of solder paste area calculation in this manual. (The text then abruptly shifts to a different topic: "Read from the design file...") Pad design area for each target pad Calculate the solder paste coverage ratio. ; During normal printing, the value is typically around 0.75 to 1.10 (an upper limit exceeding 1 indicates that the solder paste printing has extended beyond the pad design area). The first image is obtained from an automated optical inspection image. Measured coverage area of ​​the weld points of each target weld point The calculation method is described in the subsequent description of solder area calculation in this manual.

[0080] Will With the acceptable threshold for solder joint area Compare, The setting method is as follows: for the 0201 package (pad design area) in this specific example According to the IPC-A-610 standard The production line process team can also adjust based on historical good product data. .

[0081] when At that time, the judgment of the first The target solder joints have insufficient solder defects, and further... The threshold for the qualified ratio of solder paste amount Comparison: When When the solder paste printing amount is insufficient, the defect is determined to be due to insufficient printing; when At that time, the solder paste printing volume was sufficient, but the solder area after reflow was below standard, leading to the conclusion that the defect originated from poor reflow. This specific example uses... =0.80, based on: Under normal reflow conditions for the 0201 package, the solder body coverage area is basically equivalent to the printed solder paste area (the expansion amount usually does not exceed 10%); if ≥0.80 indicates sufficient solder paste volume; if the solder joint area is below standard, it is attributed to the reflow process. If the solder paste volume is less than 0.80, it is insufficient and is the direct cause of the solder area not meeting the standard. The classification accuracy can be optimized within the range of 0.70 to 0.85 based on production line statistics. When the ratio is below 0.70, pads with a solder paste coverage ratio below 0.70 (corresponding to edges with significantly insufficient printing coverage) and pads with sufficient solder paste but poor reflow risk overlapping at the classification boundary. Actual production line statistics show that setting the lower limit to 0.70 can avoid misclassifying such insufficiently printed pads as having poor reflow. It should not be lower than 0.70.

[0082] when At that time, the first If the area of ​​a solder joint is acceptable and there is no solder shortage defect, skip the defect source determination and proceed to the next solder joint. The defect source determination process is illustrated using a solder shortage defect as an example.

[0083] It should be noted that the defect source determination results are used to provide common process troubleshooting directions. For insufficient solder defects, when the measured coverage area of ​​the solder joint is lower than the acceptable threshold for solder joint area and the solder paste area coverage ratio is lower than the acceptable percentage threshold for solder paste amount, insufficient solder paste printing is the preferred source. When the solder paste area coverage ratio is not lower than the acceptable percentage threshold for solder paste amount, and the component placement position meets the acceptable placement offset condition, the pad contamination detection result meets the acceptable condition, and the SPI detection does not indicate abnormal solder paste height, the defect source is determined to be reflow failure. If any of the above additional conditions are not met, the solder joint is marked as the source for re-inspection.

[0084] Example 8: Based on Example 7, this example elaborates on the method for obtaining the measured solder paste coverage area of ​​the target pad based on the SPI image, including: An initial affine transformation of the SPI image is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the SPI image and their design coordinates. Based on the initial affine transformation of the SPI image and the design coordinates and dimensions of the target pad, the image sub-region of the target pad in the SPI image is determined. Median filtering is performed on the target pad image sub-region to obtain a solder paste filtered image. The solder paste filtered image is binarized using the Otsu thresholding method, where the Otsu threshold is automatically calculated from the grayscale histogram of the solder paste filtered image to obtain a solder paste binary image. Morphological closing operation is performed on the solder paste binary image to obtain a solder paste closed image. The number of pixels belonging to the solder paste region in the solder paste closed image is counted to obtain the number of pixels in the solder paste region. The measured coverage area of ​​the solder paste is obtained by dividing the number of pixels in the solder paste region by the square of the SPI pixel resolution.

[0085] To better understand the specific implementation of this embodiment, exemplarily, solder paste is used to detect the initial affine transformation. (The existing bare copper pad inspection description in this manual can be reused) ) combined with the first Based on the design coordinates and geometric dimensions of each target pad, the image sub-region of the target pad in the solder paste inspection image is determined. .

[0086] Pair of sub-regions Median filtering (3×3 pixel kernel) and Otsu threshold adaptive binarization (threshold) are performed sequentially. The solder paste highlights are automatically separated from the substrate area by sub-region grayscale histogram calculation, and morphological closing operation (3×3 pixel square kernel, filling the voids inside the solder paste outline, the voids are caused by the stencil mesh occlusion causing local solder paste absence) is used to obtain the solder paste filtered image. Solder paste binary image Solder paste closed-loop image .

[0087] statistics The number of pixels with a value of 1 is recorded as the number of pixels in the solder paste area. Using solder paste to test pixel resolution Converted to actual area ; In this specific example, we take the 0201 package as an example. When printing is normal ≈680 pixels , Printing is acceptable; insufficient printing ≈560 pixels , The printing was deemed insufficient.

[0088] Example 9: Based on Example 7, this example elaborates on the method for obtaining the measured coverage area of ​​the target solder joint from the AOI image, including: An initial affine transformation for the AOI (Automated Optical Inspection) is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the AOI image and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of the target solder joint, the image sub-region of the target solder joint in the AOI image is determined. Median filtering is applied to the image sub-region of the target solder joint to obtain a filtered solder area image. The filtered solder area image is binarized using the Otsu thresholding method to separate the solder area from the printed circuit board substrate area. The Otsu threshold is automatically calculated from the grayscale histogram of the filtered solder area image to obtain a binary image of the solder area. Morphological closing operation is performed on the binary image of the solder area to obtain a closed solder area image. The number of pixels belonging to the solder area in the closed solder area image is counted to obtain the number of pixels in the solder area. The measured coverage area of ​​the solder joint is obtained by dividing the number of pixels in the solder area by the square of the AOI pixel resolution.

[0089] To better understand the specific implementation of this embodiment, an example is provided using automatic optical detection of the initial affine transformation. (The target solder joint coordinate extraction description already established in this manual can be reused) ) combined with the first The design coordinates and pad geometry of each target solder joint are used to determine the image sub-region of the target solder joint in the automated optical inspection image (compared to the target solder joint coordinate extraction description in this specification). For the same sub-region, the already determined sub-region location parameters can be reused.

[0090] Median filtering (3×3 pixel kernel) and Otsu threshold adaptive binarization (threshold) are performed sequentially on the sub-region. Automatic calculation of the sub-region grayscale histogram to separate the solder area from the substrate area, and morphological closing operation (3×3 pixel square kernel) to obtain the solder area filtered image. Binary image of solder area Solder area closed operation image .

[0091] The above steps correspond to the three steps described in the target solder joint coordinate extraction section of this manual (calculated separately). , , Performing identical image processing operations on the same solder joint and the same image sub-region can, in practice, be combined into a single calculation, simultaneously outputting the centroid coordinates of the solder's outer contour (bench coordinates). Used to calculate the distance between processes. ) and the number of pixels in the solder area (used to calculate the area) ).

[0092] statistics The number of pixels with a value of 1 is recorded as the number of pixels in the solder area. Using automated optical resolution detection Converted to actual area ; In this specific example, the acceptable threshold for solder joint area is... Qualified solder joints ≈1172 pixels Solder joint area is acceptable; solder joints with insufficient solder ≈703 pixels The system was determined to have a tin deficiency defect, and proceeded to the defect source determination stage. and =0.80 indicates the source of the defect.

[0093] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting solder joint defects on circuit boards based on visual recognition, characterized in that, include: The design coordinates of each pad are read from the PCB design file. The design coordinates of auxiliary pads that have no openings in the solder paste layer but have openings in the solder mask layer are extracted. The auxiliary pads are distributed outside the boundary of the BGA package device, and there is at least one auxiliary pad in each of the four directions of the BGA package device boundary. Pads in the bare copper state are detected in the SPI image. The SPI image coordinates of each bare copper pad are converted by pixel resolution and distortion correction to obtain the SPI rig coordinates. The SPI rig coordinates are paired with the design coordinates of each bare copper pad to form the SPI control point set. The coordinate mapping from the SPI rig coordinates to the design coordinates is established using the SPI control point set as the interpolation node to obtain the SPI coordinate mapping function. In the AOI image, auxiliary pads are detected. The AOI image coordinates of each auxiliary pad are converted by pixel resolution and distortion correction to obtain the AOI bench coordinates. The AOI bench coordinates are paired with the design coordinates of each auxiliary pad to form an AOI control point set. The AOI control point set is used as the interpolation node to establish a coordinate mapping from the AOI bench coordinates to the design coordinates, and the AOI coordinate mapping function is obtained. Extract the centroid coordinates of the solder paste outline of the target pad from the SPI image. After pixel resolution conversion and distortion correction, input the results into the SPI coordinate mapping function to obtain the SPI mapping design coordinates of the target pad. Extract the centroid coordinates of the solder outline of the target solder joint from the AOI image. After pixel resolution conversion and distortion correction, input the results into the AOI coordinate mapping function to obtain the AOI mapping design coordinates of the target solder joint. Calculate the cross-process association distance based on the SPI mapping design coordinates and the AOI mapping design coordinates. When the cross-process association distance does not exceed the association distance threshold, it is determined that the detection results of the SPI image and the AOI image belong to the same solder joint. Based on the solder paste coverage state of the target pad in the SPI image and the solder coverage state of the target solder joint in the AOI image, the solder joint defect source determination result of the target solder joint is obtained.

2. The method for detecting circuit board solder joint defects based on visual recognition according to claim 1, characterized in that, The method for detecting pads in bare copper state in an SPI image includes: An initial affine transformation of the SPI image is established by using the image coordinates of the corner positioning marks on the printed circuit board surface and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of each pad, the image sub-region of the i-th pad is determined in the SPI image. The SPI image is binarized to obtain a binary image of the SPI coverage. The number of solder paste pixels in the i-th pad's image sub-region and the total number of pixels in the SPI coverage binary image are counted to obtain the solder paste coverage of the i-th pad. When the solder paste coverage of the i-th pad is lower than the bare copper determination threshold, the i-th pad is marked as a bare copper pad, and auxiliary pads are also included in the bare copper pad category. Based on the initial affine transformation and the design coordinates and dimensions of each bare copper pad, the image sub-region of each bare copper pad is determined in the SPI image. Contour extraction is performed on the image sub-region of each bare copper pad to obtain the SPI image coordinates of each bare copper pad.

3. The method for detecting circuit board solder joint defects based on visual recognition according to claim 2, characterized in that, The method for extracting the contour of each bare copper pad in the image sub-region to obtain the SPI image coordinates of each bare copper pad includes: The bare copper pads are subjected to median filtering in the image sub-region to obtain a filtered bare copper image. The filtered bare copper image is then binarized using the Otsu thresholding method, with the Otsu threshold automatically calculated from the grayscale histogram of the filtered bare copper image, resulting in a binary bare copper image. Morphological closing operations are performed on the binary bare copper image to fill the internal holes of the contour, resulting in a closed bare copper image. Canny edge detection is applied to the closed bare copper image to extract the pixel set of the outer contour of the pads. The geometric center is calculated based on the coordinates of each pixel in the pixel set of the outer contour of the pads, resulting in the SPI image coordinates of the bare copper pads.

4. The method for detecting circuit board solder joint defects based on visual recognition according to claim 1, characterized in that, The methods for establishing a coordinate mapping from SPI bench coordinates to design coordinates using the SPI control point set as interpolation nodes to obtain the SPI coordinate mapping function, and establishing a coordinate mapping from AOI bench coordinates to design coordinates using the AOI control point set as interpolation nodes to obtain the AOI coordinate mapping function, include: Using the SPI rig coordinates of each control point in the SPI control point set as interpolation nodes, thin-plate spline interpolation functions are established for the horizontal and vertical directions respectively; through the first... The control point and the first Substituting the Euclidean distance between the SPI platform coordinates of each control point into the basis function, the resulting basis function value is used as the kernel matrix. Line 1 The elements of the column are used to construct the kernel matrix, when equal The corresponding elements are set to 0; an affine matrix is ​​constructed using the horizontal and vertical components of the SPI bench coordinates of each control point, and a constant term with a value of one; the kernel matrix, affine matrix, transpose of the affine matrix, and zero matrix are combined to obtain the coefficient matrix; the horizontal component of the pad design coordinates of each control point is used as the first segment of the right-hand vector, and the remaining components in the right-hand vector are set to 0, forming a system of block linear equations with the coefficient matrix, and the weight coefficients and affine parameters of each control point in the horizontal direction are obtained by solving the system; the first segment of the right-hand vector is replaced with the vertical component of the pad design coordinates of each control point, and the weight coefficients and affine parameters of each control point in the vertical direction are obtained by solving the system with the same coefficient matrix; the Euclidean distance between the bench coordinates to be mapped and the bench coordinates of each control point is substituted into the basis function, and the corresponding affine parameters are superimposed after weighted summation with the horizontal and vertical weight coefficients respectively, to obtain the mapped design coordinates in the horizontal and vertical directions, which together constitute the SPI coordinate mapping function; the AOI coordinate mapping function is obtained through the AOI control point set using the same method.

5. The method for detecting circuit board solder joint defects based on visual recognition according to claim 4, characterized in that, The passage through the first The control point and the first Methods for substituting the Euclidean distance between the SPI platform coordinates of each control point into the basis function include: Let the first The SPI bench coordinates of each control point are: , No. The SPI bench coordinates of each control point are: Euclidean distance between two points for: ； when When it is greater than 0, by substituting the basis function Obtain the basis function values; when When the value is 0, the basis function value is 0.

6. The method for detecting circuit board solder joint defects based on visual recognition according to claim 1, characterized in that, The method for extracting the centroid coordinates of the solder contour of the target solder joint in an AOI image includes: An initial affine transformation for AOI is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the AOI image and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of the target solder joint, the image sub-region of the target solder joint in the AOI image is determined. Median filtering is applied to the image sub-region of the target solder joint to suppress the noise of speckled spots on the solder spherical surface, resulting in a solder-filtered image. The solder-filtered image is binarized using the Otsu thresholding method to separate the solder area from the printed circuit board substrate area. The Otsu threshold is automatically calculated from the grayscale histogram of the solder-filtered image, resulting in a solder binary image. Morphological closing operations are performed on the solder binary image to obtain a solder closing operation image. Canny edge detection is applied to the solder closing operation image to extract the pixel set of the solder outer contour. The geometric center is calculated based on the coordinates of each pixel in the solder outer contour pixel set, resulting in the centroid coordinates of the solder contour of the target solder joint.

7. The method for detecting circuit board solder joint defects based on visual recognition according to claim 1, characterized in that, The method for determining the source of solder joint defects of a target solder joint based on the solder paste coverage state of the target pad in the SPI image and the solder coverage state of the target solder joint in the AOI image includes: The measured solder paste coverage area of ​​the target pad is obtained from the SPI image. The designed area of ​​the target pad is read from the design file. The solder paste area coverage ratio is obtained from the measured solder paste coverage area and the designed pad area. The measured solder joint coverage area of ​​the target solder joint is obtained from the AOI image. The measured solder joint coverage area is compared with the solder joint area qualification threshold. When the measured solder joint coverage area is lower than the solder joint area qualification threshold, it is determined that the target solder joint has a solder shortage defect. The solder paste area coverage ratio is further compared with the solder paste quantity qualification ratio threshold. When the solder paste area coverage ratio is lower than the solder paste quantity qualification ratio threshold, the defect source is determined to be insufficient printing. When the solder paste area coverage ratio is not lower than the solder paste quantity qualification ratio threshold, the defect source is determined to be poor reflow.

8. The method for detecting circuit board solder joint defects based on visual recognition according to claim 7, characterized in that, The method for obtaining the measured solder paste coverage area of ​​the target pad based on the SPI image includes: An initial affine transformation of the SPI image is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the SPI image and their design coordinates. Based on the initial affine transformation of the SPI image and the design coordinates and dimensions of the target pad, the image sub-region of the target pad in the SPI image is determined. Median filtering is performed on the target pad image sub-region to obtain a solder paste filtered image. The solder paste filtered image is binarized using the Otsu thresholding method, where the Otsu threshold is automatically calculated from the grayscale histogram of the solder paste filtered image to obtain a solder paste binary image. Morphological closing operation is performed on the solder paste binary image to obtain a solder paste closed image. The number of pixels belonging to the solder paste region in the solder paste closed image is counted to obtain the number of pixels in the solder paste region. The measured coverage area of ​​the solder paste is obtained by dividing the number of pixels in the solder paste region by the square of the SPI pixel resolution.

9. The method for detecting circuit board solder joint defects based on visual recognition according to claim 7, characterized in that, The method for obtaining the measured coverage area of ​​the target solder joint based on the AOI image includes: An initial affine transformation for the AOI (Automated Optical Inspection) is established by using the image coordinates of the corner positioning marks on the printed circuit board surface in the AOI image and their design coordinates. Based on the initial affine transformation and the design coordinates and dimensions of the target solder joint, the image sub-region of the target solder joint in the AOI image is determined. Median filtering is applied to the target solder joint image sub-region to obtain a solder area filtered image. The solder area filtered image is binarized using the Otsu thresholding method to separate the solder area from the printed circuit board substrate area. The Otsu threshold is automatically calculated from the grayscale histogram of the solder area filtered image to obtain a solder area binary image. Morphological closing operation is performed on the solder area binary image to obtain a solder area closed operation image. The number of pixels belonging to the solder area in the solder area closed operation image is counted to obtain the number of solder area pixels. The measured coverage area of ​​the solder joint is obtained by dividing the number of solder area pixels by the square of the AOI pixel resolution.

Images