A PCB defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction

Abstract

The application relates to the technical field of three-dimensional measurement, in particular to a PCB defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction, which adopts a double-DLP projector system, obtains high-precision system parameters through a joint calibration method containing complementary Gray codes and phase shift patterns, adopts a heterogeneous coding strategy in the reconstruction stage, the first DLP projects complementary Gray codes and a phase shift sequence, the second DLP projects a frequency division-composite coding sequence, and the absolute phase is decoded by using a self-checking and interpolation repair mechanism; the double-path height maps are reconstructed through triangulation, and the optimal height map is obtained through intelligent fusion based on confidence; finally, the fused height map is matched with a CAD model, the average height, surface area and volume features of the component area are calculated, and the quantitative judgment of height-related defects and consistency defects is realized. The application effectively improves the precision, robustness and automation degree of the three-dimensional defect detection of the PCB.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional measurement technology, and in particular to a method for detecting defects in PCB boards based on three-dimensional reconstruction using multi-source complex frequency structured light. Background Technology

[0002] Printed circuit boards (PCBs) are fundamental components of modern electronic devices, and the assembly quality of their surface mount technology directly affects product performance. During PCB assembly, various defects related to three-dimensional morphology exist, such as component warping, cold solder joints, insufficient solder paste, and component tombstoning. These defects are difficult to effectively identify using traditional two-dimensional visual inspection. Currently, industrial automated optical inspection systems mainly rely on two-dimensional image analysis, which has inherent limitations in inspection capabilities. To acquire three-dimensional information, structured light 3D measurement technology has been introduced, but it faces many challenges in actual PCB inspection production: PCB surfaces have highly reflective pads, components of varying colors, and complex obstructions; single-projector structured light systems are prone to phase decoding errors; existing technologies generally employ a single encoding strategy, resulting in insufficient robustness; system calibration accuracy and reconstruction algorithm accuracy directly affect inspection accuracy; and the automatic judgment from 3D point clouds to defects lacks quantitative and reliable criteria. Therefore, developing a PCB 3D inspection method that can adapt to complex conditions and integrate high-precision calibration, robust 3D reconstruction, and quantitative defect judgment has significant industrial value. Summary of the Invention

[0003] The purpose of this invention is to provide a PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction, which solves the problems of insufficient three-dimensional measurement robustness, low reconstruction accuracy, and low degree of automation in defect judgment when facing PCB surfaces with multiple materials, high reflectivity, and complex obstruction.

[0004] To achieve the above objectives, this invention provides a PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction, comprising the following steps:

[0005] Using dual DLP projectors and a camera, complementary Gray codes and phase-shift patterns are projected, and dual DLP spatial consistency constraints are introduced to jointly optimize and obtain global system parameters. The global system parameters include the intrinsic parameter matrices, distortion coefficients, rotation matrices and translation vectors of the camera and the two DLP projectors, as well as the position parameters of the calibration board in each attitude.

[0006] After obtaining the absolute phase of both channels using a heterogeneous coding strategy, the dual-channel height maps are reconstructed using triangulation. Intelligent fusion based on confidence levels is then performed to output the optimal height map. The intelligent fusion based on confidence levels includes:

[0007] Calculate the difference map between the two height maps; divide the pixels into high-confidence regions and low-confidence regions according to the preset difference threshold; use weighted average fusion in the high-confidence regions; in the low-confidence regions, first use the data from the high-confidence regions to interpolate and generate a trend height map, and then use the original height value with a difference less than the set requirement to perform weighted fusion with the trend height map to obtain the optimal height map;

[0008] The optimal height map is registered with the CAD model of the PCB, the component areas are located, and the average height, surface area and volume characteristics of each component area are calculated. The calculated average height, surface area and volume are compared with the standard value and allowable tolerance range of the corresponding component one by one. If any feature exceeds its tolerance range, the component is determined to have a height-related defect. At the same time, the low confidence areas marked during the fusion process are mapped to the component areas to determine consistency defects.

[0009] This method employs dual DLP projectors and cameras, projecting complementary Gray codes and phase-shifted patterns, and introduces dual DLP spatial consistency constraints to jointly optimize and obtain global system parameters, including:

[0010] A ceramic calibration plate with circular markers is used and placed in multiple different positions;

[0011] A dual-DLP time-division projection of coded pattern sequences in the horizontal and vertical directions, the sequences being composed of N-step phase-shift patterns and M complementary Gray code patterns;

[0012] After the camera acquires images, it extracts the sub-pixel coordinates of the marker points and performs absolute phase decoding on the image sequence of each DLP. The sub-pixel coordinates of the marker points on the projector image plane are calculated by linear interpolation.

[0013] A global optimization objective function containing reprojection error terms and dual-DLP spatial consistency constraint error terms is constructed. The intrinsic and extrinsic parameters and distortion coefficients of the camera and dual-DLP are obtained by iteratively solving the function using a nonlinear optimization algorithm.

[0014] The calculation method for the dual DLP spatial consistency constraint error term is as follows:

[0015] For each calibration board marker point, it is back-projected into three-dimensional space through the parameters of the first DLP. The resulting ray intersects the known calibration board plane at a point. Then, this three-dimensional point is projected onto its image plane through the parameters of the second DLP to obtain the predicted coordinates. The sum of squared errors between the predicted coordinates and the actual coordinates extracted from the second DLP image is calculated and summed over all marker points.

[0016] The global optimization objective function is in the form of: the total error equals the sum of the reprojection error term and the dual DLP spatial consistency constraint error term; where the reprojection error term is the sum of the squared Euclidean distances between all 3D calibration board markers projected back to the camera and dual DLP image plane using the current parameters and the extracted sub-pixel coordinates.

[0017] The formula for determining the sub-pixel coordinates of the projector using the linear interpolation method is as follows: if the absolute phase of a camera pixel falls between the phase values ​​corresponding to adjacent pixel indices in the projector phase diagram, then the sub-pixel coordinates of the projector are calculated by linear interpolation of the phase values.

[0018] Specifically, the heterogeneous coding strategy is as follows: the first DLP projects a complementary Gray code combined with a phase shift sequence; the second DLP projects a frequency division-composite coding sequence, which projects high-frequency sine stripes in the red channel and low-frequency sine stripes in the blue channel, carrying two frequency information simultaneously in a single projection.

[0019] The absolute phase decoding of the second DLP specifically includes: separating the color channels to calculate the high-frequency wrapping phase and the low-frequency wrapping phase; expanding the low-frequency wrapping phase to obtain the low-frequency absolute phase; using the dual-frequency heterodyne principle to calculate the order of high-frequency fringes from the low-frequency absolute phase and the high-frequency wrapping phase to obtain the initial high-frequency absolute phase; calculating the self-checking residual and comparing it with a preset threshold to mark the error points; and using the natural neighborhood interpolation method to repair the error points using the phase values ​​of surrounding reliable points to obtain the corrected absolute phase.

[0020] The average height, surface area, and volume features are calculated as follows: the average height is the mean of the heights of all pixels within the element region; the surface area is the sum of the local micro-element areas calculated based on the height gradient at each pixel within the element region; and the volume is the height integral of each pixel within the element region relative to the reference plane, assuming the PCB substrate is the reference plane.

[0021] The method for determining consistency defects is as follows: the low-confidence regions marked during the fusion process are mapped to the element regions. If the proportion of low-confidence pixel area in a certain element region exceeds a preset threshold, the element is determined to have consistency defects, indicating that the 3D reconstruction result of the region is unreliable.

[0022] This invention discloses a PCB board defect detection method based on multi-source complex frequency structured light 3D reconstruction. It employs a dual-DLP projector system and obtains high-precision system parameters through a joint calibration method involving complementary Gray code and phase-shift patterns. In the reconstruction stage, a heterogeneous coding strategy is used: the first DLP projects complementary Gray code and a phase-shift sequence, while the second DLP projects a frequency-division-composite coding sequence. A self-verification and interpolation repair mechanism is used to decode the absolute phase. Dual-path height maps are reconstructed using triangulation, and intelligent fusion based on confidence levels yields the optimal height map. Finally, the fused height map is registered with a CAD model, and the average height, surface area, and volume characteristics of component regions are calculated to quantitatively determine height-related defects and consistency defects. This invention effectively improves the accuracy, robustness, and automation of PCB board 3D defect detection. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0024] Figure 1 This is a schematic diagram of the steps of a PCB board defect detection method based on three-dimensional reconstruction of multi-source complex frequency structured light provided by the present invention.

[0025] Figure 2 This is a flowchart illustrating step S1 provided by the present invention.

[0026] Figure 3 This is a flowchart illustrating step S2 provided by the present invention.

[0027] Figure 4 This is a flowchart illustrating step S3 provided by the present invention. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0029] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0031] Please see Figures 1-3 This invention provides a PCB board defect detection method based on three-dimensional reconstruction using multi-source complex frequency structured light, comprising the following steps:

[0032] S1. Using dual DLP projectors and cameras, complementary Gray codes and phase-shifted patterns are projected, and dual DLP spatial consistency constraints are introduced to jointly optimize and obtain global system parameters.

[0033] Specifically, a ceramic calibration plate with circular markers is used and placed in multiple (no fewer than 5) different poses within the measurement field of view. For each pose, a first DLP projector and a second DLP projector sequentially project coded pattern sequences in the horizontal and vertical directions, time-division multiplexing. This sequence consists of an N-step phase-shift pattern and M complementary Gray code patterns (e.g., N=8, M=6, including 5 traditional Gray codes and 1 complementary code for boundary error avoidance). A high-resolution industrial camera simultaneously acquires images from all projection states.

[0034] Subpixel-precision coordinates of all circular markers in the camera image are extracted. Then, absolute phase decoding is performed on each image sequence acquired by the DLP projector. The absolute phase decoding process is as follows: first, the wrap-around phase is calculated using the phase-shifting method, and then the absolute phase is obtained by combining the complementary Gray code pattern. For each marker, using its absolute phase value in the camera image, its corresponding subpixel-precision coordinates on the projector image plane are calculated using linear interpolation in the absolute phase distribution map of the projector. The linear interpolation formula is as follows:

[0035] Let the camera pixel point The absolute phase is In the projector phase diagram, this phase value falls at the pixel index. and Between, the corresponding phase values ​​are respectively and ,but Corresponding projector subpixel coordinates for:

[0036] (1)

[0037] After obtaining a large number of two-dimensional corresponding points between the camera and the two DLP projectors, a global optimization objective function is constructed to jointly optimize all intrinsic and extrinsic parameters (including intrinsic parameter matrices, distortion coefficients, extrinsic rotation matrices, and translation vectors) of the camera and the two DLP projectors. Initialization parameter settings are obtained using traditional methods such as Zhang's calibration method. The optimization objective function... Defined as:

[0038] (2)

[0039] Wherein, λ is a weighting coefficient used to balance the magnitude of the reprojection error term and the dual DLP spatial consistency constraint error term. Its value range is usually 0.1 to 10. In this embodiment, λ=1 is taken.

[0040] in, This is the reprojection error term, calculated as the sum of the squared Euclidean distances between the projected 3D calibration board markers back to the camera and two projector image planes using the current parameters and the extracted sub-pixel coordinates:

[0041] (3)

[0042] in, , These are the sub-pixel coordinates extracted from the first DLP and second DLP image planes, respectively.

[0043] The dual DLP spatial consistency constraint error term is the core innovation of the calibration stage of this invention. Its calculation method is as follows: for each marker point, it is back-projected into three-dimensional space using the parameters of the first DLP projector. This ray intersects the known calibration plate plane at a single point. Then this three-dimensional point The predicted coordinates are obtained by projecting the parameters of the second DLP projector onto its image plane. Calculate the predicted coordinates and the actual coordinates extracted from the second DLP image. Sum of squared errors:

[0044] (4)

[0045] By minimizing The Levenberg-Marquardt nonlinear optimization algorithm was used to iteratively solve the problem, and finally, high-precision global system parameters with consistent spatial relationships were obtained.

[0046] like Figure 2As shown, firstly, a high-precision ceramic calibration plate with a circular marker array is prepared. This plate is placed within the common field of view of the dual DLP projectors and the camera, and its transformation into at least five different poses is precisely controlled for image acquisition. For each pose, the first DLP projector (DLP_A) and the second DLP projector (DLP_B) project horizontal and vertical structured light patterns in a time-division manner.

[0047] The pattern sequence for each direction consists of an 8-step phase-shift pattern and 6 complementary Gray code patterns (5 of which are traditional Gray codes and 1 is a complementary code). The camera needs to synchronously trigger acquisition during each projection.

[0048] Using subpixel-level corner detection algorithms from computer vision libraries (such as OpenCV), extract the precise coordinates of all circular marker points in the camera image. Subsequently, absolute phase calculation was performed on the image sequences acquired by each DLP projector.

[0049] Wrap-up phase calculation: Taking the 8-step phase-shifted image of the first DLP projector (DLP_A) as an example, the phase shift of the image captured by the camera in the first step... Light intensity distribution of the image It can be represented as:

[0050] (8)

[0051] in, This represents the total number of phase shift steps. Indicates background light intensity. Indicates modulation intensity. This is the enclosed phase we are looking for. The enclosed phase can be calculated using the standard N-step phase shift formula:

[0052]

[0053] For an 8-step phase shift, this formula can effectively suppress higher-order harmonic noise.

[0054] Complementary Gray code decoding and absolute phase acquisition: to wrap the phase Unfolding into absolute phase This requires the use of projected complementary Gray code sequences. The decoding process is as follows:

[0055] The first five traditional Gray code images are binarized and decoded to obtain an initial periodic series. .

[0056] Adding a sixth complementary Gray code image, we can decode it together to obtain another periodic series. . and They intersect at the phase period boundaries.

[0057] Final absolute phase The following rules are used to avoid using unreliable series at the boundaries:

[0058] (9)

[0059] For each marker point with known world coordinates Using its coordinates on the camera image and the calculated absolute phase value The sub-pixel projector coordinates are obtained by inversely calculating the absolute phase map of the projector using linear interpolation. The specific interpolation formula is consistent with that described in the invention description.

[0060] In obtaining a large amount and After establishing the correspondence, construct the joint optimization objective function as described in the invention. The core idea of ​​dual DLP spatial consistency constraints is that a physical point, regardless of which DLP projector's perspective it is observed from, should ultimately have a unique position projected back into 3D space. This forces the optimization process to find a set of system parameters that ensure the observations of DLP_A and DLP_B converge to the same point in 3D space through the common geometric constraint of the calibration plate plane.

[0061] Specifically, for each marker point The calculation of its consistency constraint error follows the rigorous steps below:

[0062] Ray backprojection from the DLP_A image plane to 3D space: using the DLP_A projector parameters (intrinsic parameters) of the current iteration ,distortion External reference ), and set its image plane coordinates Back-projecting onto three-dimensional space forms a ray. Points on this ray... In the camera coordinate system of DLP_A, it can be represented as:

[0063] (10)

[0064] In the above formula, These are the pixel coordinates after distortion correction. It is the equivalent focal length of DLP_A. These are the coordinates of its principal point. It is the depth parameter on the ray. Then, through the extrinsic parameters of DLP_A Transform the ray to the world coordinate system:

[0065] (11)

[0066] Intersection of the ray and the calibration plate plane:

[0067] The calibration board in the current iteration is in the current pose. Below, its plane equation is derived from the camera's extrinsic parameters. Determined. Plane normal vector. for The third column, a point on the plane for The ray equation Substitute the point-normal equation into the plane equation: Solving this equation yields a unique depth parameter. .Will Substituting back into the ray equation, the three-dimensional intersection point of the ray and the calibration plate plane can be obtained. This point is a three-dimensional point determined jointly by the observations of DLP_A and the calibration plate plane under the current system parameters.

[0068] The three-dimensional intersection points obtained in the previous step Using the DLP_B projector parameters (intrinsic parameters) of the current iteration External reference The predicted pixel coordinates are obtained by projecting the image onto the DLP_B image plane. The projection formula is:

[0069] in It is a scale factor.

[0070] Finally, calculate this predicted coordinate. The corresponding point coordinates extracted and decoded from the DLP_B image The square of the Euclidean distance between them:

[0071] Summing over all marker points yields the consistency constraint error term. .

[0072] The above-constructed With reprojection error Combined, to form the overall objective function .parameter To balance the magnitudes of the two errors, the Levenberg-Marquardt algorithm from a nonlinear optimization library (such as CeresSolver) is used to minimize... Meanwhile, optimize the camera. First projector Second projector All intrinsic and extrinsic parameters and distortion coefficients. This consistency constraint forces the optimization algorithm to find a "globally optimal solution" that not only minimizes the individual reprojection error of each sensor, but also ensures a highly consistent geometric relationship between the observations of the two DLP projectors in three-dimensional space, thereby significantly improving the overall accuracy and reliability of the system parameters.

[0073] S2. After decoding the absolute phase of the two channels using a heterogeneous coding strategy, the dual-channel height map is reconstructed by triangulation and intelligently fused based on confidence level to output the optimal height map.

[0074] Specifically, in the reconstruction phase, a heterogeneous coding strategy is employed to obtain redundant and complementary 3D information. The first DLP projector projects the same complementary Gray code and phase-shift combination sequence as in the calibration phase. The second DLP projector projects a frequency-division-composite coding sequence; for example, high-frequency sinusoidal fringes are projected in the red channel, and low-frequency sinusoidal fringes are projected in the blue channel, allowing both frequency information to be carried simultaneously in a single projection.

[0075] The camera acquires image sequences from two DLP projectors simultaneously. For the first DLP image, a complementary Gray code method consistent with the calibration phase is used for decoding to obtain a robust absolute phase. .

[0076] For images in the second DLP, the decoding process is more complex to address potential errors: first, the color channels are separated, and the high-frequency wrapper phase is calculated from the red channel. Calculate the low-frequency wrap phase from the blue channel Low-frequency wrapping phase Expand to obtain the absolute phase Subsequently, utilizing the built-in dual-frequency heterodyne principle, by and high-frequency wrapping phase Calculate the order of high-frequency fringes The formula is:

[0077] (5)

[0078] in and These represent the frequencies of the high and low frequency stripes, respectively. This allows us to obtain the initial high-frequency absolute phase. .

[0079] To identify decoding errors, calculate the self-checking residual. :

[0080] (6)

[0081] Set threshold ,like If a point is found to be faulty, it is marked as an error point. For all error points, natural neighborhood interpolation is used, utilizing the data from reliable points around the point. The values ​​are repaired, and finally the corrected absolute phase of the second DLP is obtained. .

[0082] In obtaining and Then, using the system parameters obtained from step one, and applying the sub-pixel phase interpolation method again, the sub-pixel coordinates of each pixel in the camera image on the two DLP projectors are calculated. The three-dimensional coordinates of each pixel are then reconstructed using triangulation. In this invention, the focus is primarily on height information; therefore, only the Z coordinate is retained, and height maps from the first DLP are generated. and height map from the second DLP .

[0083] Subsequently, the two independent height maps are intelligently fused:

[0084] 1. For ease of description, let H1 be the height map obtained from the first DLP reconstruction and H2 be the height map obtained from the second DLP reconstruction. Calculate the difference map: .

[0085] 2. Based on the preset threshold Divide the pixels into high-confidence regions and low confidence areas .

[0086] 3. In the high-confidence region, a simple weighted average is used: .

[0087] 4. In the low-confidence region, first use data from the high-confidence region for interpolation (such as natural neighborhood interpolation) to generate a trend height map. Then, calculate. and respectively with Differences and Select the original height value with the smaller difference and denote it as... Finally, a weighted fusion is performed:

[0088] The weight According to and The relative size is dynamically adjusted. = / ( + ).

[0089] The final output is the optimal height map after fusion. .

[0090] like Figure 3 As shown, after calibration, remove the calibration board and place the PCB board to be tested at the measurement station.

[0091] The first DLP projector (DLP_A) projects a complementary Gray code and phase-shift combination sequence that is exactly the same as the calibration phase.

[0092] The second DLP projector (DLP_B) projects a frequency-division-composite coded sequence. In this embodiment, its projection period in the red channel is... The high-frequency sinusoidal fringes, projected in the blue channel, have a period of Low-frequency sine stripes ( Both cameras employ a 4-step phase shift mechanism. The camera acquires all images projected by the two projectors in a time-division manner.

[0093] DLP_A sequence decoding: The decoding process is exactly the same as the calibration stage, ultimately yielding an absolute phase map. .

[0094] DLP_B sequence decoding: The specific steps are as follows:

[0095] Channel Separation and Wrap-up Phase Calculation: The red and blue channels of the DLP_B image sequence acquired by the camera are separated. For each channel's 4-step phase-shifted image, the wrap-up phase is calculated using a 4-step phase-shift formula. (High frequency) and (Low frequency). The formula for calculating the wrapped phase using the 4-step phase shift method is as follows:

[0096] in arrive The light intensity is represented by the four phase-shifted images.

[0097] Low-frequency phase expansion: wrapping the low-frequency phase A robust time-domain expansion is performed to obtain the low-frequency absolute phase. .

[0098] High-frequency phase initial expansion and self-verification:

[0099] Calculate the initial order of high-frequency fringes using the built-in dual-frequency heterodyne principle. :

[0100] (12)

[0101] This leads to the initial high-frequency absolute phase. .

[0102] Calculate self-checking residuals :

[0103] (13)

[0104] Set a threshold (For example ),like If the error is not detected, the pixel is marked as an error. The self-checking residual is the difference between the theoretical high-frequency absolute phase and the high-frequency absolute phase calculated from the low-frequency expansion; its magnitude is related to factors such as system noise and projection defocus. Threshold The residual distribution can be determined based on the statistical distribution of residuals from multiple measurements on the standard plane after system calibration, typically taken as 2 to 3 times the root mean square (RMS) of the residuals. In this example, the measured residual RMS is approximately 0.3 rad, therefore, it is taken as... =0.8πrad (approximately 2.51 rad) is used as a safety margin. In practical applications, it can also be adjusted according to the detection accuracy requirements, and the value range is generally 0.5~1.5π rad.

[0105] For all pixels marked as errors, natural neighborhood interpolation is used, utilizing the values ​​of their surrounding reliable pixels. The values ​​are repaired, and finally a high-quality absolute phase map with DLP_B correction is obtained. .

[0106] 3D Reconstruction and Height Map Generation: In obtaining and Then, using the system parameters obtained from the calibration, each pixel of the camera image is... Perform 3D reconstruction:

[0107] Based on its absolute phase value or The subpixel coordinates were found on the projector phase maps of DLP_A and DLP_B respectively using subpixel phase interpolation. and .

[0108] Based on the camera pixel coordinates and the corresponding projector coordinates, the 3D coordinates are reconstructed using triangulation. This process is mathematically equivalent to solving an overdetermined system. Specifically, for the camera and any projector (taking DLP_A as an example), the following equations exist:

[0109] (14)

[0110] (15)

[0111] in and It is the calibrated 3x4 camera and DLP_A projection matrix. It is a world coordinate system. and It is the scaling factor. By eliminating the scaling factor, at least four linear equations can be constructed and solved using least squares. In this plan, we primarily focus on altitude information, therefore only retain [the relevant information]. coordinate.

[0112] Use respectively and The above reconstruction generates two independent height maps that correspond one-to-one with the camera image pixels. and .

[0113] Intelligent heightmap fusion: Execution as Figure 3 The fusion process is shown.

[0114] Calculate the difference plot: .

[0115] Confidence level division: based on a preset difference threshold (The difference threshold is determined based on the depth measurement noise standard deviation σ obtained after system calibration, generally taken as 3 to 5 times σ, or set as 0.3% to 1.0% of the system measurement range according to the actual measurement accuracy requirements. In this embodiment, the system measurement depth range is ±5mm, and the noise standard deviation is approximately 2μm, so Thresh = 0.5% × 10mm = 50μm is taken. This threshold can also be obtained by calibration using standard step blocks according to the actual scene.) Each pixel is divided into:

[0116] Let H A = H1 represents the height map obtained from the first DLP reconstruction, H B = H2 represents the height map obtained from the second DLP reconstruction. The high-confidence region is: This indicates that the two measurement results are highly consistent. Low confidence region: This indicates that there is significant uncertainty in the measurement at this location.

[0117] Partition merging:

[0118] High-confidence areas: direct averaging fusion is used. To suppress random noise.

[0119] Low-confidence regions: employ more sophisticated restorative fusion.

[0120] a. Trend Height Interpolation: Using height data from high-confidence regions, a smooth trend height map is generated through natural neighborhood interpolation. .

[0121] b. Optimal selection: Calculation and respectively with Differences and Choose the original height value with the smaller difference, and denote it as... .

[0122] c. Weighted fusion: Final fusion height . = / ( + ), weight According to and The relative size is dynamically adjusted; if the difference between the two is significant, then more trust is placed on them. .

[0123] Output the final optimal height map after fusion. .

[0124] S3. Register the optimal height map with the CAD model of the PCB, locate the component area, calculate the average height, surface area and volume characteristics of each component area, and compare the calculated average height, surface area and volume with the standard value and allowable tolerance range of the corresponding component one by one. If any feature exceeds its tolerance range, it is determined that the component has a height-related defect. At the same time, the low confidence area marked during the fusion process is mapped to the component area to determine the consistency defect.

[0125] Specifically, such as Figure 4 As shown, the optimal height map obtained by fusion is first... Precise registration with the standard PCB CAD design model is achieved. This is done by extracting common feature points (such as corner points of specific components, pad centers, etc.) from the CAD model and heightmap, and then calculating an optimal spatial transformation matrix. After registration, each component to be inspected (such as chips, resistors, and capacitors) can be precisely located in the heightmap based on the component layout information in the CAD model. The pixel area covered in .

[0126] For each component area Calculate the following three-dimensional topographic features:

[0127] average height: ,in .

[0128] Surface area: Calculate the area of ​​local infinitesimal elements based on the height gradient at each pixel, and then apply the result to the entire region. Summation. Specifically, for pixels... Its infinitesimal area:

[0129] (7)

[0130] in and It is the physical size of a pixel. and This is the gradient of height in the u and v directions (which can be calculated using the Sobel operator, etc.). The total surface area is... .

[0131] Volume: Assuming the PCB substrate is the reference plane, its height is... The volume of the component is:

[0132] The summation region is .

[0133] Defect assessment is divided into two categories:

[0134] 1. Highly correlated defects: The calculated... , , The value is compared with the pre-defined standard value and allowable tolerance range for this type of component. If any characteristic quantity exceeds the tolerance range, the component is determined to have a highly relevant defect.

[0135] 2. Consistency Defect: Directly marking the low-confidence regions (i.e., those marked in the fusion stage of step two) The region is mapped onto the element region. If a certain element region contains a sufficiently large area (e.g., more than 5% of the element area) of low-confidence pixels, the element is judged to have a consistency defect, indicating that the 3D reconstruction result of the region is unreliable and requires manual review.

[0136] By employing a multi-source heterogeneous coding strategy with dual DLP projectors, the robustness and data integrity of the 3D measurement system in complex industrial scenarios are effectively improved. An innovative joint calibration method, combined with spatial consistency constraints, ensures high precision and global consistency of system parameters. The introduction of sub-pixel phase interpolation technology further enhances the accuracy of 3D reconstruction. A confidence-based intelligent fusion algorithm fully utilizes the redundant information from the dual-channel data and automatically identifies measurement uncertainty areas. Finally, by combining high-precision 3D data with component-level quantitative feature extraction, automated and high-precision detection of 3D defects on PCB boards is achieved, significantly improving detection coverage and reliability.

[0137] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0138] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for PCB board defect detection based on three-dimensional reconstruction using multi-source complex frequency structured light, characterized in that, Includes the following steps: Using dual DLP projectors and a camera, complementary Gray codes and phase-shift patterns are projected, and dual DLP spatial consistency constraints are introduced to jointly optimize and obtain global system parameters. The global system parameters include the intrinsic parameter matrices, distortion coefficients, rotation matrices and translation vectors of the camera and the two DLP projectors, as well as the position parameters of the calibration board in each attitude. After obtaining the absolute phase of both channels using a heterogeneous coding strategy, the dual-channel height maps are reconstructed using triangulation. Intelligent fusion based on confidence levels is then performed to output the optimal height map. The intelligent fusion based on confidence levels includes: Calculate the difference map between the two height maps; divide the pixels into high-confidence regions and low-confidence regions according to the preset difference threshold; use weighted average fusion in the high-confidence regions; in the low-confidence regions, first use the data from the high-confidence regions to interpolate and generate a trend height map, and then use the original height value with a difference less than the set requirement to perform weighted fusion with the trend height map to obtain the optimal height map; The optimal height map is registered with the CAD model of the PCB, the component areas are located, and the average height, surface area and volume characteristics of each component area are calculated. The calculated average height, surface area and volume are compared with the standard value and allowable tolerance range of the corresponding component one by one. If any feature exceeds its tolerance range, the component is determined to have a height-related defect. At the same time, the low confidence areas marked during the fusion process are mapped to the component areas to determine consistency defects.

2. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 1, characterized in that, By employing dual DLP projectors and cameras, complementary Gray codes and phase-shifted patterns are projected, and dual DLP spatial consistency constraints are introduced to jointly optimize and obtain global system parameters, including: A ceramic calibration plate with circular markers is used and placed in multiple different positions; A dual-DLP time-division projection of coded pattern sequences in the horizontal and vertical directions, the sequences being composed of N-step phase-shift patterns and M complementary Gray code patterns; After the camera acquires images, it extracts the sub-pixel coordinates of the marker points and performs absolute phase decoding on the image sequence of each DLP. The sub-pixel coordinates of the marker points on the projector image plane are calculated by linear interpolation. A global optimization objective function containing reprojection error terms and dual-DLP spatial consistency constraint error terms is constructed. The intrinsic and extrinsic parameters and distortion coefficients of the camera and dual-DLP are obtained by iteratively solving the function using a nonlinear optimization algorithm.

3. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 2, characterized in that, The calculation method for the dual DLP spatial consistency constraint error term is as follows: For each calibration board marker point, it is back-projected into three-dimensional space through the parameters of the first DLP. The resulting ray intersects the known calibration board plane at a point. Then, this three-dimensional point is projected onto its image plane through the parameters of the second DLP to obtain the predicted coordinates. The sum of squared errors between the predicted coordinates and the actual coordinates extracted from the second DLP image is calculated and summed over all marker points.

4. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 2, characterized in that, The global optimization objective function is in the form of: the total error is equal to the sum of the reprojection error term and the dual DLP spatial consistency constraint error term; where the reprojection error term is the sum of the squared Euclidean distances between all 3D calibration board markers projected back to the camera and dual DLP image plane through the current parameters and the extracted sub-pixel coordinates.

5. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 2, characterized in that, The formula for determining the sub-pixel coordinates of the projector using the linear interpolation method is as follows: if the absolute phase of the camera pixel falls between the phase values ​​corresponding to the adjacent pixel indices in the projector phase map, then the sub-pixel coordinates of the projector are calculated by linear interpolation of the phase values.

6. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 1, characterized in that, The heterogeneous coding strategy is as follows: the first DLP projects a complementary Gray code combined with a phase shift sequence; the second DLP projects a frequency division-composite coding sequence, which projects high-frequency sine stripes in the red channel and low-frequency sine stripes in the blue channel, carrying two frequency information simultaneously in a single projection.

7. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 6, characterized in that, The second DLP absolute phase decoding specifically includes: separating the color channels to calculate the high-frequency wrapping phase and the low-frequency wrapping phase; expanding the low-frequency wrapping phase to obtain the low-frequency absolute phase; using the dual-frequency heterodyne principle to calculate the order of high-frequency fringes from the low-frequency absolute phase and the high-frequency wrapping phase to obtain the initial high-frequency absolute phase; calculating the self-checking residual and comparing it with a preset threshold to mark the error points; and using the natural neighborhood interpolation method to repair the error points using the phase values ​​of surrounding reliable points to obtain the corrected absolute phase.

8. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 1, characterized in that, The average height, surface area, and volume features are calculated as follows: the average height is the mean of the heights of all pixels within the element region; The surface area is the sum of the local micro-areas calculated based on the height gradient at each pixel and then applied to the component region; the volume is the height integral of each pixel within the component region relative to the reference plane, assuming the PCB substrate is the reference plane.

9. The PCB board defect detection method based on multi-source complex frequency structured light three-dimensional reconstruction as described in claim 1, characterized in that, The method for determining consistency defects is as follows: the low-confidence regions marked during the fusion process are mapped to the element regions. If the proportion of low-confidence pixel area in a certain element region exceeds a preset threshold, the element is determined to have consistency defects, indicating that the 3D reconstruction result of the region is unreliable.

Images