PCB defect detection method based on surface structure light three-dimensional reconstruction

Abstract

The application discloses a PCB defect detection method based on a surface structure light three-dimensional reconstruction, belongs to the field of computer vision, optical detection and automation testing, and is used for solving the problem of PCB defect detection based on surface structure light three-dimensional reconstruction, and points are that phase shift codes and complementary gray code gratings are projected to a to-be-detected PCB board to obtain light intensity images of the to-be-detected PCB board; three-dimensional point clouds and shadow area mask images are obtained according to the light intensity images; according to the shadow area mask images, two-dimensional pixel coordinates corresponding to the shadow area are converted into three-dimensional coordinate points; the interpolation fitting method is used to complete the three-dimensional point clouds in the coordinate area of the three-dimensional coordinate points; PCB component point clouds and PCB substrate plane point clouds are separated; the PCB component point clouds are matched with standard point clouds, difference points between the PCB component point clouds and the standard point clouds are calculated under a given distance threshold, and if the number of the difference points is not less than a threshold number, then the component is defective.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, optical inspection and automated testing, and relates to a PCB (Printed Circuit Board) defect detection method based on surface structured light three-dimensional reconstruction. Background Technology

[0002] With the continuous development of electronic product technology, PCBs, as core components of electronic products, directly affect the performance and reliability of the products. During the PCB manufacturing process, defects often occur due to factors such as materials, processes, and environment, including open circuits, short circuits, component misalignment, and surface contamination. Traditional PCB defect detection methods mainly rely on manual inspection or two-dimensional image-based defect detection techniques. These methods suffer from insufficient detection accuracy, low efficiency, and high labor costs. To improve the accuracy and efficiency of defect detection, three-dimensional reconstruction technology has gradually become an important means of PCB defect detection. Structured light technology is a mature three-dimensional reconstruction technique that uses projection gratings to acquire three-dimensional information of an object's surface, offering advantages such as high accuracy and high efficiency. However, how to accurately detect PCB surface defects based on structured light three-dimensional reconstruction remains a pressing technical challenge. Summary of the Invention

[0003] This invention provides a PCB defect detection method based on 3D reconstruction using structured light. It utilizes structured light technology to reconstruct the PCB surface in three dimensions, and detects and evaluates potential defects by analyzing changes in the 3D surface. This method improves detection accuracy, reduces manual intervention, and is suitable for large-scale industrial production lines.

[0004] In a first aspect, a PCB defect detection method based on three-dimensional reconstruction using surface structured light according to some embodiments of this application includes:

[0005] A phase-shifted code and complementary Gray code grating are projected onto the PCB board under test to obtain a light intensity image of the PCB board under test.

[0006] Obtain 3D point cloud and shadow area mask map from light intensity image;

[0007] Based on the shadow area mask, the two-dimensional pixel coordinates corresponding to the shadow area are transformed into three-dimensional coordinate points;

[0008] The 3D point cloud is completed within the coordinate region of the 3D coordinate points using interpolation fitting.

[0009] Separate the point cloud of PCB components from the point cloud of the PCB substrate planar plane;

[0010] Register the PCB component point cloud with the standard point cloud, calculate the difference points between the PCB component point cloud and the standard point cloud at a given distance threshold. If the number of difference points is not less than the threshold number, then the component is defective.

[0011] The PCB defect detection method based on surface structured light 3D reconstruction according to some embodiments of this application further includes identifying defect types, including accumulating the FPFH feature values ​​of difference points under a given distance threshold by dimension to obtain the feature vector of the defect region, normalizing the feature vector, and using a support vector machine (SVM) to classify the normalized feature vector to obtain component defects.

[0012] According to some embodiments of this application, the PCB defect detection method based on surface structured light 3D reconstruction has a threshold of one-thousandth of the number of point clouds of PCB components.

[0013] According to some embodiments of the PCB defect detection method based on surface structured light 3D reconstruction, the shadow region mask image is given by the following formula:

[0014]

[0015] In the formula, Represents the mask image of the shadow area; Indicates the first Zhang Guangqiang's image Indicates the determination of changes in light intensity. This indicates that the set of pixels on both the left and right sides that meet the threshold condition is subjected to a bitwise AND operation.

[0016] According to some embodiments of the PCB defect detection method based on surface structured light 3D reconstruction, the PCB component point cloud is registered with a standard point cloud, wherein the PCB component point cloud is the source point cloud. The standard point cloud is the target point cloud. The registration methods include:

[0017] S1. In the source cloud Randomly sample n points from the sampled data, and the distance between the sampled points is greater than a set threshold.

[0018] S2. Search the source point cloud using the nearest neighbor method. Each sampling point in the target point cloud The source point cloud is obtained by finding the points with the most similar FPFH features. Each sampling point in the target point cloud The set of matching point pairs in;

[0019] S3. Calculate the initial rigid body transformation matrix based on the set of matching point pairs;

[0020] S4, Calculate the points in the source point cloud based on the initial rigid body transformation matrix. In the target point cloud Registration points in Calculate the registration error;

[0021] S5. If the maximum number of iterations is reached, the initial rigid body transformation matrix with the smallest registration error is taken as... Proceed to step S6;

[0022] Otherwise, proceed to steps S1-S4;

[0023] S6. Based on the current initial rigid body transformation matrix Calculate the source point cloud Points in Transformed points ;

[0024] S7. From the target point cloud Search distance transformed points nearest point Establish points In the target point cloud The set of nearest neighbor pairs;

[0025] S8. Based on the point In the target point cloud Calculate the transformation matrix for the set of nearest neighbor pairs. ;

[0026] S9. Based on the transformation matrix Calculate points in the source point cloud In the target point cloud Registration points in Calculate the registration error;

[0027] S10. If the registration error is lower than the error threshold, then the registration process ends;

[0028] Otherwise, if it is not contained, it will cause Perform steps S6-S10;

[0029] If convergence occurs, the optimal transformation matrix will be... Registration is then performed.

[0030] If the maximum number of iterations is reached, the optimal transformation matrix is ​​the one that minimizes the registration error. Registration is then performed.

[0031] According to some embodiments of the present application, a PCB defect detection method based on surface structured light 3D reconstruction includes calculating FPFH features, which includes:

[0032] S21. Source point cloud and target point cloud Perform voxel filtering downsampling;

[0033] S22. Calculate the source point cloud and target point cloud The direction of the normal at each point in the middle;

[0034] S23. Based on the source cloud and target point cloud Calculate the source point cloud based on the normal direction of each point. and target point cloud The angle between each point in the region and the normal direction of its neighboring points;

[0035] S24. Construct the FPFH feature histogram based on the angular relationship to obtain the source point cloud. and target point cloud Each point in the FPFH features.

[0036] According to some embodiments of the PCB defect detection method based on surface structured light 3D reconstruction, the registration error is given by the following formula:

[0037]

[0038] In the formula, Indicates the error between point pairs. Indicates the error threshold. This represents the minimum error value.

[0039] According to some embodiments of the PCB defect detection method based on surface structured light 3D reconstruction, in step S5, the points in the source point cloud are calculated. Transformed points As shown in the following formula:

[0040]

[0041] In the formula, Represents the current initial rigid body transformation matrix .

[0042] The PCB defect detection method based on surface structured light 3D reconstruction according to some embodiments of this application

[0043] In the formula, This is the optimal rigid body transformation matrix. This represents the number of point cloud pairs. For target point cloud The i-th point in the array. Source Point Cloud The i-th point in the array. It is a rotation matrix. is the translation transformation vector of the point cloud in three-dimensional space.

[0044] In a second aspect, embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to perform the first aspect and any possible technical solution of the first aspect.

[0045] Beneficial effects: The method of this invention can effectively identify defects that traditional two-dimensional inspection methods cannot detect by accurately reconstructing the PCB surface in three dimensions, exhibiting high detection accuracy and strong robustness. Furthermore, due to its high degree of automation, this method can significantly improve the inspection efficiency of the production line, reduce labor costs, and has good adaptability, applicable to the detection of various types of PCB defects.

[0046] In the first aspect, the present invention is based on phase-shifting method and complementary Gray code for three-dimensional reconstruction. The three-dimensional reconstruction algorithm processes multiple acquired illumination images to recover the three-dimensional morphological information of the PCB surface.

[0047] In the second aspect, the present invention performs PCB point cloud shadow segmentation and substrate completion to solve the problem that when a monocular single-sided structure scans a PCB, the mutual occlusion of components causes the substrate point cloud data to be missing, thus affecting subsequent point cloud preprocessing.

[0048] On the third-party side, this invention preprocesses the point cloud of PCB components. Point cloud data often contains a lot of noise and redundant information. Effective preprocessing of point cloud data is performed to eliminate noise.

[0049] In the fourth aspect, this invention proposes a point cloud registration method based on improved ICP to improve registration accuracy.

[0050] In the fifth aspect, the present invention identifies and classifies defects in PCB components by combining the three-dimensional morphological characteristics of the defects, automatically classifying and marking the defects, and outputting the location and type of the defects. Attached Figure Description

[0051] Figure 1 Hardware diagram of a PCB defect detection system.

[0052] Figure 2 Reprojection error of industrial camera and surface structured light.

[0053] Figure 3 : Raster projection on PCB surface.

[0054] Figure 4 Initial noisy point cloud for PCB 3D reconstruction.

[0055] Figure 5 : Column pixel value visualization chart.

[0056] Figure 6 Flowchart of shadow segmentation software.

[0057] Figure 7 : The substrate point cloud is missing (the point cloud has been preprocessed and colored).

[0058] Figure 8 Point cloud completion flowchart.

[0059] Figure 9 PCB defect detection flowchart.

[0060] Figure 10 : Posture correction.

[0061] Figure 11 : Final segmentation point cloud cavity boundary map.

[0062] Figure 12 Registration algorithm flowchart.

[0063] Figure 13 Example of registration result (green is the standard plate, red is the plate to be registered).

[0064] Figure 14 Flowchart of defect identification algorithm.

[0065] Figure 15 The following are the results of various defect identifications: (a) pin lifting, (b) component floating, (c) component offset, (d) missing component, (e) damaged component, and (f) foreign object in component.

[0066] Figure 16 Point cloud completion effect diagram.

[0067] Figure 17 Comparison of point cloud completion on substrate.

[0068] Figure 18 : Boundary map of segmented cloud voids (including elements).

[0069] Figure 19 Table 2 shows the point cloud diagrams of the components processed. Detailed Implementation

[0070] The present invention will be further described below with reference to specific embodiments, but these are not intended to limit the invention. This embodiment provides a PCB defect detection method based on three-dimensional reconstruction using surface structured light. The hardware system is as follows: Figure 1As shown, grating images of the PCB surface are acquired using an industrial camera and structured light, followed by point cloud reconstruction using a grating decoding algorithm. Noise removal from the point cloud is then achieved by combining shadow segmentation, point cloud completion techniques, and preprocessing. Finally, PCB defect identification and classification are implemented based on an improved ICP algorithm and vector machine.

[0071] The PCB defect detection method based on surface structured light 3D reconstruction in this embodiment includes the following steps:

[0072] S10. Three-dimensional reconstruction based on phase-shifting method and complementary Gray code.

[0073] S20. PCB point cloud shadow segmentation and substrate completion.

[0074] S30. Preprocessing of PCB component point cloud.

[0075] S40. PCB component defect identification and classification based on improved ICP point cloud registration.

[0076] Step S10, 3D reconstruction based on phase-shifting method and complementary Gray code, includes:

[0077] S1.1 To ensure the accuracy of point cloud measurement and the effectiveness of 3D surface reconstruction, the 3D scanning system needs to be calibrated. The industrial camera is calibrated using the dot detection method. The surface structured light is calibrated using the inverse camera calibration method. After fixing the camera, surface structured light, and moving platform in their positions, the calibration plate is continuously moved to adjust their positions within the camera's field of view, and the reprojection error is obtained. Figure 2 As shown, the reprojection error is less than 0.1 pixels. To verify the measurement accuracy of the 3D surface defect detection system, standard gauge blocks of 5mm, 15mm, and 20mm were selected to verify the accuracy of the calibrated detection system. Table 1 shows the surface data of the gauge blocks obtained from the 3D reconstruction of the detection system.

[0078] Table 1: Repeatability Measurement of Standard Gauge Blocks

[0079]

[0080] After S1.2 completes the calibration and accuracy verification, the obtained parameters are imported into the grating decoding program, and a phase-shifted code and complementary Gray code grating are projected onto the PCB board under test. This allows the structured light 3D reconstruction system to perform static grating projection on the PCB board in a non-contact manner, such as... Figure 3 As shown, 24 phase-shifted images and 14 complementary Gray code images were obtained. Finally, based on the correspondence between the absolute phase and the surface height of the measured object, the 3D point cloud of the PCB board was obtained from the absolute phase, as shown below. Figure 4 As shown.

[0081] Step S20, PCB point cloud shadow segmentation and substrate completion, includes:

[0082] S2.1 Due to factors such as varying grating projection angles and component heights, the fringe images acquired by monocular single-sided structured light suffer from shadowing issues. When sinusoidal fringes are projected onto the shadowed area of ​​the object under test, no significant change in brightness is produced, resulting in the inability to obtain the correct phase value through demodulation of the grating in that area. This leads to a large amount of irregular phase noise in the shadowed area, affecting the final defect detection results. This invention uses grating projection for 3D reconstruction, requiring the projection of sinusoidal grating fringes with a fixed phase shift (e.g., a phase shift of π / 2) onto the object under test. Sinusoidal fringes are projected onto a standard gauge block. Figure 5 As shown, the pixel values ​​of the two columns containing the red dashed lines in the intensity image are extracted and visualized. The column containing the second red dashed line contains only sinusoidal grating fringes, which shift with a fixed phase of π / 2 within one period. The first red dashed line passes through the shadow area, while the fringes in the area framed by the intensity image (the shadow area) remain essentially unchanged, and the non-shadow area still shifts with a fixed phase of π / 2, meaning that there is essentially no change in intensity during fringe projection. Based on this characteristic, the specific operation procedure is as follows: Figure 6 As shown, a threshold for judging light intensity changes is set by the absolute difference between adjacent values ​​during the phase shift process, and the common shadow region is extracted as the final shadow mask through an AND operation. The specific formula is as follows.

[0083]

[0084] In the formula, Represents the mask image of the shadow area; Indicates the first Zhang Guangqiang's image Indicates the determination of changes in light intensity. This indicates that the set of pixels on both the left and right sides that meet the threshold condition is subjected to a bitwise AND operation.

[0085] S2.2 To address the issue of missing point cloud data on the substrate due to mutual occlusion of components when scanning the PCB using a monocular single-sided structure, thus affecting subsequent point cloud preprocessing, we consider completing the point cloud by combining shadow masking and interpolation fitting methods without moving the camera, surface structured light, or PCB. Figure 7 The white boxes shown represent missing point clouds on the substrate caused by component occlusion. This invention combines PCB shadow masks from 2D images with 3D point cloud spline interpolation to complete the point cloud. Common spline interpolation methods include B-splines and cubic splines. This method is suitable for cases where the PCB surface has relatively smooth variations and missing parts can be completed through local fitting.

[0086] First, based on the shadow region mask of the 2D image, the shadow region projected into the scene is determined. Using methods such as triangulation and depth calculation, the 2D pixel coordinates corresponding to the shadow region mask are converted into 3D coordinate points. Then, interpolation fitting is applied to complete the point cloud within the coordinate region of the 3D coordinate points. During this process, the normal vector information of the point cloud and the relationships between neighboring point clouds are considered to ensure that the completed point cloud is geometrically continuous with the original point cloud. The process is as follows: Figure 8 As shown. The final point cloud completion image and the comparison image with the substrate are shown below. Figure 16 and Figure 17 As shown.

[0087] Step S30, PCB component point cloud preprocessing, includes:

[0088] S3.1 To accurately segment the point cloud of PCB components, point cloud preprocessing is first performed, such as... Figure 9 As shown, the main functions include pose correction, noise reduction filtering, substrate segmentation, and component positioning. Although this system sets a fixed reconstruction field of view to limit the placement of the object to be reconstructed, PCBs will inevitably have some placement error. Therefore, based on the knowledge of the standard PCB board, the pose correction of the substrate is performed by geometric transformation of the point cloud using the relative position of the board under test and the standard board.

[0089] After substrate pose correction (S3.2), the point cloud needs to be downsampled and noise suppressed. Noise mainly originates from scattering when the grating encounters inflection points or irregular surfaces on the target surface during projection, causing reflections in different directions and forming points not on the target surface, thus introducing noise. Point cloud noise is mainly concentrated in the PCB component point cloud data; in contrast, the point cloud of a smooth PCB substrate planar surface has almost no noise. Therefore, this invention designs a point cloud denoising algorithm based on shape analysis. The main idea of ​​shape analysis denoising is to utilize the point cloud holes formed by PCB component occlusion during grating projection to filter the noisy point cloud in the PCB component point cloud data, thereby achieving denoising. The specific algorithm steps are as follows:

[0090] (1) First, voxel downsampling is performed on the point cloud (to reduce the point cloud density and reduce subsequent processing time) and statistical filtering is used to remove outliers (such as non-existent outliers around a smooth plane, which may be related to the camera or the field environment). Radius filtering is used to remove isolated points.

[0091] (2) Then, the point cloud data of PCB components and the point cloud data of PCB substrate planar data are separated for subsequent processing and analysis. The RANSAC algorithm and the pass-through filtering method are combined to complete the separation of the two types of point cloud data.

[0092] (3) After separation, the PCB board point cloud data is obstructed by PCB components, preventing the scanning equipment from acquiring the point cloud data of the obstructed portion of the substrate. This results in obvious void areas on the PCB board plane. These voids reflect the physical occupancy of the PCB components and are natural features in the point cloud data. For PCB component point cloud data, only the point cloud data located above these voids can be considered a true reflection of the PCB components. Point cloud data located below the PCB substrate plane or far from the void areas (defined as far away) are likely caused by errors, noise, or other interference factors during the scanning process and should therefore be considered noise point clouds.

[0093] Accurate extraction of the boundaries of point cloud holes is required. Since there are no PCB components on the substrate holes, extraction of substrate hole boundaries is not necessary. This invention selects a boundary extraction method based on normal estimation to achieve this goal. The final result is as follows: Figure 11 , 18 As shown in Table 2, the specific point cloud preprocessing data is as follows. Figure 19 As shown.

[0094] Table 2: Point Cloud Preprocessing

[0095]

[0096] In S40. PCB component defect identification and classification based on improved ICP point cloud registration, this invention uses an improved ICP point cloud registration algorithm to detect and locate three-dimensional defects in components. The specific operation process is as follows: Figure 8 As shown, the component point cloud is registered with the standard point cloud, and finally the defect point cloud is segmented by setting a distance threshold.

[0097] Specifically, step S40. Based on improved ICP point cloud registration, PCB component defect identification and classification includes:

[0098] S4.1 Based on Improved ICP Point Cloud Registration:

[0099] Point cloud registration plays a crucial role in PCB 3D defect detection research. Its core task is to align and match two or more point cloud datasets in 3D space, minimizing the sum of spatial distances between corresponding points in the source point cloud and the target point cloud after rigid body transformation. By comparing the registered PCB component point cloud with the PCB template point cloud, 3D defect location can be achieved. Keypoints, also known as feature points, are stable, distinguishable points with rich feature information in a point cloud set. Keypoints are usually fewer than the original point cloud, serving as a compact representation of the original point cloud. Using a smaller number of keypoints instead of the original point set in computation can accelerate algorithm processing efficiency. Fast Point Feature Histograms (FPH) are a technique for describing the local geometric features of point clouds. It extracts features based on the neighborhood information of the point cloud, effectively capturing the shape and geometric information of local structures. FPFH is widely used in tasks such as point cloud matching, registration, and object recognition.

[0100] The specific algorithm for improved ICP point cloud registration includes the following steps:

[0101] Step 1. Estimate the normals of the neighborhood of each point. A common method is to use PCA (Principal Component Analysis) to estimate the normal direction of each point in the point cloud.

[0102] Step 2. Calculate the angle between the normal direction of each point and its neighboring points. This angle reflects the difference in local geometry between the neighboring points and the center point.

[0103] Step 3. Based on these angular relationships, FPFH constructs a feature histogram. The feature of each point is the normal angle in its local neighborhood.

[0104] Step 4. The FPFH feature of each point is a fixed-dimensional vector that describes the geometry of the point's neighborhood. The FPFH feature vector can be used to match and align with points in other point clouds.

[0105] After extracting the feature vectors of the point cloud, point cloud registration is performed. Registration is divided into coarse registration and fine registration. The RANSAC algorithm is often used for coarse registration, which means that by randomly selecting point pairs, estimating the transformation matrix (rotation matrix R and translation vector T) to roughly align the source point cloud and the target point cloud, and selecting the largest set of interior points that conforms to the model in each iteration, and finally determining the optimal model parameters by using the largest set of interior points.

[0106] Iterative Closest Point (ICP) is a classic point cloud registration algorithm that uses least squares iteration to continuously find corresponding point pairs between two point clouds and calculates the coordinate transformation matrix to minimize the distance between the source and target point clouds after transformation. However, the traditional ICP algorithm requires that the initial positions of the two point clouds not differ too much to achieve good convergence. Therefore, this invention proposes a point cloud registration method based on an improved ICP, which uses the idea of ​​RANSAC to estimate the initial transformation matrix, providing a suitable initial position for subsequent fine point cloud registration. The algorithm flowchart is shown below. Figure 12 As shown. Figure 13 This is a registration example image. In the image, green represents the source point cloud, and red represents the target point cloud.

[0107] like Figure 12 As shown, the point cloud registration method based on improved ICP of the present invention includes the following steps:

[0108] Step 1: First, analyze the source point cloud. and target point cloud Voxel filtering downsampling is performed. While preserving the geometric structure and shape features of the point cloud itself, downsampling reduces the number of points, improving processing efficiency and accelerating subsequent calculations.

[0109] Step 2: Extract FPFH (FastPointFeatureHistogram) features based on the method proposed in step S4.1. FPFH features can effectively describe the local geometric information of the point cloud, providing a robust feature description for subsequent point cloud matching.

[0110] Step 3: In the source cloud Randomly sample n points, ensuring that the distance between the sampled points is greater than a set threshold. This ensures a uniform distribution of sampling points, avoids excessive concentration of sampling points, and improves the stability and computational efficiency of point cloud matching.

[0111] Step 4: For each point cloud in the source cloud The points sampled in the middle are in the target point cloud. In this step, the nearest neighbor search is used to find the point with the most similar FPFH features, which is then used as the corresponding matching point. The goal of this step is to establish point pair matching relationships in order to perform rigid body transformation estimation.

[0112] Step 5: Calculate the rigid body transformation matrix for the above point pairs, and determine whether to continue repeating the above steps based on the transformation quality and the maximum number of iterations.

[0113] The registration error formula is as follows:

[0114]

[0115] In the formula The error between corresponding points in the i-th group after transformation is... For the error threshold, To minimize the error value, the initial rigid body transformation matrix is ​​calculated. .

[0116] Step 6: For each point in the source point cloud Apply initial rigid body transformation matrix Calculate the transformed points The formula is as follows:

[0117]

[0118] Step 7: Search for distance points from the target point cloud nearest point Establish a set of matching point pairs and solve for the optimal transformation matrix. Based on these sets of matching point pairs, calculate the optimized rigid body transformation matrix, making the source point cloud... After transformation, it should be as close as possible to the target point cloud. Alignment. The formula for solving the optimal transformation matrix is ​​shown below.

[0119]

[0120] In the formula, This is the optimal rigid body transformation matrix. This represents the number of point cloud pairs. For target point cloud The i-th point in the array. Source Point Cloud The i-th point in the array. It is a rotation matrix. is the translation transformation vector of the point cloud in three-dimensional space.

[0121] Step 8: Finally, evaluate the transformation error. If the error is lower than the preset threshold, the registration is considered complete. Otherwise, continue iterative optimization until convergence or the maximum number of iterations is reached.

[0122]

[0123] As described above, this invention achieves high-precision point cloud registration, accurately aligning the 3D point cloud of the target element (to be detected) with the 3D point cloud of the template element (ideal model), ensuring a perfect match in spatial position and orientation. An improved registration algorithm (such as the ICP algorithm) from the PCL library is used to iteratively adjust the coordinates of the target point cloud until it coincides with the template point cloud. Only the aligned point clouds can accurately compare their differences; subsequent steps rely on the aligned coordinates for spatial analysis.

[0124] S4.2. PCB Component Defect Identification and Classification:

[0125] This invention, based on the aforementioned point cloud registration method, performs high-precision registration of the point cloud of the target PCB component and the point cloud of the template PCB component. Then, it uses point cloud spatial query to locate the three-dimensional defect position on the registered aligned point cloud set.

[0126] This invention utilizes the PCL point cloud processing library and combines it with the KD-tree algorithm to search for corresponding points. It then uses the pcl::SegmentDifferences class to segment and obtain a set of difference points in the registered point cloud at a given distance threshold. These difference points represent the 3D defects of the component. If the number of point clouds selected based on the given distance threshold is less than 1 / 1000 of the total number of point clouds in the component, the component is considered a qualified part; otherwise, it is a defective part. After locating the defect, this invention classifies and identifies the defect to ultimately determine its type.

[0127] To determine the defect type, this invention uses a point cloud processing library to calculate the FPFH (pcl::FPFHSignature) feature descriptors of all key points in the segmented defect point set. The values ​​are then accumulated along each dimension of the vector to form a feature vector for the three-dimensional defect point set. Since the number of key points in each point set varies, the simply superimposed feature vectors cannot be directly compared. Therefore, this invention normalizes the feature vectors, fully statistically analyzing the overall FPFH features of the point cloud while eliminating the influence of the number of points on the results. Finally, a Support Vector Machine (SVM) is used to classify the feature vectors. The specific algorithm flow is as follows. Figure 14 The following are the results of various defect identification methods. Figure 15 As shown.

[0128] Step 1: The present invention identifies the differences between the target component and the template component. These points may be defects or abnormal areas, that is, obtaining the set of difference points of the registered point cloud under a given distance threshold, including KD-Tree search: quickly establishing the spatial index of the point cloud and efficiently querying the nearest neighbor of each point (similar to "quickly locating adjacent points"). Segmenting the difference points: By using the pcl::SegmentDifferences class, setting a distance threshold (for example, 0.5 mm), if the distance between the target point and the corresponding point of the template exceeds the threshold, it is marked as a difference point. Thus, the filtered set of difference points is the potential defect area, which needs to be further verified and classified later.

[0129] Step 2: Determination of the defect quantity threshold: Initially judge whether the component is qualified to avoid unnecessary complex analysis of minor differences. Count the number of difference points. If it is less than 1 / 1000 of the total number of points of the component (for example, the total number of points is 1 million and the number of difference points < 1000), it is directly determined as a qualified part and the process terminates. Only when the number of difference points exceeds the threshold does it enter the defect classification process, saving computing resources.

[0130] Step 3: FPFH feature extraction: Convert the geometric features of the defect points into quantifiable mathematical vectors to provide a data basis for classification. Calculate the FPFH feature (33-dimensional vector) for each defect point to describe its local geometric properties (such as curvature, normal direction, etc.), and then accumulate the feature values of all points by dimension to form a 33-dimensional feature vector representing the entire defect area. Compress the complex three-dimensional point cloud information into a feature vector with a fixed dimension for subsequent processing.

[0131] Step 4: Feature vector normalization: Eliminate the influence of the number of defect points on the feature values to ensure the comparability of defects of different sizes. Divide the accumulated 33-dimensional feature vector by the total number of defect points to normalize the values of each dimension to the same scale. The normalized data is more suitable for processing by classifiers such as SVM, avoiding classification bias caused by differences in numerical magnitudes.

[0132] Step 5: Defect type classification: Judge the defect type according to the feature vector. Input the normalized 33-dimensional feature into the pre-trained SVM model, and the model outputs the defect category.

[0133] Based on the above embodiments, the embodiments of the present application also provide a computer program. When the computer program runs on a computer, it causes the computer to execute the method provided by the above embodiments.

[0134] Based on the above embodiments, the embodiments of the present application also provide a computer storage medium. The computer storage medium stores a computer program. When the computer program is executed by a computer, it causes the computer to execute the method provided by the above embodiments.

[0135] The storage medium can be any available medium that a computer can access. For example, but not limited to, a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0136] Based on the above embodiments, this application also provides a chip for reading a computer program stored in a memory to implement the method provided in the above embodiments.

[0137] Based on the above embodiments, this application provides a computer program product that implements the methods provided in the above embodiments when the computer program product is run on an electronic device.

[0138] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0142] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A PCB defect detection method based on a structured light three-dimensional reconstruction of a surface, characterized in that, The method comprises: projecting a phase shift code and a complementary Gray code grating to a to-be-tested PCB board to obtain an optical intensity image of the to-be-tested PCB board; obtaining a three-dimensional point cloud and a shadow area mask according to the optical intensity image; converting two-dimensional pixel coordinates corresponding to the shadow area into three-dimensional coordinate points according to the shadow area mask; completing the three-dimensional point cloud in the coordinate area of the three-dimensional coordinate points by using an interpolation fitting method; separating PCB component point cloud and PCB substrate plane point cloud; registering the PCB component point cloud with a standard point cloud, calculating difference points between the PCB component point cloud and the standard point cloud under a given distance threshold, and determining that there is an element defect if the number of the difference points is not less than a threshold number; wherein the PCB element point cloud is registered with a standard point cloud, wherein the PCB element point cloud is a source point cloud and the standard point cloud is a target point cloud ; wherein the registration method comprises: S1. Randomly sampling n points in the source point cloud with a distance between the sampled points greater than a set threshold; S2. Search the source point cloud using the nearest neighbor method. Each sampling point in the target point cloud The source point cloud is obtained by finding the points with the most similar FPFH features. Each sampling point in the target point cloud The set of matching point pairs in; S3. According to the matching point pair set, an initial rigid body transformation matrix is calculated; S4, calculating points in the source point cloud according to the initial rigid transformation matrix In the target point cloud , calculating registration errors ​ S5. If the maximum number of iterations is reached, the initial rigid body transformation matrix with the smallest registration error is taken as S6 is performed. otherwise, steps S1-S4 are performed; S6. Transform the points in the source point cloud according to the current initial rigid body transformation matrix S7. Calculate the transformed points in the source point cloud ; S7. Search for the nearest point in the target point cloud from the transformed point nearest point , establish the point in the target point cloud set of nearest point pairs; S8. The method of point In the target point cloud Set of pairs of neighboring points, compute transformation matrix ; S9. Based on the transformation matrix Calculate points in the source point cloud In the target point cloud Registration points in Calculate the registration error; S10. If the registration error is lower than an error threshold, the registration is ended; Otherwise, if not converged, make , perform steps S6-S10; If converged, make optimal transform matrix , perform registration; If the maximum iteration number is reached, the optimal transformation matrix is the transformation matrix with the minimum registration error , registration is performed.

2. The method of claim 1, wherein the method further comprises: The method further comprises identifying a defect type, including accumulating feature values of the difference points under the given distance threshold according to dimensions to obtain a feature vector of a defect area, normalizing the feature vector, classifying the normalized feature vector by using a support vector machine (SVM), and obtaining the element defect.

3. The method of claim 2, wherein the method further comprises: The threshold is 1 / 1000 of the number of the PCB component point cloud.

4. The method of claim 3, wherein the method further comprises: The shadow area mask is shown by the following formula: In the formula, represents a shadow area mask image; represents the first intensity image, represents the light intensity change determination, represents the left and right sides of the pixel set satisfying the threshold condition are operated with.

5. The method of claim 4, wherein the method further comprises: wherein, The FPFH feature is calculated, including: S21. voxel-filtering down-sampling the source point cloud and the target point cloud ; S22. Compute the normal direction of each point in the source point cloud and the target point cloud S23. Compute the distance between each point in the source point cloud and the target point cloud S23. Compute, for each point in the source point cloud and the normal direction of each point in the target point cloud the angle between the normal direction of each point in the source point cloud and the normal direction of each point in the target point cloud and the normal direction of the neighboring points. S24. Constructing FPFH feature histograms according to the relationship of angles, obtaining respective points FPFH features in the source point cloud and the target point cloud .

6. The method of claim 5, wherein the method further comprises: The registration error is shown by the following formula: wherein denotes the error between the points pairs, denotes the error threshold, denotes the minimum error value.

7. The method of claim 6, wherein the method further comprises: In step S5, the points in the source point cloud are calculated transformed points are shown by the following equation: wherein represents the current initial rigid body transformation matrix .

8. The method of claim 7, wherein the method further comprises: wherein, is the optimal rigid transformation matrix, is the number of point cloud pairs, is the target point cloud is the i-th point in the target point cloud, is the source point cloud is the i-th point in the source point cloud, is the rotation matrix, is the translation transformation vector of the point cloud in three-dimensional space.

9. An electronic device, comprising: One or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, when the instructions are executed by the electronic device, causing the electronic device to perform any one of the methods in claims 1-8.

Images