PCB defect detection method based on surface structured light three-dimensional reconstruction

Abstract

A PCB defect detection method based on surface structured light three-dimensional reconstruction belongs to the field of computer vision, optical detection and automatic testing, is used for solving the PCB defect detection problem of surface structured light three-dimensional reconstruction, and is characterized in that a phase shift code and a complementary Gray code grating are projected to a to-be-detected PCB, and a light intensity image of the to-be-detected PCB is acquired; obtaining a three-dimensional point cloud and a shadow area mask image according to the light intensity image; converting the two-dimensional pixel coordinates corresponding to the shadow region into three-dimensional coordinate points according to the shadow region mask map; complementing the three-dimensional point cloud in the coordinate area of the three-dimensional coordinate points by using an interpolation fitting method; separating the PCB element point cloud from the PCB substrate plane point cloud; the PCB element point cloud is registered with the standard point cloud, difference points of the PCB element point cloud and the standard point cloud under a given distance threshold value are calculated, and if the number of the difference points is not smaller than the number of the threshold value, the element is defective.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision, optical detection and automation testing, and relates to a PCB defect detection method based on surface structured light three-dimensional reconstruction. BACKGROUND

[0002] With the continuous development of electronic product technology, the quality of PCB directly affects the performance and reliability of products as the core component of electronic products. In the production process of PCB, due to factors such as material, process and environment, some defects such as open circuit, short circuit, component position deviation and surface contamination often occur. Traditional PCB defect detection methods mainly rely on manual inspection or two-dimensional image-based defect detection technology, which has problems such as insufficient detection accuracy, low efficiency and high labor cost. In order to improve the accuracy and efficiency of defect detection, three-dimensional reconstruction technology has gradually become an important means of PCB defect detection. Surface structured light technology is a mature three-dimensional reconstruction technology that uses projection grating to obtain three-dimensional information of the object surface, and has the advantages of high precision and high efficiency. However, how to accurately detect PCB surface defects based on surface structured light three-dimensional reconstruction is still a technical problem to be solved. SUMMARY

[0003] The application provides a PCB defect detection method based on surface structured light three-dimensional reconstruction, which uses surface structured light technology to perform three-dimensional reconstruction on the PCB surface, and detects and evaluates possible defects by analyzing the changes of the three-dimensional surface. This method can improve the detection accuracy, reduce manual intervention, and is suitable for large-scale industrial production lines.

[0004] In a first aspect, the PCB defect detection method based on surface structured light three-dimensional reconstruction according to some embodiments of the application comprises:

[0005] Projecting a phase shift code and a complementary Gray code grating onto a PCB to be tested to obtain an optical intensity image of the PCB to be tested;

[0006] Obtaining a three-dimensional point cloud and a shadow area mask according to the optical intensity image;

[0007] According to the shadow area mask, converting the two-dimensional pixel coordinates corresponding to the shadow area into three-dimensional coordinate points;

[0008] Using an interpolation fitting method to complete the three-dimensional point cloud in the coordinate region of the three-dimensional coordinate points;

[0009] Separating the PCB component point cloud and the PCB substrate plane point cloud;

[0010] The PCB element point cloud is registered with the standard point cloud, and the difference points between the PCB element point cloud and the standard point cloud under a given distance threshold are calculated. If the number of difference points is not less than a threshold number, the element defect is determined.

[0011] The PCB defect detection method based on the surface structured light three-dimensional reconstruction according to some embodiments of the present application further includes identifying the defect type, including accumulating the FPFH feature values of the difference points under the given distance threshold by dimension to obtain a feature vector of the defect area, normalizing the feature vector, and classifying the normalized feature vector using a support vector machine (SVM) to obtain the element defect.

[0012] The PCB defect detection method based on the surface structured light three-dimensional reconstruction according to some embodiments of the present application has a threshold value of one thousandth of the number of PCB element point clouds.

[0013] The PCB defect detection method based on the surface structured light three-dimensional reconstruction according to some embodiments of the present application has a shadow area mask map shown by the following formula:

[0014]

[0015] In the formula, the shadow area mask map is represented by S; the first intensity image is represented by I; T represents the light intensity change determination, and the left and right sides of the formula represent the pixel set satisfying the threshold condition performing an AND operation.

[0016] The PCB defect detection method based on the surface structured light three-dimensional reconstruction according to some embodiments of the present application has a source point cloud and a target point cloud The registration method includes:

[0017] S1. Randomly sampling n points in the source point cloud , and the distance between the sampling points is greater than a set threshold value;

[0018] S2. Searching for each sampling point in the source point cloud in the target point cloud having the most similar FPFH feature, to obtain a matching point pair set of each sampling point in the source point cloud in the target point cloud ;

[0019] S3. According to the matching point pair set, an initial rigid transformation matrix is calculated;

[0020] S4. According to the initial rigid transformation matrix, the points in the source point cloud in the target point cloud in the target point cloud , calculate the registration error;

[0021] S5. If the maximum number of iterations is reached, the initial rigid transformation matrix with the minimum registration error is taken as the optimal transformation matrix , and step S6 is performed;

[0022] Otherwise, steps S1-S4 are performed;

[0023] S6. According to the current initial rigid transformation matrix , calculate the points in the source point cloud after transformation ;

[0024] S7. Search for the nearest point to the transformed point in the target point cloud , and establish a near neighbor point pair set of point in the target point cloud ;

[0025] S8. According to the near neighbor point pair set of point in the target point cloud , calculate the transformation matrix ;

[0026] S9. According to the transformation matrix , calculate the registration points of the points in the source point cloud in the target point cloud , and calculate the registration error;

[0027] S10. If the registration error is lower than the error threshold, end the registration;

[0028] Otherwise, if it does not converge, make , and perform steps S6-S10;

[0029] If it converges, make the optimal transformation matrix , and perform the registration;

[0030] If the maximum number of iterations is reached, make the optimal transformation matrix the transformation matrix with the minimum registration error , and perform the registration.

[0031] The PCB defect detection method based on surface structure light three-dimensional reconstruction according to some embodiments of the present application, wherein the FPFH feature is calculated, comprising:

[0032] S21. Perform voxel filtering and down-sampling on the source point cloud and the target point cloud ;

[0033] S22. Calculate the normal direction of each point in the source point cloud and the target point cloud .

[0034] S23. Calculate the angle between each point in the source point cloud and the target point cloud and the normal direction of the neighboring points in each of the source point cloud and the target point cloud .

[0035] S24. Construct the FPFH feature histogram according to the relationship of the angle, to obtain the FPFH feature of each point in the source point cloud and the target point cloud .

[0036] According to some embodiments of the PCB defect detection method based on surface structured light three-dimensional reconstruction, the registration error is shown by the following formula:

[0037]

[0038] In the formula, represents the error between the point pairs, represents the error threshold, represents the minimum error value.

[0039] According to some embodiments of the PCB defect detection method based on surface structured light three-dimensional reconstruction, in step S5, the point in the source point cloud after transformation is shown by the following formula:

[0040]

[0041] In the formula, represents the current initial rigid body transformation matrix .

[0042] According to some embodiments of the PCB defect detection method based on surface structured light three-dimensional reconstruction,

[0043] In the formula, is the optimal rigid body transformation matrix. is the number of point cloud pairs. is the i-th point in the target point cloud . is the i-th point in the source point cloud . is the rotation matrix. is the translation transformation vector of the point cloud in the three-dimensional space.

[0044] In a second aspect, the embodiments of the present application also provide an electronic device, comprising: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions which, when executed by the electronic device, cause the electronic device to perform the first aspect and any possible technical solutions of the first aspect.

[0045] Beneficial effects: The method of the present application can effectively identify defects that cannot be found by traditional two-dimensional detection methods by accurately reconstructing the three-dimensional surface of the PCB, has high detection accuracy and strong robustness. At the same time, due to the high degree of automation, the method can greatly improve the detection efficiency of the production line, reduce labor costs, and has good adaptability and can be applied to different types of PCB defect detection.

[0046] In the first aspect, the present application is based on the three-dimensional reconstruction of the phase shift method and the complementary Gray code, and the three-dimensional reconstruction algorithm is used to process the collected multiple light images to restore the three-dimensional shape information of the PCB surface.

[0047] In the second aspect, the present application performs PCB point cloud shadow segmentation and substrate completion, solves the problem that the single-monomer structure is blocked by elements when scanning the PCB, resulting in missing of the substrate point cloud data, thereby affecting the subsequent point cloud preprocessing.

[0048] In the third aspect, the present application preprocesses the PCB element point cloud, and the point cloud data often contains a large amount of noise and redundant information, and the point cloud data is effectively preprocessed to eliminate noise.

[0049] In the fourth aspect, the present application proposes a point cloud registration method based on improved ICP to improve the registration accuracy.

[0050] In the fifth aspect, the present application identifies and classifies the defects of the PCB element, automatically classifies and marks the defects in combination with the three-dimensional morphological characteristics of the defects, and outputs the defect position and type. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 : Hardware diagram of PCB defect detection system.

[0052] Figure 2 : Reprojection error of industrial camera and surface structured light.

[0053] Figure 3 : PCB surface light projection.

[0054] Figure 4 : Initial noisy point cloud of PCB three-dimensional reconstruction.

[0055] Figure 5 : Column pixel value visualization figure.

[0056] Figure 6 : Shadow segmentation software flow chart.

[0057] Figure 7 : Substrate point cloud missing (point cloud has been pre-processed and colored).

[0058] Figure 8 : Point cloud completion flow chart.

[0059] Figure 9 : PCB defect detection flow chart.

[0060] Figure 10 : Pose correction.

[0061] Figure 11 : Final segmentation point cloud hollow boundary figure.

[0062] Figure 12 : Registration algorithm flow chart.

[0063] Figure 13 : Registration example effect figure (green is the standard board, and red is the board to be registered).

[0064] Figure 14 : Defect recognition algorithm flow chart.

[0065] Figure 15 : Various defect recognition result figures, (a) pin warping, (b) component floating, (c) component offset, (d) component missing, (e) component damage, and (f) component foreign matter.

[0066] Figure 16 : Point cloud completion effect figure.

[0067] Figure 17 : Substrate point cloud completion comparison figure.

[0068] Figure 18 : Segmentation point cloud hollow boundary figure (including components).

[0069] Figure 19 : Component point cloud schematic diagram of each processing shown in Table 2. DETAILED DESCRIPTION

[0070] The application will be further described below in conjunction with specific embodiments, but not as a limitation of the application. The embodiment is a PCB defect detection method based on surface structure light three-dimensional reconstruction. The hardware system is as follows Figure 1The shown. By calling industrial camera and surface structure light acquisition PCB surface grating picture, subsequent through grating decoding algorithm completes point cloud reconstruction. Again combined with shadow segmentation and point cloud completion technology and preprocessing complete point cloud noise removal. Finally based on improved ICP algorithm and vector machine realizes PCB defect recognition and classification.

[0071] The PCB defect detection method based on surface structure light three-dimensional reconstruction of the embodiment includes the following steps:

[0072] S10. Three-dimensional reconstruction based on phase shift method and complementary Gray code.

[0073] S20. PCB point cloud shadow segmentation and substrate completion.

[0074] S30. PCB element point cloud preprocessing.

[0075] S40. Based on improved ICP point cloud registration, PCB element defect recognition and classification.

[0076] Among them, step S10. Three-dimensional reconstruction based on phase shift method and complementary Gray code, including:

[0077] S1.1 In order to ensure the accuracy of point cloud measurement and the effect of three-dimensional surface reconstruction, the three-dimensional scanning system needs to be calibrated. The circular point detection method is used for the calibration of industrial camera. The inverse camera calibration method is used for the calibration of surface structure light. After fixing the positions of the camera, surface structure light and mobile platform, etc. Adjust its position in the camera field of view by continuously moving the calibration board, and get the re-projection error as shown in Figure 2 The re-projection error is less than 0.1 pixel. In order to verify the measurement accuracy of the three-dimensional surface defect detection system, 5mm, 15mm and 20mm standard blocks are selected to verify the accuracy of the calibrated detection system. Table 1 is the surface data of the block obtained by the three-dimensional reconstruction of the detection system.

[0078] Table 1: Repeated measurement of standard block

[0079]

[0080] S1.2 After completing the calibration and accuracy test, the obtained parameters are imported into the grating decoding program, and the phase shift code and complementary Gray code grating are projected on the measured PCB board, so that the structure light three-dimensional reconstruction system projects the PCB board by non-contact method. Static grating projection, as shown in Figure 3 Obtain 24 phase shift pictures + 14 complementary Gray code pictures. Finally, according to the corresponding relationship between absolute phase and the height of the measured object, the three-dimensional point cloud of the PCB board is obtained from the absolute phase, as shown in Figure 4

[0081] ​In the step S20.PCB point cloud shadow segmentation and substrate completion, the following steps are included:

[0082] S2.1 Due to the influence of factors such as different grating projection angles and element heights, the shadow problem exists in the stripe image collected by the monocular single-plane structured light. When the sinusoidal stripe is projected to the shadow area of the object to be measured, no obvious light and dark change can be generated, so that the correct phase value cannot be obtained in the area through demodulation of the grating, a large amount of irregular phase noise is generated in the shadow area, and the final defect detection result is affected. The present application adopts grating projection for three-dimensional reconstruction, and needs to project sinusoidal grating stripes with a fixed phase shift (such as π / 2) to the object to be measured. The sinusoidal stripe is projected to the standard block. As shown in the figure, the pixel values in the two columns of the red dashed line in the light intensity image are extracted and visualized. The column where the second red dashed line is located only exists the sinusoidal grating stripe, and the phase moves by a fixed π / 2 in one period. The first red dashed line passes through the shadow area, and the stripe in the framed area (the shadow area) of the light intensity image is basically unchanged, and the phase in the non-shadow area still moves by a fixed π / 2, that is, the light intensity change basically does not exist in the stripe projection. According to this characteristic, the specific operation process is shown in the figure, the light intensity change judgment threshold is set through the absolute difference value of adjacent phases in the phase shift process, and the common shadow area is extracted as the final shadow mask through the operation. The specific formula is as follows. Figure 5 Figure 6

[0083]

[0084]

[0085] S2.2 To solve the problem that the monocular single-plane structure is blocked by elements when scanning the PCB, resulting in missing of the substrate point cloud data, thereby affecting the subsequent point cloud preprocessing, the shadow mask and the interpolation fitting method are combined to complete the completion of the point cloud without moving the camera, the plane structure light and the PCB. Figure 7 The white frame shown is the missing substrate point cloud caused by element blocking. The present application combines the PCB shadow mask of the 2D image and the three-dimensional point cloud spline interpolation method to complete the completion of the point cloud. Common ones are B-spline, cubic spline and the like, which are suitable for the case that the surface change of the PCB is relatively smooth and the missing part can be completed through local fitting.

[0086] ​​​​​​First, according to the shadow area mask of the 2D image, the shadow area projected into the scene is determined, the two-dimensional pixel coordinates corresponding to the shadow area mask are converted into three-dimensional coordinate points through triangulation, depth calculation and other methods, and then the point cloud is completed in the coordinate region of the three-dimensional coordinate points by applying the interpolation fitting method. In the process, the normal vector information of the point cloud, the relationship between the neighborhood point clouds and the like are considered to ensure that the completed point cloud is continuous with the original point cloud in geometry. The process is as shown in Figure 8 The final point cloud completion map and the substrate comparison map are as shown in Figure 16 and Figure 17

[0087] Among them, step S30.PCB element point cloud preprocessing includes:

[0088] S3.1 In order to accurately segment the PCB element point cloud, first, the point cloud is preprocessed, as shown in Figure 9 , mainly including pose correction, noise reduction filtering, substrate segmentation and element positioning. Although the system will set a fixed reconstruction field of view to limit the position of the object to be reconstructed. However, the PCB may have a placement error to some extent, so based on the standard PCB board, the relative position of the measured board and the standard board is used to correct the pose of the substrate through the geometric transformation of the point cloud.

[0089] S3.2 After the pose correction of the substrate, the point cloud needs to be down-sampled and the noise needs to be suppressed. The noise mainly comes from the scattering when the grating projection process encounters the turning point or irregular surface of the target surface, causing the grating to reflect in different directions, forming some points not on the target surface, thereby introducing noise. The point cloud noise is mainly concentrated in the PCB element point cloud data. In contrast, the smooth PCB substrate plane point cloud is almost free of noise. Therefore, the application designs a point cloud denoising algorithm based on shape analysis. The main idea of shape analysis denoising is to use the point cloud void formed by the PCB element blocking the PCB board plane point cloud to filter the noise point cloud in the PCB element point cloud data, so as to achieve the effect of denoising. The specific algorithm steps are as follows:

[0090] (1) First, the point cloud is down-sampled (reduce the point cloud density and reduce the subsequent processing time) and the abnormal points (such as abnormal points that do not exist around the smooth plane, which may be related to the camera and the field environment) are removed by statistical filtering. Radius filtering removes isolated points.

[0091] (2) Then, the PCB element point cloud data and the PCB substrate plane point cloud data are separated for subsequent processing and analysis. The RANSAC algorithm and the straight-through filter are combined to separate the two kinds of point cloud data.

[0092] ​(3) The separated PCB board point cloud data, due to the shielding of PCB components, the scanning device cannot obtain the shielding part of the substrate point cloud data, so there are obvious hollow areas on the PCB board plane. These hollows reflect the physical occupation of PCB components and are natural features in point cloud data. For PCB component point cloud data, only the point cloud data above the hollows can be the true reflection of the PCB components. Those point cloud data below the PCB substrate plane or far away from the hollow area (defined as far away) are likely to be due to errors, noise or other interference factors in the scanning process, and therefore should be considered as noise points.

[0093] It is necessary to accurately extract the boundary of the point cloud hollow. Since there is no PCB component on the small hole of the substrate, it is not required to extract the boundary of the substrate hole. The present application selects the boundary extraction method based on normal estimation to achieve this goal. The final effect is shown in Figure 11 、 18 , the specific point cloud preprocessing data is shown in Table 2, Figure 19 .

[0094] Table 2: Point cloud preprocessing

[0095]

[0096] In the step S40. Based on the improved ICP point cloud registration, PCB component defect identification and classification, in order to detect and locate the three-dimensional defects of the component, the improved ICP point cloud registration algorithm is used. The specific operation process is shown in Figure 8 , the component point cloud is registered with the standard point cloud, and finally the defect point cloud is segmented by setting the distance threshold.

[0097] Specifically, the step S40. Based on the improved ICP point cloud registration, PCB component defect identification and classification, includes:

[0098] S4.1 Based on improved ICP point cloud registration:

[0099] Point cloud registration is an important support for PCB three-dimensional defect detection research, and plays a key role. The core task is to align and match two or more point cloud datasets in three-dimensional space, so that the sum of the spatial distances between the corresponding points of the source point cloud and the target point cloud after rigid transformation is minimized. By comparing the registered PCB component point cloud with the PCB template point cloud, three-dimensional defect location positioning can be achieved. Key points, also known as feature points, are stable, distinguishable, and feature-rich points in a point cloud set. Key points are usually fewer than the original point cloud and serve as a compact representation of the original point cloud. Using a small number of key points instead of the original point cloud to participate in computation can speed up the algorithm processing efficiency. FPFH (Fast Point Feature Histograms) is a technique for describing the local geometric features of a point cloud. It is a feature extracted based on the neighborhood information of a point cloud, which can effectively capture the shape and geometric information of the local structure. FPFH is widely used in point cloud matching, registration, and object recognition tasks.

[0100] The specific algorithm for improved ICP point cloud registration includes the following steps:

[0101] Step 1. Estimate the normal of each point's neighborhood. The commonly used method is to use PCA (Principal Component Analysis) to estimate the normal direction of each point in the point cloud.

[0102] Step 2. Calculate the angle between each point and the normal direction of the neighborhood points. This angle reflects the difference in local geometric structure between the neighborhood points and the center point.

[0103] Step 3. According to these angle relationships, FPFH constructs a feature histogram. The feature of each point is the normal angle within the local neighborhood of that point.

[0104] Step 4. The FPFH feature of each point is a fixed-dimensional vector, which describes the geometric shape of the point's neighborhood. The FPFH feature vector can be used to match and align with points in other point clouds.

[0105] After extracting the point cloud feature vector, point cloud registration is divided into coarse registration and fine registration. RANSAC algorithm is commonly used for coarse registration, that is, by randomly selecting point pairs, estimating the transformation matrix (rotation matrix R and translation vector T) to roughly align the source point cloud and the target point cloud, and in each iteration, selecting the maximum inlier set that conforms to the model, and finally determining the optimal model parameters through the maximum inlier set.

[0106] Iterative Closest Point (ICP) is a classic point cloud registration algorithm. It iteratively finds the corresponding point pairs in two point clouds by least squares iteration, calculates the coordinate transformation matrix, and makes the distance between the transformed source point cloud and the target point cloud minimum. However, the traditional ICP algorithm requires that the initial positions of the two point clouds cannot differ too much in order to obtain good convergence results. Therefore, the present application proposes a point cloud registration method based on improved ICP, which estimates the initial transformation matrix using the RANSAC idea to provide a suitable initial position for subsequent point cloud registration. The algorithm flowchart is shown in Figure 12 . Figure 13 The registration example effect diagram is shown in the figure. The green color is the source point cloud, and the red color is the target point cloud.

[0107] As shown in Figure 12 , the point cloud registration method based on improved ICP of the present application comprises the following steps:

[0108] Step 1: First, the source point cloud and the target point cloud are subjected to voxel filtering and downsampling. While maintaining the geometric structure and shape characteristics of the point cloud, the number of point clouds is reduced by downsampling to improve processing efficiency and speed up subsequent calculation.

[0109] Step 2: Extract the FPFH (Fast Point Feature Histogram) feature based on the method proposed in step S4.1. The FPFH feature can effectively describe the local geometric information of the point cloud, providing a robust feature description for subsequent point cloud matching.

[0110] Step 3: Randomly sample n points in the source point cloud and ensure that the distance between the sampled points is greater than the set threshold . This ensures uniform distribution of the sampled points and avoids over-concentration of the sampled points, improving the stability and computational efficiency of point cloud matching.

[0111] Step 4: For each sampled point in the source point cloud , find the point with the most similar FPFH feature in the target point cloud by nearest neighbor search (Nearest Neighbor Search). The goal of this step is to establish a point pair matching relationship for rigid transformation estimation.

[0112] Step 5: Calculate the rigid transformation matrix for the above point pair and determine whether to continue repeating the above steps based on the transformation quality and the maximum number of iterations.

[0113] The registration error formula is as follows:

[0114]

[0115] In the formula The error between corresponding points in the i-th group after transformation is... For the error threshold, To minimize the error value, the initial rigid body transformation matrix is ​​calculated. .

[0116] Step 6: For each point in the source point cloud Apply initial rigid body transformation matrix Calculate the transformed points The formula is as follows:

[0117]

[0118] Step 7: Search for distance points from the target point cloud nearest point Establish a set of matching point pairs and solve for the optimal transformation matrix. Based on these sets of matching point pairs, calculate the optimized rigid body transformation matrix, making the source point cloud... After transformation, it should be as close as possible to the target point cloud. Alignment. The formula for solving the optimal transformation matrix is ​​shown below.

[0119]

[0120] In the formula, This is the optimal rigid body transformation matrix. This represents the number of point cloud pairs. For target point cloud The i-th point in the array. Source Point Cloud The i-th point in the array. It is a rotation matrix. is the translation transformation vector of the point cloud in three-dimensional space.

[0121] Step 8: Finally, evaluate the transformation error. If the error is lower than the preset threshold, the registration is considered complete. Otherwise, continue iterative optimization until convergence or the maximum number of iterations is reached.

[0122]

[0123] As described above, this invention achieves high-precision point cloud registration, accurately aligning the 3D point cloud of the target element (to be detected) with the 3D point cloud of the template element (ideal model), ensuring a perfect match in spatial position and orientation. An improved registration algorithm (such as the ICP algorithm) from the PCL library is used to iteratively adjust the coordinates of the target point cloud until it coincides with the template point cloud. Only the aligned point clouds can accurately compare their differences; subsequent steps rely on the aligned coordinates for spatial analysis.

[0124] S4.2.PCB component defect identification and classification:

[0125] Based on the point cloud registration method, the target PCB component point cloud and the template PCB component point cloud are registered with high precision. Then, the three-dimensional defect position is searched on the registered alignment point cloud set through point cloud space query.

[0126] Based on the PCL point cloud processing library, the corresponding points are searched by combining the KD tree algorithm, and the difference point set of the registered point cloud under a given distance threshold is obtained by using the pcl: SegmentDifferences class. These difference points are three-dimensional defects of the component. If the number of point clouds screened based on the given distance threshold is less than 1 / 1000 of the total number of component point clouds, the component is considered to be a qualified component, otherwise it is a defective component. After locating the defect, the defect is classified and identified to finally determine the defect type.

[0127] For determining the defect type, the FPFH (pcl: FPFHSignature) feature descriptor of all key points in the segmented defect point set is calculated based on the point cloud processing library, and the value is accumulated in each dimension of the vector to form a feature vector of the three-dimensional defect point set. Since the number of key points in the point set is different, the feature vector after simple superposition cannot be directly used for comparison, so the feature vector is normalized in the present application, which fully statistics the overall FPFH feature of the point cloud while eliminating the influence of the number of point clouds on the result. Finally, the support vector machine (SVM) is used to classify the feature vector. The specific algorithm flow Figure 14 The identification results of various defects are as shown in the following table. Figure 15

[0128] Step 1: The present application finds the difference points between the target component and the template component, which may be defects or abnormal areas, i.e. the difference point set of the registered point cloud under a given distance threshold, including KD-Tree search: quickly establish the spatial index of the point cloud, efficiently query the nearest neighbor point of each point (similar to "quickly locate the adjacent point"). Segment the difference points: through the pcl: SegmentDifferences class, set a distance threshold (for example, 0.5mm), if the distance between the target point and the corresponding point of the template exceeds the threshold, it is marked as a difference point. Thus, the difference point set screened is a potential defect area, which needs to be further verified and classified subsequently.

[0129] ​Step2: Defect quantity threshold judgment: Preliminary judgment of whether the component is qualified, to avoid unnecessary complex analysis of minor differences. Count the number of difference points, if less than 1 / 1000 of the total number of points (for example, total points 100 million, difference points <1000), then directly determine as qualified piece, the process terminates. Only when the number of difference points exceeds the threshold, enter the defect classification process, save computing resources.

[0130] Step3: FPFH feature extraction: Convert the geometric features of the defect points into quantifiable mathematical vectors to provide data basis for classification. Calculate the FPFH feature (33-dimensional vector) for each defect point to describe its local geometric characteristics (such as curvature, normal direction, etc.), and then accumulate the feature values of all points by dimension to form a 33-dimensional feature vector representing the entire defect area. Compress the complex three-dimensional point cloud information into a fixed-dimensional feature vector for subsequent processing.

[0131] Step4: Feature vector normalization: Eliminate the influence of defect point quantity on feature value, ensure the comparability of defects of different sizes. Divide the accumulated 33-dimensional feature vector by the total number of defect points to normalize the value of each dimension to the same scale. The normalized data is more suitable for processing by classifiers such as SVM, avoiding classification bias caused by differences in numerical magnitude.

[0132] Step5: Defect type classification: Determine the defect type according to the feature vector. Input the normalized 33-dimensional feature into the pre-trained SVM model, and the model outputs the defect category.

[0133] Based on the above embodiment, the embodiment of the application also provides a computer program, which makes the computer execute the method provided by the above embodiment when the computer program runs on the computer.

[0134] Based on the above embodiment, the embodiment of the application also provides a computer storage medium, which stores a computer program, and the computer program makes the computer execute the method provided by the above embodiment when the computer program is executed by the computer.

[0135] Among them, the storage medium can be any available medium that the computer can access. For example, but not limited to: the computer readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0136] Based on the above embodiment, the embodiment of the application also provides a chip for reading a computer program stored in a memory, which implements the method provided by the above embodiment.

[0137] Based on the above embodiments, the embodiments of the present application provide a computer program product, when the computer program product runs on an electronic device, the method provided in the above embodiments is implemented.

[0138] Those skilled in the art will appreciate that embodiments of the present application can be supplied as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer usable program code.

[0139] The present application is described with reference to the flowcharts and / or block diagrams of the method, device (system), and computer program product according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0140] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0142] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A PCB defect detection method based on three-dimensional reconstruction using surface structured light, characterized in that, include: A phase-shifted code and complementary Gray code grating are projected onto the PCB board under test to obtain a light intensity image of the PCB board under test. Obtain 3D point cloud and shadow area mask map from light intensity image; Based on the shadow area mask, the two-dimensional pixel coordinates corresponding to the shadow area are transformed into three-dimensional coordinate points; The 3D point cloud is completed within the coordinate region of the 3D coordinate points using interpolation fitting. Separate the point cloud of PCB components from the point cloud of the PCB substrate planar plane; Register the PCB component point cloud with the standard point cloud, calculate the difference points between the PCB component point cloud and the standard point cloud at a given distance threshold. If the number of difference points is not less than the threshold number, then the component is defective.

2. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 1, characterized in that, It also includes identifying defect types, including summing the FPFH feature values ​​of the difference points under a given distance threshold by dimension to obtain the feature vector of the defect region, normalizing the feature vector, and using a support vector machine (SVM) to classify the normalized feature vector to obtain the component defect.

3. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 1, characterized in that, The threshold is one-thousandth of the number of point clouds of PCB components.

4. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 1, characterized in that, The shadow area mask is given by the following formula: In the formula, Represents the mask image of the shadow area; Indicates the first Zhang Guangqiang image, T represents the determination of light intensity change. This indicates that the set of pixels on both the left and right sides that meet the threshold condition is subjected to a bitwise AND operation.

5. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 1, characterized in that, in, Register the PCB component point cloud with the standard point cloud, where the PCB component point cloud is the source point cloud. The standard point cloud is the target point cloud. The registration methods include: S1. In the source cloud Randomly sample n points, and the distance between the sampled points is greater than a set threshold; S2. Search the source point cloud using the nearest neighbor method. Each sampling point in the target point cloud The source point cloud is obtained by finding the points with the most similar FPFH features. Each sampling point in the target point cloud The set of matching point pairs in; S3. Calculate the initial rigid body transformation matrix based on the set of matching point pairs; S4, Calculate the points in the source point cloud based on the initial rigid body transformation matrix. In the target point cloud Registration points in Calculate the registration error; S5. If the maximum number of iterations is reached, the initial rigid body transformation matrix with the smallest registration error is taken as... Proceed to step S6; Otherwise, proceed to steps S1-S4; S6. Based on the current initial rigid body transformation matrix Calculate the source point cloud Points in Transformed points ; S7. From the target point cloud Search distance transformed points nearest point Establish points In the target point cloud The set of nearest neighbor pairs; S8. Based on the point In the target point cloud Calculate the transformation matrix for the set of nearest neighbor pairs. ; S9. Based on the transformation matrix Calculate points in the source point cloud In the target point cloud Registration points in Calculate the registration error; S10. If the registration error is lower than the error threshold, then the registration process ends; Otherwise, if it is not contained, it will cause Perform steps S6-S10; If convergence occurs, the optimal transformation matrix will be... Registration is then performed. If the maximum number of iterations is reached, the optimal transformation matrix is ​​the one that minimizes the registration error. Registration is then performed.

6. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 5, characterized in that, in, Calculating FPFH features includes: S21. Source point cloud and target point cloud Perform voxel filtering downsampling; S22. Calculate the source point cloud and target point cloud The direction of the normal at each point in the middle; S23. Based on the source cloud and target point cloud Calculate the source point cloud based on the normal direction of each point. and target point cloud The angle between each point in the region and the normal direction of its neighboring points; S24. Construct the FPFH feature histogram based on the angular relationship to obtain the source point cloud. and target point cloud Each point in the FPFH features.

7. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 5, characterized in that, The registration error is given by the following formula: In the formula, Indicates the error between point pairs. Indicates the error threshold. This represents the minimum error value.

8. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 5, characterized in that, In step S5, the points in the source point cloud are calculated. Transformed points As shown in the following formula: In the formula, Represents the current initial rigid body transformation matrix .

9. The PCB defect detection method based on three-dimensional reconstruction using surface structured light according to claim 5, characterized in that, In the formula, This is the optimal rigid body transformation matrix. The number of point cloud pairs, For target point cloud The i-th point in Source Point Cloud The i-th point in Let be a rotation matrix. is the translation transformation vector of the point cloud in three-dimensional space.

10. An electronic device, the electronic device comprising: One or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method of any one of claims 1-9.

Images