Electronic device reference surface extraction method and device and computer readable storage medium

Abstract

The invention belongs to the technical field of 3D AOI industrial visual inspection, and relates to a datum plane extraction method and device of an electronic device and a computer readable storage medium. Acquiring a point cloud of the electronic device and setting a plurality of local neighborhood radiuses; taking the minimum value of the perpendicularity characteristic value of each point under each local neighborhood radius as the perpendicularity value of the point; clustering the points based on the perpendicularity values to obtain a plurality of perpendicularity point cloud clusters; merging the verticality point cloud clusters, and obtaining the verticality point cloud cluster with the most points after merging to obtain denoised point clouds; classifying the point clouds based on the height interval and the height values of the denoised points to obtain a plurality of height value point cloud clusters; and merging the height value point cloud clusters, and taking the height value point cloud cluster with the point cloud scale greater than or equal to the element surface order of magnitude and the minimum height value mean value of all points in the cluster in the first N height value point cloud clusters with the most points after merging as a reference surface main cluster, thereby obtaining the reference surface of the electronic device to detect the electronic device.

Description

Technical Field

[0001] This invention relates to the field of 3D AOI industrial vision inspection technology, and in particular to a method, apparatus and computer-readable storage medium for extracting reference surfaces of electronic devices. Background Technology

[0002] Electronic components include PCB boards and elements such as chips, capacitors, resistors, and diodes. The miniaturization and integration of electronic products place higher demands on the inspection of these components. Traditional manual visual inspection and 2D automated optical inspection methods are no longer sufficient due to issues such as efficiency, subjectivity, and difficulty in handling three-dimensional structural defects. 3D point cloud technology offers the possibility of accurate 3D inspection of electronic components, but its foundation and core lie in the accurate extraction of the reference surface, as the accuracy of the reference surface directly determines the reliability of all subsequent 3D measurement results.

[0003] However, traditional point cloud reference surface extraction methods (such as algorithms based on curvature, normal vectors, or global fitting) are extremely sensitive to point cloud noise, boundary effects, and high-density component distribution, easily producing erroneous extraction results. These algorithmic defects lead to geometric distortion or positional offset in the extracted reference surface, resulting in systematic errors. Ultimately, this significantly reduces the accuracy of critical quality assessments such as height detection, component mounting, and flatness based on incorrect reference surfaces.

[0004] In summary, existing methods cannot accurately extract the reference surface of electronic devices, resulting in low detection accuracy for tasks such as height detection of incorrect reference surfaces, component mounting detection, and flatness detection. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the reference surface of electronic devices cannot be accurately extracted in the prior art, which leads to low detection accuracy in the detection of the height of the erroneous reference surface, component mounting detection, and flatness detection.

[0006] To address the aforementioned technical problems, this invention provides a method for extracting the reference plane of an electronic device, comprising: Acquire the point cloud of the electronic device and set multiple local neighborhood radii; For each point, the minimum value of its verticality eigenvalue under each local neighborhood radius is taken as the verticality value of that point; All points are clustered based on their verticality values ​​to obtain multiple verticality point cloud clusters; these clusters are then merged, and the cluster with the most points after merging is obtained, resulting in a denoised point cloud. Based on multiple height ranges and the height values ​​of each point after denoising, the denoised point cloud is classified to obtain multiple height value point cloud clusters, and the height value point cloud clusters are merged. The merged height value point cloud clusters are arranged in descending order of the number of points. The height value point cloud cluster with a point cloud size greater than or equal to the order of magnitude of the component surface and the smallest average height value of all points in the cluster is taken as the reference surface main cluster, thereby extracting the reference surface of the electronic device.

[0007] Preferably, merging point cloud clusters includes: Among the neighboring clusters of a point cloud cluster, the clusters with fewer points than the point cloud cluster itself are selected as the candidate cluster set for that point cloud cluster. In the candidate cluster set of the point cloud cluster, the clusters whose ratio of the number of points to the number of points in the point cloud cluster is greater than a preset ratio threshold are selected as the cluster set to be merged for the point cloud cluster. Merge the point cloud cluster with all clusters in its set of clusters to be merged.

[0008] Preferably, the reference plane for extracting the electronic device includes: The reference surface cluster of electronic devices is used to obtain the reference surface mask of the electronic device, and a curved or planar model is selected as the reference surface fitting model. Substitute the coordinates of the points within the base plane masking area into the reference plane fitting model to obtain the distance from each point to the surface of the reference plane fitting model. The goal is to minimize the sum of squared distances from all points within the base plane masking region to the surface of the reference plane fitting model. The fitting parameters of the reference plane fitting model are then obtained, thus yielding the target fitting model of the reference plane of the electronic device.

[0009] Preferably, the formula for the sum of squared distances from all points within the base masking region to the surface of the reference fitting model is: , in, This represents the sum of squared distances from all points within the base plane masking region to the surface of the reference plane fitting model; Indicates the number of points within the base masking area; This represents the distance from the i-th point within the base plane masking region to the surface of the reference plane fitting model; This represents the fitting parameters of the datum plane fitting model. When the datum plane fitting model is a planar model, As a four-dimensional vector, when the reference surface fitting model is a curved surface model, It is a six-dimensional vector.

[0010] Preferably, after obtaining the target fitting model of the reference surface of the electronic device, the method further includes correcting the height values ​​of each point in the point cloud of the electronic device, specifically including: Calculate the height of the projection normal vector of each point in the point cloud of the electronic device onto the target fitting model; The corrected height values ​​of each point in the electronic device point cloud are obtained by comparing the height of each point with the height of the projection normal vector of the target fitting model.

[0011] Preferably, the formula for correcting the height value of each point in the point cloud of the electronic device is expressed as follows: , in, This indicates the height value of the point after correction; This indicates the correction of the previous point's height value; This represents the height of the projection normal vector of the point onto the target fitted model; This represents the fitting parameters of the datum plane fitting model. When the datum plane fitting model is a planar model, As a four-dimensional vector, when the reference surface fitting model is a curved surface model, It is a six-dimensional vector.

[0012] Preferably, after acquiring the point cloud of the electronic device, the process further includes: Preprocess the point cloud to remove noisy and outlier points; The preprocessed point cloud is downsampled, and the downsampled point cloud is used as the point cloud of the electronic device.

[0013] Preferably, before clustering all points based on the verticality value of each point, the method further includes removing points whose verticality value is greater than a preset verticality threshold.

[0014] The present invention also provides a reference plane extraction device for electronic devices, comprising: The point cloud data acquisition module is used to acquire the point cloud of electronic devices and set multiple local neighborhood radii; The verticality feature acquisition module is used to take the minimum value of the verticality feature under each local neighborhood radius as the verticality value of each point. The first clustering module is used to cluster all points based on the verticality value of each point to obtain multiple verticality point cloud clusters; the verticality point cloud clusters are merged, and the verticality point cloud cluster with the most points after merging is obtained, thus obtaining the denoised point cloud. The second clustering module is used to classify the denoised point cloud based on multiple height intervals and the height values ​​of each point after denoising, to obtain multiple height value point cloud clusters, and to merge the height value point cloud clusters. The reference surface extraction and detection module is used to arrange the merged height value point cloud clusters in descending order of the number of points, and select the height value point cloud cluster with a point cloud size greater than or equal to the order of magnitude of the component surface and the smallest average height value of all points in the cluster as the reference surface main cluster, thereby extracting the reference surface of the electronic device.

[0015] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for extracting the reference plane of an electronic device.

[0016] The reference surface extraction method for electronic devices provided in this application has the following beneficial effects; Multiple verticality features are calculated for each point using multiple local neighborhood radii. Small-radius features capture details but are susceptible to noise, while large-radius features smooth noise but lose details. Therefore, the minimum verticality value under all radii is taken as the point's feature data. This is equivalent to using large-scale stability to filter small-scale noise fluctuations. Simultaneously, by constructing a feature pyramid to select the optimal feature values ​​of the point cloud, the problem of traditional curvature calculation relying on a single local neighborhood, where noise points (such as depth camera errors or point cloud fluctuations caused by uneven illumination) directly distort curvature values ​​and lead to segmentation errors, is avoided. Furthermore, since the verticality features of the point cloud on the reference surface are highly consistent and the height values ​​are similar, this application uses hierarchical clustering based on verticality and height features to select points that conform to the characteristics of the reference surface point cloud. Specifically... Based on verticality values, the verticality point cloud clusters are merged according to the ratio of the number of points, thus eliminating clusters with fewer points as noise clusters and retaining effective verticality point cloud clusters. Then, the points in the effective verticality point cloud clusters are classified according to height intervals to obtain multiple height value point cloud clusters, separating points at different levels and avoiding mutual interference between features at different heights. The height value point cloud clusters are merged again according to the ratio of the number of points to eliminate point cloud fragmentation caused by component distribution. Finally, the reference surface main cluster is selected based on the characteristic of the main base surface having the largest area and the smallest height value, thus constructing an accurate electronic device reference surface. Based on this reference surface, the thickness, component mounting height, position and other parameters of electronic devices can be accurately detected, achieving high-precision detection of electronic devices. Attached Figure Description

[0017] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein: Figure 1 A flowchart of the reference plane extraction method for electronic devices provided in this application; Figure 2 A schematic diagram illustrating the principle of reference plane extraction for the electronic device provided in this application; Figure 3 A schematic diagram for extracting height correction of a large-scale curved surface complex SMT 3D reference provided for this application; wherein, Figure 3 (a) in the image is a grayscale image with height correction extracted from the baseline. Figure 3 (b) in the figure is a height map with reference extraction height correction. Figure 3(c) in the figure is a cross-sectional view of the reference extraction height correction; Figure 4 Color schematic diagrams of the reference planes and component planes provided in this application are extracted; wherein, Figure 4 (a) in the diagram is a color illustration of the first reference plane and the component plane. Figure 4 (b) in the diagram is a color illustration of the second reference plane and the component plane. Figure 4 (c) in the diagram is a color illustration of the third reference plane and the component plane. Figure 5 The reference plane and component plane depth maps provided in this application; wherein, Figure 5 (a) in the image is the depth map extracted from the first reference plane and the component plane. Figure 5 (b) in the image shows the depth map extracted from the second reference plane and the component plane. Figure 5 (c) in the figure represents the depth map extracted from the third reference plane and the component plane; Figure 6 This application provides a schematic diagram of the datum plane and component plane segmentation results; wherein, Figure 6 (a) in the diagram is a schematic diagram of the segmentation result of the first reference plane and the component plane. Figure 6 (b) in the diagram is a schematic diagram of the segmentation result of the second reference plane and the component plane. Figure 6 (c) in the diagram is a schematic diagram of the segmentation result of the third reference plane and the component plane. Detailed Implementation

[0018] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0019] Please refer to the following: Figure 1 and Figure 2 , Figure 1 The diagram shown is a flowchart of the reference plane extraction method for electronic devices provided in this application. Figure 2 The diagram shown illustrates the principle of reference plane extraction for electronic devices provided in this application. The method specifically includes: S10: Obtain the point cloud of the electronic device and set multiple local neighborhood radii.

[0020] Specifically, laser scanning can be used to obtain point clouds on the surface of electronic devices, thereby reflecting the three-dimensional morphology of the electronic device surface. Each point in the point cloud contains X, Y, and Z three-dimensional coordinate information.

[0021] Furthermore, after acquiring the point cloud of the electronic device, the process also includes: preprocessing the point cloud to remove noise points and outliers; downsampling the preprocessed point cloud and using the downsampled point cloud as the point cloud of the electronic device.

[0022] S20: For each point, take the minimum value of its verticality eigenvalue under each local neighborhood radius as the verticality value of that point.

[0023] Specifically, based on the coordinates of each point and the normal vector of each local neighborhood radius, the perpendicularity feature value of each point under each local neighborhood radius is calculated, thereby constructing a feature pyramid for each point. Smaller local neighborhood radii focus on the microscopic details of the point. For example, when inspecting electronic devices, they can capture the perpendicularity features of fine structures such as solder joints and component pins. Larger local neighborhood radii focus on macroscopic structures and can be used to judge the overall shape trend of electronic devices. By constructing feature pyramids, features under different neighborhood radii can complement and verify each other, thereby improving the accuracy of reference surface extraction of electronic devices.

[0024] Specifically, the formula for calculating the perpendicularity feature value of a point under each local neighborhood radius is as follows: , in, Represents the verticality characteristic value; Indicates the first The normal vector of the radius of a local neighborhood; Represents the coordinates of a point.

[0025] Specifically, the minimum value of the perpendicularity eigenvalue of a point across all local neighborhood radii is taken as the perpendicularity value of that point, and the formula is expressed as: , in, This represents the perpendicularity value of a point; This represents the coordinates of a point within the current local neighborhood radius after upsampling. Indicates upsampling; Represents the coordinates of a point; This represents the verticality feature value of a point at the k-th local neighborhood radius.

[0026] S30: Cluster all points based on the verticality value of each point to obtain multiple verticality point cloud clusters; merge the verticality point cloud clusters and obtain the verticality point cloud cluster with the most points after merging, thereby obtaining the denoised point cloud.

[0027] Furthermore, before clustering all points based on their verticality values, the process also includes removing points whose verticality values ​​are greater than a preset verticality threshold.

[0028] Specifically, points with verticality values ​​greater than a preset verticality threshold are removed, which is equivalent to downsampling the point cloud. The formula is as follows: , in, This represents the point cloud within the k-th local neighborhood radius after downsampling; Represents the coordinates of a point; The downsampling parameter represents the radius of the k-th local neighborhood. Indicates the filtering operator; This represents the result after filtering the (k-1)th local neighborhood radius.

[0029] S40: Based on multiple height ranges and the height values ​​of each point after denoising, classify the denoised point cloud to obtain multiple height value point cloud clusters, and merge the height value point cloud clusters.

[0030] Specifically, multiple height intervals refer to dividing the height value range from the minimum to the maximum height value among all points at equal intervals, thus obtaining multiple height intervals. For example, if the minimum height value among all points is 0.1mm and the maximum height value is 1mm, then the height intervals can be: [0.1mm, 0.2mm), [0.2mm, 0.3mm), [0.3mm, 0.4mm), [0.4mm, 0.5mm), [0.5mm, 0.6mm), [0.6mm, 0.7mm), [0.7mm, 0.8mm), [0.8mm, 0.9mm), [0.9mm, 1mm]. S50: Arrange the merged height value point cloud clusters in descending order of the number of points. Select the height value point cloud cluster with a point cloud size greater than or equal to the order of magnitude of the component surface and the smallest average height value of all points within the cluster as the reference surface main cluster, thereby extracting the reference surface of the electronic device.

[0031] Specifically, the order of magnitude of a component surface refers to the scale level corresponding to the number of points contained in a point cloud cluster. The point cloud scale of a point cloud cluster related to a base surface is generally [missing information]. There are 10 points, while the point cloud size of the point cloud clusters related to the component surface is generally 100 points. Each point is much smaller than the point cloud cluster on the base surface. Therefore, by selecting point cloud clusters whose size is greater than or equal to the order of magnitude of the element surface, and then selecting the point cloud cluster with the smallest average height value, the point cloud cluster corresponding to the base surface can be obtained.

[0032] This application calculates multiple verticality features for each point using multiple local neighborhood radii. Small radius features capture details but are susceptible to noise, while large radius features smooth noise but lose details. Therefore, the minimum verticality value under all radii is taken as the feature data of the point. This is equivalent to using large-scale stability to filter small-scale noise fluctuations. At the same time, by constructing a feature pyramid to select the best feature values ​​of the point cloud, this application avoids the problem that traditional curvature calculation relies on a single local neighborhood, and that noisy points (such as depth camera errors and point cloud fluctuations caused by uneven lighting) will directly distort the curvature values, leading to segmentation errors. Furthermore, since the verticality features of points on the reference surface are highly consistent and the height values ​​are similar, this application uses hierarchical clustering based on verticality features and height value features to select points that conform to the characteristics of the reference surface point cloud. Specifically, after clustering based on verticality values, verticality point cloud clusters are merged according to the ratio of the number of points, thereby eliminating clusters with fewer points as noise clusters and retaining effective verticality point cloud clusters. Then, the points in the effective verticality point cloud clusters are classified according to height intervals to obtain multiple height value point cloud clusters, in order to separate points at different levels and avoid mutual interference between features at different heights. The height value point cloud clusters are merged again according to the ratio of the number of points to eliminate point cloud data fragmentation caused by component distribution. Finally, the reference surface main cluster is selected by selecting the main base surface with the largest area and the smallest height value, thereby constructing an accurate electronic device reference surface. Based on this reference surface, the thickness, component mounting height, position and other parameters of electronic devices can be accurately detected, realizing high-precision detection of electronic devices.

[0033] Furthermore, in steps S30 and S40, the point cloud clusters are merged based on the ratio of the number of points in the clusters, which specifically includes: Among the neighboring clusters of a point cloud cluster, the clusters with fewer points than the point cloud cluster itself are selected as the candidate cluster set for that point cloud cluster. In the candidate cluster set of the point cloud cluster, the clusters whose ratio of the number of points to the number of points in the point cloud cluster is greater than a preset ratio threshold are selected as the cluster set to be merged for the point cloud cluster. Merge the point cloud cluster with all clusters in its set of clusters to be merged.

[0034] Specifically, since the method of dividing point cloud clusters is actually to use a given threshold to divide them in the vertical or height direction, adjacent point cloud clusters refer to point cloud clusters that are directly adjacent in position. For example, after dividing the point cloud in the height direction based on the given threshold, each point cloud cluster cannot intersect with other clusters in the height direction. Therefore, the adjacent clusters of the current point cloud cluster are two clusters that are one bin_size lower and one bin_size higher than the current point cloud cluster.

[0035] For example, if the number of points in consecutively adjacent point cloud clusters a, b, c, d, e, f, g, h, i are 1, 5, 9, 6, 7, 8, 2, 10, and 11 respectively, and the preset ratio threshold is 0.5, then the point cloud cluster merging process is as follows: For point cloud cluster a, the number of points in its neighboring cluster b is greater than that of point cloud cluster a, therefore point cloud cluster a and point cloud cluster b are not merged; For point cloud cluster c, the number of points in its neighboring clusters b and d is less than that of point cloud cluster c, and the ratio of the number of points in cluster b to the number of points in point cloud cluster c is greater than 0.5, therefore clusters b and d are taken as clusters to be merged into point cloud cluster c; For point cloud cluster e, the number of points in its neighboring cluster d is less than that of point cloud cluster e, and the ratio of the number of points in cluster d to the number of points in point cloud cluster e is greater than 0.5. 0.5, therefore cluster d is the cluster to be merged from point cloud cluster e; for point cloud cluster f, the number of points in its neighboring cluster e is less than that in point cloud cluster f, and the ratio of the number of points in cluster e to the number of points in point cloud cluster f is greater than 0.5, so cluster e is the cluster to be merged from point cloud cluster f; for point cloud cluster i, the number of points in its neighboring cluster h is less than that in point cloud cluster i, and the ratio of the number of points in h to the number of points in point cloud cluster i is greater than 0.5, so cluster h is the cluster to be merged from point cloud cluster i; based on the above process, the final merging result is: point cloud clusters b, c, d, e, and f are merged, point cloud clusters h and i are merged, and point cloud clusters a and g are not merged, that is, after merging, four point cloud clusters are obtained: point cloud cluster a, point cloud cluster b+c+d+e+f, point cloud cluster g, and point cloud cluster h+i.

[0036] Furthermore, the reference plane of the electronic device extracted in step S50 includes: Step 1: Obtain the base surface mask of the electronic device based on the reference surface main cluster, and select a curved or planar model as the reference surface fitting model.

[0037] Optionally, if the point cloud obtained in step S10 is downsampled, that is, each point after downsampling corresponds to a region in the initial point cloud, then after obtaining the base mask of the electronic device in step 1, the base mask needs to be resampled, that is, all points in each region of the initial point cloud corresponding to each point in the base mask are restored.

[0038] Step 2: Substitute the coordinates of the points within the base plane masking area into the reference plane fitting model to obtain the distance from each point to the surface of the reference plane fitting model.

[0039] Step 3: With the objective of minimizing the sum of squared distances from all points within the base plane masking region to the surface of the reference plane fitting model, fit the fitting parameters of the reference plane fitting model to obtain the target fitting model of the electronic device's reference plane.

[0040] Furthermore, by using the target fitting model of the reference surface as a spatial reference, electronic devices can be inspected.

[0041] Specifically, the formula for the sum of squared distances from all points within the base plane masking region to the surface of the reference plane fitting model is: , in, This represents the sum of squared distances from all points within the base plane masking region to the surface of the reference plane fitting model; Indicates the number of points within the base masking area; This represents the distance from the i-th point within the base plane masking region to the surface of the reference plane fitting model; This represents the fitting parameters of the datum plane fitting model. When the datum plane fitting model is a planar model, As a four-dimensional vector, when the reference surface fitting model is a curved surface model, It is a six-dimensional vector.

[0042] Furthermore, after obtaining the target fitting model of the reference surface of the electronic device, the process also includes correcting the height values ​​of each point in the point cloud of the electronic device, specifically including: Calculate the height of the projection normal vector of each point in the point cloud of the electronic device onto the target fitting model; The corrected height values ​​of each point in the electronic device point cloud are obtained by comparing the height of each point with the height of the projection normal vector of the target fitting model.

[0043] Specifically, the formula for correcting the height values ​​of each point in the point cloud of the electronic device to be tested is expressed as follows: , in, This indicates the height value of the point after correction; This indicates the correction of the previous point's height value; This represents the height of the projection normal vector of the point onto the target fitted model; This represents the fitting parameters of the datum plane fitting model. When the datum plane fitting model is a planar model, As a four-dimensional vector, when the reference surface fitting model is a curved surface model, It is a six-dimensional vector.

[0044] It should be noted that when the area of ​​the electronic device is large, the height range of the main base surface is first obtained. For example, if the height of the main base surface is concentrated in the range of 0~0.5mm, points whose height values ​​exceed this range are directly removed to quickly filter out prominent features that are significantly higher than the main base surface. The remaining points are divided into multiple sets of point clouds, so that each set of point clouds corresponds to a local area of ​​the electronic device. Then, the reference surface main cluster of each local area is obtained using the method provided in the above embodiment. Finally, the reference surface main clusters of each local area are merged to obtain the reference surface of the entire electronic device.

[0045] A specific example of this application uses the above method to extract a large-scale, complex SMT 3D reference surface, such as... Figure 3 The diagram shown is a schematic diagram of height correction for large-scale curved surface complex SMT 3D benchmark extraction provided in this application; wherein, Figure 3 (a) in the image is a grayscale image with height correction extracted from the baseline. Figure 3 (b) in the figure is a height map with reference extraction height correction. Figure 3 (c) in the figure is a cross-sectional view of the reference extraction height correction. Figure 4 The image shown is a color schematic diagram illustrating the extraction of the reference plane and component plane provided in this application; wherein, Figure 4 (a) in the diagram is a color illustration of the first reference plane and the component plane. Figure 4 (b) in the diagram is a color illustration of the second reference plane and the component plane. Figure 4 (c) in the diagram is a color schematic of the third reference plane and the component plane. Figure 5 The image shown is a depth map of the datum plane and component plane extraction provided in this application; wherein, Figure 5 (a) in the image is the depth map extracted from the first reference plane and the component plane. Figure 5 (b) in the image shows the depth map extracted from the second reference plane and the component plane. Figure 5 (c) in the figure represents the depth map extracted from the third reference plane and the component plane. Figure 6 The diagram shown is a schematic representation of the datum plane and component plane segmentation results provided in this application; wherein, Figure 6 (a) in the diagram is a schematic diagram of the segmentation result of the first reference plane and the component plane. Figure 6 (b) in the diagram is a schematic diagram of the segmentation result of the second reference plane and the component plane. Figure 6 (c) in the diagram is a schematic representation of the segmentation result between the third reference plane and the component plane. It should be noted that... Figure 6 The blue part in (c) is the reference plane.

[0046] from Figures 3-6 As can be seen, the reference surface extraction method for electronic devices provided in this application can clearly extract the component surface and reference surface on the electronic device, and segment the component surface and reference surface, thereby enabling relevant testing of the electronic device based on the reference surface.

[0047] In summary, the method provided in this application automatically extracts the reference surface of electronic devices using 3D information. Local and global features are constructed through verticality features and pyramids. Fine local and global features construct texture features, while large-scale local features construct normal features. Image preprocessing is performed by analyzing normal features and combining prior knowledge of the electronic device to remove some smoothed large-scale PCB board-component transition areas, weakening the interference of small-scale components occupying the main field of view on the extraction of the main texture area. Clustering is performed by analyzing texture features to obtain the main PCB board texture, achieving filtering of complex transition areas, non-primary components (such as trenches), and background noise. An adaptive downsampling method is used in conjunction with pyramid extraction of feature information to ensure the robustness of segmentation, accelerate feature analysis speed, and eliminate the need for pre-setting. By localizing clustering features under a large field of view, the method is compatible with the extraction of reference surfaces under a large field of view and ensures real-time performance under such conditions. Simultaneously, distorted and tilted board surfaces are straightened to the 0 plane as the reference height, enabling rapid application in electronic device measurement scenarios, improving 3D measurement efficiency, and providing a reference for the inspection of other similar products. In addition, in the actual test data collected, the detection method provided in this application has a fitting error of ±10um for the PCB board surface with an imaging error of 20um. In the actual detection process, the average detection time for a single image (500*500) is less than 1ms, and the average time for a large field of view (4096*3072) is 70ms.

[0048] Based on the reference surface extraction method for electronic devices provided in the above embodiments, this application also provides a reference surface extraction device for electronic devices, which specifically includes: The point cloud data acquisition module is used to acquire the point cloud of electronic devices and set multiple local neighborhood radii.

[0049] The verticality feature acquisition module is used to take the minimum value of the verticality feature of each point under each local neighborhood radius as the verticality value of that point.

[0050] The first clustering module is used to cluster all points based on the verticality value of each point to obtain multiple verticality point cloud clusters; the verticality point cloud clusters are merged, and the verticality point cloud cluster with the most points after merging is obtained, thus obtaining the denoised point cloud.

[0051] The second clustering module is used to classify the denoised point cloud based on multiple height intervals and the height values ​​of each point after denoising, to obtain multiple height value point cloud clusters, and then merge the height value point cloud clusters.

[0052] The reference surface extraction and detection module is used to arrange the merged height value point cloud clusters in descending order of the number of points, and select the height value point cloud cluster with a point cloud size greater than or equal to the order of magnitude of the component surface and the smallest average height value of all points in the cluster as the reference surface main cluster, thereby extracting the reference surface of the electronic device.

[0053] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for extracting the reference surface of an electronic device.

[0054] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0055] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0056] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0057] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0058] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for extracting the reference surface of an electronic device, characterized in that, include: Acquire the point cloud of the electronic device and set multiple local neighborhood radii; For each point, the minimum value of its verticality eigenvalue under each local neighborhood radius is taken as the verticality value of that point; All points are clustered based on their verticality values ​​to obtain multiple verticality point cloud clusters; these clusters are then merged, and the cluster with the most points after merging is obtained, resulting in a denoised point cloud. Based on multiple height ranges and the height values ​​of each point after denoising, the denoised point cloud is classified to obtain multiple height value point cloud clusters, and the height value point cloud clusters are merged. The merged height value point cloud clusters are arranged in descending order of the number of points. The height value point cloud cluster with a point cloud size greater than or equal to the order of magnitude of the component surface and the smallest average height value of all points in the cluster is taken as the reference surface main cluster, thereby extracting the reference surface of the electronic device.

2. The method for extracting the reference surface of an electronic device according to claim 1, characterized in that, Merging point cloud clusters includes: Among the neighboring clusters of a point cloud cluster, the clusters with fewer points than the point cloud cluster itself are selected as the candidate cluster set for that point cloud cluster. In the candidate cluster set of the point cloud cluster, the clusters whose ratio of the number of points to the number of points in the point cloud cluster is greater than a preset ratio threshold are selected as the cluster set to be merged for the point cloud cluster. Merge the point cloud cluster with all clusters in its set of clusters to be merged.

3. The method for extracting the reference surface of an electronic device according to claim 1, characterized in that, The reference planes of the extracted electronic devices include: The reference surface cluster of electronic devices is used to obtain the reference surface mask of the electronic device, and a curved or planar model is selected as the reference surface fitting model. Substitute the coordinates of the points within the base plane masking area into the reference plane fitting model to obtain the distance from each point to the surface of the reference plane fitting model. The goal is to minimize the sum of squared distances from all points within the base plane masking region to the surface of the reference plane fitting model. The fitting parameters of the reference plane fitting model are then obtained, thus yielding the target fitting model of the reference plane of the electronic device.

4. The method for extracting the reference surface of an electronic device according to claim 3, characterized in that, The formula for the sum of squared distances from all points within the base plane masking region to the surface of the fitted model on the base plane is: ， in, This represents the sum of squared distances from all points within the base plane masking region to the surface of the reference plane fitting model; Indicates the number of points within the base masking area; This represents the distance from the i-th point within the base plane masking region to the surface of the reference plane fitting model; This represents the fitting parameters of the datum plane fitting model. When the datum plane fitting model is a planar model, As a four-dimensional vector, when the reference surface fitting model is a curved surface model, It is a six-dimensional vector.

5. The method for extracting the reference surface of an electronic device according to claim 3, characterized in that, After obtaining the target fitting model of the reference surface of the electronic device, the process also includes correcting the height values ​​of each point in the point cloud of the electronic device, specifically including: Calculate the height of the projection normal vector of each point in the point cloud of the electronic device onto the target fitting model; The corrected height values ​​of each point in the electronic device point cloud are obtained by comparing the height of each point with the height of the projection normal vector of the target fitting model.

6. The method for extracting the reference surface of an electronic device according to claim 5, characterized in that, The formula for correcting the height values ​​of each point in the point cloud of electronic devices is expressed as follows: ， in, This indicates the height value of the point after correction; This indicates the correction of the previous point's height value; This represents the height of the projection normal vector of the point onto the target fitted model; This represents the fitting parameters of the datum plane fitting model. When the datum plane fitting model is a planar model, As a four-dimensional vector, when the reference surface fitting model is a curved surface model, It is a six-dimensional vector.

7. The method for extracting the reference surface of an electronic device according to claim 1, characterized in that, After obtaining the point cloud of the electronic device, the following steps are also included: Preprocess the point cloud to remove noisy and outlier points; The preprocessed point cloud is downsampled, and the downsampled point cloud is used as the point cloud of the electronic device.

8. The method for extracting the reference surface of an electronic device according to claim 1, characterized in that, Before clustering all points based on their verticality values, the process also includes removing points whose verticality values ​​are greater than a preset verticality threshold.

9. A reference surface extraction device for an electronic device, characterized in that, include: The point cloud data acquisition module is used to acquire the point cloud of electronic devices and set multiple local neighborhood radii; The verticality feature acquisition module is used to take the minimum value of the verticality feature under each local neighborhood radius as the verticality value of each point. The first clustering module is used to cluster all points based on the verticality value of each point to obtain multiple verticality point cloud clusters; the verticality point cloud clusters are merged, and the verticality point cloud cluster with the most points after merging is obtained, thus obtaining the denoised point cloud. The second clustering module is used to classify the denoised point cloud based on multiple height intervals and the height values ​​of each point after denoising, to obtain multiple height value point cloud clusters, and to merge the height value point cloud clusters. The reference surface extraction and detection module is used to arrange the merged height value point cloud clusters in descending order of the number of points, and select the height value point cloud cluster with a point cloud size greater than or equal to the order of magnitude of the component surface and the smallest average height value of all points in the cluster as the reference surface main cluster, thereby extracting the reference surface of the electronic device.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the reference surface extraction method for the electronic device according to any one of claims 1 to 8.

Images