Sheet metal hole group classification and full-size measurement method based on 3D point cloud processing

Abstract

The present application relates to a kind of based on 3D point cloud processing's sheet metal hole group classification and full-size measurement method, belong to machine vision measurement technical field.First by line scanning camera to the processed sheet metal covering piece is scanned, obtains point cloud data, and for original point cloud data is preprocessed, including downsampling, filtering and segmentation.After boundary point extraction is carried out to the point cloud after pre-processing, and the clustering algorithm based on density is used to store separately to round hole data.Then based on statistics, the variance threshold of data point and the centroid distance of the data set is set, the classification of data is completed and the corresponding label is given.Finally, according to the label type, the corresponding fitting algorithm is used, the round hole aperture and position degree measurement and the full-size measurement of waist-shaped hole are realized.The present application first proposes a non-contact measurement model of waist-shaped hole on the basis of ensuring measurement accuracy, and realizes the automatic classification of round hole, waist-shaped hole and noise, which significantly improves the full-size measurement efficiency of hole group.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to the technical field of machine vision measurement, and more particularly to a sheet metal hole group classification and full-size measurement method based on 3D point cloud processing. BACKGROUND

[0002] Sheet metal parts are widely used in the fields of automobiles and aerospace. Due to the characteristics of its production, i.e. fast processing speed and large production batch, it is difficult to realize online monitoring of the size of sheet metal parts. The size and position of the key hole formed by blanking play an important role in the assembly stage. The traditional production line adopts manual sampling inspection, which is low in efficiency, large in error and cannot realize full inspection.

[0003] With the progress of computer science, non-contact measurement based on optical and machine vision technology has been continuously developed. The continuous progress of three-dimensional point cloud processing technology and the continuous improvement of hardware performance make it possible to realize online non-contact measurement through three-dimensional information. Common expressions of three-dimensional information include space grid, point cloud and voxel, etc. When the point cloud density is large, the three-dimensional appearance information of the object can be accurately described. At present, the way to obtain point cloud is divided into active measurement and passive measurement. Passive measurement is to obtain the image of the object by camera only and complete the measurement through image processing algorithm to obtain the result. The measurement accuracy of this method is greatly affected by the environment. The active measurement method generally uses a special artificial light source such as a laser and a camera to form a measurement system to obtain the three-dimensional information of the object surface.

[0004] At present, there are few related technologies about automatic classification and measurement of waist-shaped holes and circular holes, and most of the methods only measure the hole diameter of the circular hole and do not measure the spatial position information. Most of the current technologies for measuring waist-shaped holes are two-dimensional, and since the waist-shaped hole needs to determine the center position of the two semicircles to obtain the size information of the hole, it causes the problems of inaccurate measurement and low efficiency by manual measurement. SUMMARY

[0005] The purpose of the present application is to provide a sheet metal part hole group classification and full size measurement method based on 3D point cloud processing, which fundamentally solves the problems of low efficiency, inability to measure online and low precision of traditional contact measurement, and improves the problem of missed detection in the stamping mass production process. The present application uses a line scan camera to scan the processed thin-walled sheet metal part, obtains its three-dimensional point cloud, and obtains its geometric and size characteristics; based on the obtained point cloud, the point cloud is denoised, and the process includes straight-through filtering, statistical filtering and region growing segmentation algorithm, which can effectively remove the environmental background point cloud, error points caused by reflection of the edge and spatial outliers. For the processed point cloud, the algorithm efficiency of subsequent processing is improved by completing downsampling through voxel filtering. For the down-sampled point cloud, the Angle Criterion is used to extract the boundary points, and the extracted boundary points include the outer contour of the sheet metal cover, the internal circular hole and the waist-shaped hole; the extracted boundary point cloud is subjected to the Density-Based Spatial Clustering of Applications with Noise algorithm, and the hole can be stored as independent point cloud data in the container. In order to complete the classification of the circular hole and the waist-shaped hole, the present application sets a variance threshold according to the statistical method, which can automatically classify the independent point cloud data. It will be divided into three categories: noise, circular hole and waist-shaped hole, and the data in the container will be screened one by one and labeled through traversal. For circular hole fitting, the least square space circle fitting based on random sampling consensus (RANSAC) is used, and the spatial coordinates of the center position of the circular hole on the thin-walled sheet metal part, i.e. the position degree and the diameter of the circular hole, can be obtained after fitting. The fitting of the waist-shaped hole is divided into two steps: first, the RANSAC is used to complete the fitting of the two straight lines in space, the inner points of the two straight edges of the waist-shaped hole are determined after solving the spatial equation of the two straight lines in space, and the point cloud is obtained. The average distance from the inner points on one straight line to the other straight line in space is calculated, and the hole width of the waist-shaped hole is obtained. Then, the corresponding point cloud is removed according to the point cloud index, and the remaining two semicircular arcs are subjected to the least square space circle fitting based on RANSAC, and the distance between the spatial coordinates of the centers of the two circles obtained and the hole width calculated above is the hole length of the waist-shaped hole. Thus, the sheet metal part hole group classification and full size measurement are completed.

[0006] The above purpose of the present application is achieved by the following technical scheme:

[0007] The sheet metal part hole group classification and full size measurement method based on 3D point cloud processing uses point cloud processing traditional algorithm to realize the sheet metal part hole group classification and full size measurement, and the specific process includes the following steps:

[0008] Step 1) install a line scan device and obtain a point cloud;

[0009] 1.1) The sheet metal cover moves at a specified speed on the conveyor belt and passes through the line scan field range area;

[0010] 1.2) The calibrated line scanner is used to obtain the real-time point cloud of the sheet metal cover, which is stored in the device hard disk;

[0011] Step 2) Traverse the acquired sheet metal cover point cloud, implement denoising and downsampling, obtain a point cloud with 5-10 times less data than the original point cloud and less noise than the original point cloud, in order to improve the running efficiency of the subsequent algorithm;

[0012] Step 3) Extract the boundary of the processed point cloud, and independently store the hole data and noise data in the boundary point cloud through clustering method; classify and label the round holes, waist-shaped holes and noise according to the data variance threshold;

[0013] Step 4) Remove the point cloud data with noise label, and fit the round holes and waist-shaped holes respectively to complete full-size measurement; the round holes are fitted by random sampling consensus least squares based on space circle to complete the measurement of round hole diameter and position; for the waist-shaped hole, the two straight edges are fitted by random sampling consensus space straight line, and the straight line part is removed according to the index, and the remaining two circular arcs are fitted by random sampling consensus least squares based on space circle.

[0014] Step 2) is to traverse the acquired different sheet metal cover point clouds, implement denoising and downsampling, and the specific steps are as follows:

[0015] 2.1) First, the acquired point cloud removes the background point cloud and the point cloud deviating from the measured part through a pass-through filter, which greatly reduces the point cloud data and preliminarily filters the point cloud data;

[0016] 2.2) The downsampling of the point cloud is completed by voxel filtering. The point cloud density obtained by the line scanner is too large, in order to reduce the running time of the algorithm and realize online monitoring, the point cloud needs to be downsampled. The point cloud after voxel filtering and downsampling can reduce the data amount while retaining the original geometric features;

[0017] 2.3) The obvious outliers in the point cloud data are removed by statistical filtering. The characteristics of outliers are often sparse in space. By calculating the average distance between the statistical points and their surrounding points and constructing a Gaussian distribution, outliers can be removed by giving a threshold value;

[0018] 2.4) In the scanning process, due to reflection and light reflection, there will be point clouds perpendicular to the surface at the edge. By using a region growing segmentation algorithm, given the smoothness parameter and the curvature radius, the segmentation of different regions of the point cloud data can be completed, and finally the sheet metal cover surface point cloud data with fewer noise points than step (2.3) is obtained.

[0019] Step 3) is to extract the boundary of the point cloud data, cluster and store each hole edge point cloud separately, and give a label according to the variance threshold. The specific steps are as follows:

[0020] 3.1) Given the neighborhood range, solve the point cloud normal vector information by Kdtree acceleration and principal component analysis;

[0021] 3.2) According to the existing normal vector information of the point cloud, the boundary points are extracted by the Angle Criterion;

[0022] 3.3) For the extracted boundary point data, use the Density-Based Spatial Clustering of Applications with Noise algorithm to process, so as to segment and store each hole separately according to the spatial distance. According to the upper limit of the point number threshold, the most points of the outer contour are removed, and according to the lower limit of the point number threshold, the scattered boundary with calculation error is removed;

[0023] 3.4) Traverse the separately stored data, calculate the distance between each point and the centroid and solve the distribution variance; give the corresponding threshold according to the variance characteristics;

[0024] 3.5) According to the variance threshold, classify the waist-shaped hole, round hole and noise and give the corresponding label.

[0025] Step 4) is to bring different data label types into different algorithm models for fitting and solving. The specific steps are as follows:

[0026] 4.1) According to the difference of point cloud label, input the point cloud data into the corresponding algorithm model;

[0027] 4.2) When the label is a round hole, use the least square space circle fitting based on random sampling consensus (RANSAC). The spatial circle equation obtained by this algorithm is less affected by error points than the least square fitting, and finally the diameter and center coordinates of the round hole can be obtained according to the fitting result. The characteristics include the geometric and position information of the round hole;

[0028] 4.3) When the label is a waist-shaped hole, firstly, the random sample consensus method of spatial straight line is used for fitting to obtain two spatial straight line equations; one of the straight lines is selected, and the inliers under the equation are obtained according to a given threshold value and the index thereof is obtained; the inliers are traversed, the distance of the inliers to the spatial straight line is solved, and the average value thereof is calculated, which is the hole width of the waist-shaped hole; the points near the two spatial straight lines are removed, and the remaining point cloud is two semicircular arcs; the least square spatial circle fitting based on the random sample consensus (RANSAC) is used for the two semicircular arcs respectively, the center coordinates and the radius value of the two semicircular arcs are solved, and the distance between the two center coordinates and the sum of the two semicircular arc radii are the hole length of the waist-shaped hole;

[0029] 4.4) When the label is noise, the part of data is emptied and the memory space is released.

[0030] The beneficial effects of the present application are:

[0031] 1) Firstly, the line scanning camera is used to scan the processed sheet metal cover and obtain point cloud data, the original point cloud data is preprocessed, including downsampling, filtering and segmentation. The boundary points are extracted from the preprocessed point cloud, and the density-based clustering algorithm is used to store the circular holes and the like separately. Based on statistics, the threshold value of the distance variance of the data points and the centroid of the data set is set, the data is classified and the corresponding label is given. Finally, according to the label type, the corresponding fitting algorithm is used to realize the measurement of the circular hole diameter and the position degree, and the full size measurement of the waist-shaped hole. On the basis of ensuring the measurement accuracy, the present application firstly proposes a non-contact measurement model of the waist-shaped hole and realizes the automatic classification of the circular hole, the waist-shaped hole and the noise, which significantly improves the efficiency of the full size measurement of the hole group.

[0032] 2) Compared with the traditional contact measurement method, the present application significantly improves the measurement efficiency and reduces the measurement cost of the sheet metal cover hole group; compared with other non-contact measurement methods, the present application firstly realizes the online detection of the hole group size.

[0033] 3) The present application uses the traditional three-dimensional point cloud algorithm, the measurement method occupies less resources, and the operation and maintenance cost is low.

[0034] 4) The application field of the present application can be expanded to the multi-hole structure in the automobile box and the aviation sheet metal structure, and the expansion range is large.

[0035] 5) Compared with the existing non-contact measurement hole, the present application firstly gives the measurement method of the waist-shaped hole, and through the classification of the data, the size measurement of the multi-hole structure is completed in an integrated manner. And thanks to the point cloud data format, the position degree information of the structure can be easily obtained. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. The detailed description of the application is set forth in connection with the accompanying drawings.

[0037] Figure 1 A flowchart of the sheet metal part hole group classification and full-size measurement method based on 3D point cloud processing of the application;

[0038] Figure 2 For the obtained sheet metal part point cloud, the gray scale information is proportional to the height information, that is, the higher the Z height, the closer the gray color to white.

[0039] Figure 3 For the point cloud after downsampling and filtering in the application;

[0040] Figure 4 For the edge points extracted in the application;

[0041] Figure 5 For the clustering results after setting the threshold in the application, the gray values of different categories are different;

[0042] Figure 6 For the variance threshold classification principle of the point and centroid distance distribution of different geometric shapes;

[0043] Figure 7 For the least squares spatial circle fitting schematic flowchart based on random sample consensus (RANSAC);

[0044] Figure 8 For the schematic diagram of solving the hole width based on the known single-side straight line equation of the waist-shaped hole. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the application will be described below in detail with reference to the drawings. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application. In order to make the above objects, features and advantages of the application more obvious and easy to understand, the application will be further described in detail below with reference to the drawings and specific embodiments.

[0046] Since active measurement has good robustness and is less affected by the external environment, the point cloud used in the application is obtained based on active measurement. The classification of the hole group of the sheet metal cover part and the full-size measurement are completed through the traditional point cloud processing algorithm, which greatly improves the measurement accuracy and efficiency, and does not cause deformation and damage to the part itself due to contact.

[0047] Referring toFigures 1 to 8 As shown, the metal plate hole group classification and full-size measurement method based on 3D point cloud processing of the present application can efficiently and automatically measure the geometric size and position of round holes and waist-shaped holes online. Not only is it the first to realize non-contact measurement of waist-shaped holes, but it can also automatically classify according to data statistical characteristics. It fundamentally solves the problems of low efficiency and low precision caused by traditional contact measurement, and improves the problem of missed detection in the stamping mass production process. First, the point cloud data of the processed metal plate is obtained by a line scanning camera, and the original data is preprocessed. Second, the boundary points are extracted from the preprocessed data, and the independent holes in space are stored separately. The individually stored point cloud data is traversed, the distance of each data set point to the centroid is calculated and the distribution variance is solved, and the corresponding threshold is set according to the variance characteristics, and the automatic classification of round holes and waist-shaped holes is completed and labeled. Finally, according to the different types of labels, put them into different models for solving. When the data is determined as a round hole, the least square space circle fitting based on random sample consensus (RANSAC) is used to solve the size and position of the round hole. When the data is determined as a waist-shaped hole, first, the least square fitting based on random sample consensus of a space straight line is used to solve the equation of the two straight edges of the waist-shaped hole. The distance of the inner points on one straight line to the other straight line is calculated to obtain the width of the waist-shaped hole. The inner point data of the straight line model is removed, and the remaining points are respectively fitted based on the random sample consensus (RANSAC) least square fitting of the space circle to solve the coordinates of the two circle centers and the radius values. The distance between the two circle center coordinates and the sum of the two semicircle radii is the length of the waist-shaped hole. When the data is noise, it is removed.

[0048] Embodiment

[0049] The present embodiment provides a metal plate hole group classification and full-size measurement method based on 3D point cloud processing, which mainly combines the straight-through filtering technology, statistical filtering technology, automatic growth segmentation technology, angle judgment criterion to judge the boundary points, etc. in the traditional algorithm of 3D point cloud, to complete the metal plate hole group classification and full-size measurement. The technical roadmap for measuring the hole group of the body covering metal plate is shown in Figure 1 , and the specific process includes the following steps:

[0050] Step (1) Install a line scanning device and obtain point cloud data.

[0051] Step (1.1) The metal plate covering part moves at a specified speed on the conveyor belt and passes through the line scanning field range area.

[0052] In this embodiment, according to the principle that the precision of placing the sheet metal cover in the near field of the line scanning device is higher than that in the far field, the conveying belt carrying the sheet metal part is arranged at a suitable height and the sheet metal part is placed in a stable posture. Considering the volume of the sheet metal cover and in order to ensure the scanning accuracy, the conveying belt running speed is set to 30 mm / s, so that the sheet metal cover passes through the best field of view range of the line scanning device at a constant speed.

[0053] Step (1.2) obtains the real-time point cloud of the sheet metal cover through the calibrated line scanner and stores it in the device hard disk.

[0054] In order to eliminate errors caused by distortion, the line scanning device is calibrated, and in order to convert the acquired point cloud data from the camera coordinate system to the world coordinate system, the line scanning device needs to be calibrated until the conveying belt plane is parallel to the XOY plane and the Z coordinate is 0. According to the reflection of the sheet metal cover, set a reasonable sensor exposure time and start scanning, and store the original point cloud after scanning in the device hard disk. The original point cloud data is shown in Figure 2

[0055] Step (2) traverses the acquired sheet metal cover point cloud, removes noise and reduces sampling to obtain a point cloud with relatively small data volume and less data noise, in order to improve the running efficiency of the subsequent algorithm.

[0056] Step (2.1) first removes the background point cloud and the point cloud deviating from the measured part through the pass-through filter, greatly reducing the point cloud data volume while realizing the preliminary screening of the point cloud data.

[0057] Pass-through filtering allows signals within a specified frequency range to pass through, and removes signals outside the range. In the point cloud, it intuitively represents the removal of point cloud data outside the specified range of XYZ dimensions, thereby removing background point cloud and error points generated due to reflection, and realizing the screening of the original point cloud data. The given interval range can be set according to the obtained data, and after the setting is completed, the relative position between the sensor and the conveying belt described in step (1.1) remains unchanged, which can be used continuously.

[0058] Step (2.2) completes the downsampling of the point cloud through voxel filtering. The point cloud acquired through the line scanner has a high density, in order to reduce the running time of the algorithm and realize online monitoring, the point cloud needs to be downsampled. After downsampling through voxel filtering, the point cloud can reduce the data volume while retaining the original geometric features.

[0059] ​In the present example, the number of point clouds after step (2.1) is 14 million, and the huge amount of point cloud data undoubtedly puts great pressure on hardware, algorithm and running time. In order to improve the operation efficiency and reduce the operation amount while maintaining the original geometric characteristics of the structure, the present application uses voxel filtering to complete the downsampling of the point cloud data. The brief implementation principle of voxel filtering is as follows: first, according to the point cloud, the maximum values x max , y max , z max and the minimum values x min , y min , z min of the data in the X, Y and Z dimensions are determined, and the minimum bounding box side length l x , l y , l z is calculated.

[0060]

[0061] The voxel grid side length r is set, and the X, Y and Z coordinate axes are equally divided into L, M and N parts. The calculation principle of L, M and N is shown in formula (2), wherein is the floor function, and in the present embodiment, the voxel grid is selected as 0.25.

[0062]

[0063] Each voxel grid is numbered (i, j, k), and the voxel grid to which each point (x i , y i , z i ) belongs is confirmed, and the numbering process is as follows:

[0064]

[0065] Finally, the center of gravity of the point cloud in each voxel grid or the data point closest to the center of gravity is used to replace all the points in the grid, thereby completing the downsampling of the data. The larger the voxel grid side length is selected, the smaller the amount of point cloud data after processing is, and the more serious the loss of geometric features is. In order to better preserve the original data, the point closest to the center of gravity of the voxel grid can be used to replace the point cloud data in the voxel grid, and the reason for this is that the data value of this data point does not change compared with the direct use of the center of gravity to replace it.

[0066] Step (2.3) removes obvious outliers in the point cloud data through statistical filtering. The characteristics of outliers are often sparse and discrete in space. By calculating the average distance of the points from their surrounding points and constructing a Gaussian distribution, outliers can be removed by setting a threshold;

[0067] For the point cloud data processed by step (2.2), there are still discrete and sparse points near the point cloud of the sheet metal part that cannot be removed. The present application processes the sparse discrete points by introducing statistical filtering, and the basic principle of statistical filtering is as follows.

[0068] Statistical filtering assumes that the distance of points in the point cloud satisfies Gaussian distribution. The distribution is determined by the mean μ and the standard deviation σ. The main idea of statistical filtering is to calculate the distance dis of the point p i from any point p j in its neighborhood, and solve the distance dis:

[0069] dis = ||p i -p j || (4)

[0070] and solve the mean and standard deviation of the distance, where n is the number of sample points:

[0071]

[0072]

[0073] By setting the search range and the standard deviation coefficient, the outlier points can be removed. In this embodiment, the standard deviation coefficient is selected as 2.

[0074] Step (2.4) can cause point clouds perpendicular to the surface to appear at the edge during the scanning process due to reflection and reflection. By using a region growing segmentation algorithm, a smoothing parameter and a curvature radius are given to complete the segmentation of different regions of the point cloud data, and finally the sheet metal cover part surface point cloud data with relatively less noise can be obtained.

[0075] For the noise caused by reflection in the point cloud perpendicular to the tangent plane, the present application uses a region growing segmentation algorithm to remove it. This algorithm sets a seed point and uses the angle between the connecting line of the neighborhood point and the seed point and the normal line of the seed point as the smoothing value. When the calculation result is less than the set smoothing threshold, the neighborhood is added to the clustering region. By calculating the curvature of the neighborhood points and comparing it with the set curvature threshold, if it is less than the given threshold, the point is added to the empty seed point sequence. After the initial neighborhood point judgment is completed, the seed point is deleted and a new seed point is selected in the seed point sequence and the above steps are repeated until the seed point sequence is empty, and a clustering region is generated. The selection of the initial seed point is related to the curvature, and the selected seed point has the smallest curvature in the point set. In this embodiment, the curvature threshold is set to 10 and the smoothing threshold is set to 80. The effect of the final denoising by the above step (2) is shown in Figure 3 .

[0076] Step (3) extracts the boundary of the processed point cloud and stores each hole data and noise data in the boundary point cloud independently through a clustering method. The data variance threshold is used to classify and label the circular holes, waist-shaped holes, and noise.

[0077] Step (3.1) gives a neighborhood range, which is accelerated by Kdtree and uses principal component analysis to solve the point cloud normal vector information;

[0078] In order to provide normal information for step (3.2), the processed point cloud needs to be calculated for normal information. The specific idea is to index the neighborhood points through Kdtree, and fit the local plane through the least square method. The direction vector of the local plane can be obtained through principal component analysis.

[0079] Step (3.2) extracts the boundary points according to the existing normal vector information of the point cloud through the angle criterion;

[0080] The basic idea of the angle criterion for boundary point determination is to select a point p, determine the tangent plane of the point, and project the neighborhood points to the plane. Rotate in a fixed direction with p as the center, connect the searched points with p, and solve the included angle between the adjacent lines. A series of angles θ can be obtained by traversing the neighborhood.

[0081] θ = {θ1, θ2,...., θn} (7) n} (7)

[0082] Where n = neighborhood point number - 1.

[0083] Find the maximum value in the included angle and compare it with the set angle threshold value. If it is greater than the set threshold value ξ, it is determined that the point is a boundary point. In this embodiment, the threshold value ξ is selected as 1.88496.

[0084] θ max > ξ (8)

[0085] The finally obtained point cloud boundary points are shown in Figure 4 .

[0086] Step (3.3) processes the data of the extracted boundary points based on the density-based clustering algorithm (Density-Based Spatial Clustering of Applications with Noise), thereby segmenting each hole with spatial distance and storing them separately. The outer contour with more points is removed according to the upper point number threshold, and the scattered boundary with calculation error is removed according to the lower point number threshold;

[0087] In order to facilitate subsequent steps of model classification and fitting of data, and based on the characteristics of the hole edge point cloud in space, the present application uses a density-based clustering algorithm to complete the separate storage of the hole information for subsequent traversal.

[0088] The algorithm clusters according to density connectivity and has strong shape interference resistance. The number of clusters does not need to be pre-set before application, and has good robustness when the number of input sheet metal holes is unknown, and has good classification results for nonlinear data. The algorithm needs to provide a neighborhood point radius and a minimum number of neighborhood points threshold. When the number of neighborhood points in the point radius range meets the minimum number threshold, it is defined as a core point. The core point searches the density reachable objects by iteration and merges them, and the clustering process ends when no point can be added to the cluster. Continue to find new core points and loop the above process. In this embodiment, the neighborhood point radius is selected as 3.7mm, and the minimum number of neighborhood points threshold is selected as 20.

[0089] The point cloud extracted by the edge extraction contains a large amount of data in the outer contour, and the edge error extraction point cloud caused by the data loss of the gully position due to reflection. The outer contour is far more than the number of hole edge points, and can be successfully removed by setting the upper limit of the cluster points. And the type whose data amount is far less than the hole edge point can be removed by setting the lower limit of the cluster points. The final clustering result is shown in Figure 5 , different gray values represent different clusters. The clustered data types include round holes, waist-shaped holes and noise.

[0090] Step (3.4) traverses the separately stored data, calculates the distance of each point to the centroid and solves the distribution variance. According to the variance characteristics, the corresponding threshold is given;

[0091] In order to automatically distinguish the data type of the clustering result in step (3.3), the present application sets a threshold for the variance of the distance of the data point to the centroid. Considering the shape and spatial distribution characteristics of the data, the dispersion degree of the distance to the centroid can be summarized as follows: the noise class is greater than the waist-shaped hole class, and the waist-shaped hole class is greater than the round hole class. Finally, through the traversal of the three types of data, their respective data type distribution histograms are shown in Figure 6 .

[0092] Step (3.5) classifies and gives corresponding labels according to the variance threshold for waist-shaped holes, round holes and noise;

[0093] By calculating the variance distribution of different data, the data in the threshold interval is classified. And give the label, it is convenient for the automatic discrimination of the algorithm model in the subsequent steps.

[0094] Step 4) different algorithm models are used to fit and solve for different data label types, and the specific steps are as follows:

[0095] Step (4.1) input the point cloud data into the corresponding algorithm model according to the point cloud label difference;

[0096] Different data types are placed into the corresponding algorithm model through an IF statement.

[0097] Step (4.2) when the label is a round hole, a least square space circle fitting based on random sample consensus (RANSAC) is adopted. The space circle equation obtained by solving the algorithm is less affected by error points, and finally the diameter and center coordinates of the round hole can be obtained according to the fitting result. This feature contains the geometric and position information of the round hole.

[0098] Since direct application of least square fitting will cause the final fitting model to be affected by outliers, the RANSAC idea is introduced to avoid this problem. Compared with the traditional two-dimensional camera non-contact measurement aperture, the present application can more accurately obtain the position degree information of the inclined surface round hole due to the use of space circle fitting.

[0099] The specific process of its implementation is shown in Figure 7 Three parameters need to be selected in RANSAC according to experience, which are the number of sampling points n, the inlier judgment distance threshold t and the inlier ratio d. The number of sampling points n is the data point used to solve the model parameters in each iteration, the inlier judgment distance threshold t is used to include the points within the specified range of the solved model into the inliers for later model quality evaluation, and the inlier ratio d is the minimum inlier number value that identifies the better model and parameters. The iteration number k can be calculated according to the following formula.

[0100]

[0101] Where: p represents the probability of the RANSAC algorithm result being useful, and w represents the probability of the data being an inlier set, which satisfies formula (10):

[0102] 1-p=(1-w n ) k (10)

[0103] In this embodiment, the maximum iteration number is set to 100, and the distance threshold is set to 0.2mm.

[0104] The space circle to be solved in this process can be solved by a plane equation and a spherical equation. The representation is shown in formula (11), that is, the center coordinates, radius and space circle plane normal vector need to be known.

[0105]

[0106] In the formula, A, B, C, D are plane equation coefficients, R is the radius of the sphere, (x, y, z) is the sample point coordinate, and (x0, y0, z0) is the sphere center coordinate.

[0107] The model estimates the spatial equation coefficients by the least square method and continuously iterates to obtain an optimal solution.

[0108] In the embodiment, the vernier caliper measurement result is the true value. Compared with the vernier caliper measurement result, the average error is 0.05 mm.

[0109] When the label of step (4.3) is a waist-shaped hole, first, the random sample consensus method of a spatial straight line is used for fitting to obtain two spatial straight line equations. One of the straight lines is selected, and the inliers under the equation are obtained according to a given threshold value and the index thereof is obtained. The inliers are traversed, the distance thereof to the spatial straight line is solved, and the average value is calculated. The average value is the hole width of the waist-shaped hole. The points near the two spatial straight lines are removed, and the remaining point cloud is two semicircular arcs. The least square spatial circle fitting based on the random sample consensus (RANSAC) is used for the two semicircular arcs respectively, and the center coordinates and the radius value of the two semicircular arcs are solved. The distance between the two center coordinates and the sum of the two semicircular arc radii are the hole length of the waist-shaped hole.

[0110] Since the two sides of the waist-shaped hole are two spatial straight lines, the RANSAC model fitting method in step (4.2) is replaced by the least square method fitting a spatial straight line. When the inliers are solved, the index thereof is recorded, and when the model is the best model, the inliers and the neighborhood points participating in the calculation are removed. One of the spatial straight lines is selected, the distance of the inliers of the other straight line model to the straight line equation is solved, and the average value is solved. The average value is the hole width of the waist-shaped hole. The fitting method in step (4.2) is used for solving the remaining two semicircular arcs respectively, and the center distance of the two semicircular arcs is obtained. Thus, the full-size measurement of the waist-shaped hole is completed.

[0111] Compared with the size measured by the vernier caliper, the average error can be guaranteed to be within 0.1 mm.

[0112] When the label of step (4.4) is noise, the part of data is emptied and the memory space is released

[0113] If the data is determined to be noise, the data is directly deleted and the memory space is released.

[0114] The above only describes preferred examples of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made to the present application shall be included in the protection scope of the present application.

Claims

1. A sheet metal part hole group classification and full-size measurement method based on 3D point cloud processing, characterized by: Comprising the following steps: Step 1) install line scanning equipment and obtain point cloud; Step 2) traverse the obtained sheet metal cover point cloud, implement denoising and downsampling, obtain a point cloud with 5~10 times less data than the original point cloud and less noise than the original point cloud, so as to improve the operation efficiency of the subsequent algorithm; Step 3) extract the boundary of the processed point cloud, and independently store each hole data and noise data in the boundary point cloud by clustering method; classify and label the round holes, waist-shaped holes and noises by data variance threshold, and the specific steps are as follows: 3.1) give a neighborhood range by Kdtree acceleration, and use principal component analysis to solve the point cloud normal vector information; 3.2) according to the existing normal vector information of the point cloud, the boundary points are extracted by angle judgment criterion; 3.3) the data of the extracted boundary points are used to solve the spatial distance of each hole by using the density-based clustering algorithm, and the data are stored separately; the outer contour with the most points is removed according to the upper limit of the point number threshold, and the scattered boundary with calculation error is removed according to the lower limit of the point number threshold; 3.4) traverse the separately stored data, calculate the distance of each point to the centroid and solve the distribution variance; a threshold is given according to the variance characteristics; 3.5) classify and label the waist-shaped holes, round holes and noises according to the variance threshold; Step 4) remove the point cloud data with noise label, and fit the round holes and waist-shaped holes respectively to complete the full-size measurement; the round holes are fitted by using the random sample consensus least squares method based on spatial circle to complete the diameter and position measurement of the round holes; the waist-shaped holes are fitted by using the random sample consensus spatial straight line fitting for the two straight sides, and the straight line part is removed according to the index, and the remaining two circular arcs are fitted by using the random sample consensus least squares fitting based on spatial circle; Different algorithm models are used to fit and solve different data label types, and the specific steps are as follows: 4.1) according to the difference of point cloud label, the point cloud data is input into the corresponding algorithm model; 4.2) when the label is a round hole, the least squares spatial circle fitting based on random sample consensus is used; the spatial circle equation obtained by this algorithm is less disturbed by error points than the least squares fitting, and the diameter and center coordinates of the round hole are finally obtained according to the fitting result; the features include the geometry and position information of the round hole; 4.3) when the label is a waist-shaped hole, first, the random sample consensus method of spatial straight line is used for fitting to obtain two spatial straight line equations; one of the straight lines is selected, and the inliers under the equation are obtained according to the given threshold to obtain their index; the inliers are traversed and the distance from the inliers to the spatial straight line is solved, and the average distance is calculated, which is the hole width of the waist-shaped hole; the points near the two spatial straight lines are removed, and the remaining point cloud is two semicircular arcs; the least squares spatial circle fitting based on random sample consensus is used for the two semicircular arcs respectively to solve the center coordinates and radius of the two semicircular arcs, and the distance between the two center coordinates and the sum of the radii of the two semicircular arcs are the hole length of the waist-shaped hole; 4.4) when the label is noise, the data in this part is emptied and the memory space is released.

2. The method for sheet metal hole group classification and full size measurement based on 3D point cloud processing according to claim 1, characterized in that: Step 2) The obtained sheet metal cover point cloud is traversed to implement denoising and downsampling, and the specific steps are as follows: 2.1) First, the point cloud is filtered through a pass-through filter to remove background point clouds and point clouds that deviate too much from the measured part, significantly reducing the amount of point cloud data while preliminarily filtering the point cloud data; 2.2) The point cloud is downsampled by voxel filtering. The point cloud density obtained by the line scanner is high. In order to reduce the running time of the algorithm and realize online monitoring, the point cloud needs to be downsampled. The point cloud after voxel filtering and downsampling can reduce the data amount while retaining the original geometric features; 2.3) The statistical filter is used to remove obvious outliers in the point cloud data. The feature of outliers is sparse distribution in space. The average distance of points and their surrounding points is calculated and a Gaussian distribution is constructed. Outliers are removed by setting a threshold; 2.4) During the scanning process, due to reflection and reflection, there will be point clouds perpendicular to the surface at the edge. By using the region growing segmentation algorithm, the smoothness parameter and the curvature radius are given to complete the segmentation of different regions of the point cloud data, and finally the sheet metal cover surface point cloud data with fewer noise points is obtained than step (2.3).

Images