Defect detection method based on multi-scale constraint segmentation
By using a multi-scale constrained segmentation method to smooth point cloud data and perform region growing segmentation, the problem of false detection of defect regions and inherent geometric features in point cloud segmentation is solved, and efficient and accurate defect detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-09
- Publication Date
- 2026-03-24
AI Technical Summary
Existing point cloud segmentation methods struggle to accurately distinguish between defect areas and inherent geometric features of complex workpieces, leading to false detections. Furthermore, these methods are computationally time-consuming, highly sensitive to parameters, and have limited applicability.
A multi-scale constrained segmentation method is adopted. By smoothing point cloud data, calculating normals and curvature, and combining multi-scale region growth segmentation and geometric parameter quantization, defect regions on the surface of objects are detected.
It improves the accuracy of defect detection, reduces false detections of inherent geometric features of objects, and significantly enhances detection efficiency and accuracy.
Smart Images
Figure CN121724928A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of computer vision, and particularly relates to a defect detection method based on multi-scale constraint segmentation. BACKGROUND
[0002] With the continuous progress of three-dimensional measurement technology, three-dimensional point cloud data has become an important data form indispensable in the field of computer vision and intelligent manufacturing. Surface defect detection technology based on point cloud data has become a key link in industrial production process, which can detect defects in products early and improve production efficiency.
[0003] When using three-dimensional scanning technology to obtain point cloud data, the object surface usually has inherent geometric features, free-form surfaces and various surface defects at the same time. Existing surface defect detection methods mainly include point cloud registration-based detection methods and point cloud segmentation-based detection methods. The former is only suitable for regular products with standard three-dimensional models, and the latter can realize defect detection of complex workpieces without standard models by extracting point cloud geometric features, and has a wider range of application. Existing methods mainly include region growing-based point cloud segmentation method, clustering-based point cloud segmentation method and model fitting-based point cloud segmentation method. Among them, the region growing-based point cloud segmentation method has problems of unstable local segmentation performance, long calculation time and sensitivity to parameters, and is easy to misjudge the inherent geometric features of the object as defect regions; the clustering-based point cloud segmentation method is too dependent on point cloud density and quite time-consuming; the model fitting-based point cloud segmentation method can only segment planar point clouds, and has low applicability to irregular curved surfaces and complex defects. It can be seen that the existing methods are difficult to distinguish between defect regions and inherent geometric features of the object, and it is necessary to design a new segmentation method for defect detection. SUMMARY
[0004] To solve the problems existing in the existing methods, the application proposes a defect detection method based on multi-scale constraint segmentation, which solves the problem of false detection of structure edges in complex point cloud surface defect detection.
[0005] A defect detection method based on multi-scale constraint segmentation, comprising the following steps:
[0006] Step 1: obtaining three-dimensional point cloud data of the object surface using a three-dimensional laser scanner;
[0007] Step 2: smoothing the point cloud to obtain a smoothed point cloud;
[0008] Step 3: calculating the normal and curvature of the smoothed point cloud;
[0009] Step 4: performing multi-scale constraint region growing segmentation on the point cloud to detect the defect region on the surface of the object;
[0010] Step 5: Quantize the geometric parameters of the defective region.
[0011] Furthermore, in step two, the point cloud is smoothed to obtain a smoothed point cloud, including the following steps:
[0012] 1) For a target point in the point cloud, find the k nearest neighbors in its neighborhood to form a local neighborhood point set;
[0013] 2) Based on the local neighborhood point set, fit the local reference plane of the target point by minimizing the weighted squared error, and use a Gaussian function to control the decay rate of the weights;
[0014] 3) After determining the local reference plane, the local neighborhood point set is projected onto the reference plane to establish a local coordinate system. Weighted fitting is performed in the local coordinate system using a quadratic polynomial to construct a local fitted surface.
[0015] 4) Map the target point onto the local fitted surface to obtain the smoothed position of the target point and generate a smoothed point cloud.
[0016] Furthermore, in step three, calculating the normal and curvature of the smoothed point cloud includes the following steps:
[0017] 1) Use the kd-tree method to search for the neighborhood points of each target point p in the smoothed point cloud;
[0018] 2) Calculate the centroid of the neighborhood points. Given the covariance matrix C, perform singular value decomposition on the covariance matrix C to solve for its eigenvalues λ1, λ2, λ3 (satisfying λ1≤λ2≤λ3) and their corresponding eigenvectors v1, v2, v3;
[0019] 3) The eigenvector v1 corresponding to the minimum eigenvalue λ1 is used as the normal vector of the target point p. The curvature of the target point p is calculated based on the eigenvalue, wherein the calculation method is as follows:
[0020]
[0021] Furthermore, in step four, multi-scale constrained region growing and segmentation is performed on the point cloud to detect defect regions on the object surface, including the following steps:
[0022] 1) During the region growth initialization phase, all points in the point cloud are sorted in ascending order based on their curvature, and the point with the smallest curvature in the unlabeled point set is selected as the initial seed point p. After adding the seed point to the seed point queue, a kd-tree structure is used to quickly search the neighborhood point set N in the point cloud. pSubsequently, for each neighboring point q∈N in the neighborhood point set... p Perform geometric consistency checks sequentially;
[0023] 2) Calculate the angle between the normal of the neighboring point and the normal of the seed point. When the angle between the normals is less than the set threshold, the neighboring point and the seed point are considered to be located on the same smooth surface.
[0024] 3) If the included angle of the normal is greater than the included angle of the normal threshold, the neighboring point is directly excluded and the subsequent determination process for the neighboring point is terminated; if the included angle of the normal is less than the included angle of the normal threshold, it is further determined whether the curvature value of the neighboring point is less than the preset curvature threshold.
[0025] 4) When the included angle of the normal and the curvature value simultaneously satisfy the corresponding threshold constraints, the neighborhood point q is included in the current region and used as a candidate point for subsequent growth;
[0026] 5) For the candidate point, calculate the overall consistency metric of its normal and the normals of its k nearest neighbors to determine whether the normal directions of the candidate point and its k nearest neighbors are consistent. The consistency metric is calculated as follows:
[0027]
[0028] In the formula, N q Let |N| be the set of nearest neighbors of the candidate point. q | represents the number of nearest neighbors, n q Let n be the normal to the candidate point. j The normal to the nearest neighbor point;
[0029] When the consistency metric D q Greater than or equal to the consistency threshold D th When the candidate point is added to the current growth region;
[0030] 6) After completing the geometric consistency determination of the neighborhood points, the current seed point p is removed from the seed queue, and a new seed point is selected from the queue. The region growth segmentation based on normal, curvature and neighborhood consistency determination is repeated until the seed point queue is empty. Then the current region growth ends, and the corresponding defect region point cloud is obtained from the original point cloud.
[0031] Furthermore, in step five, the geometric parameters of the defective region are quantized, including the following steps:
[0032] 1) Using the point cloud of the defect neighborhood as a reference, the least squares method is used to fit the reference plane;
[0033] 2) Based on the reference plane, calculate the vertical distance from each point in the defect area to the reference plane, where the maximum vertical distance is defined as the maximum depth of the defect;
[0034] 3) Project the point cloud of the defect onto the reference plane to obtain a two-dimensional projection point set. By traversing the Euclidean distance between all points, the maximum diameter of the defect is obtained.
[0035] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention can accurately detect the defect region on the surface of the object by performing multi-scale constrained region growth segmentation on the point cloud. The present invention overcomes the problem of false detection of the inherent geometric features of the object in the surface defect detection of complex point clouds in the prior art, and can significantly improve the accuracy of defect detection. Attached Figure Description
[0036] Appendix Figure 1 This is a flowchart of a defect detection method based on multi-scale constraint segmentation according to the present invention. Detailed Implementation
[0037] To further understand the present invention, preferred embodiments of the present invention are described below. However, it should be understood that these descriptions are only for further illustrating the features and advantages of the present invention, and not for limiting the scope of the claims of the present invention.
[0038] A defect detection method based on multi-scale constraint segmentation, such as Figure 1 As shown, the method includes the following steps:
[0039] Step 1: Obtain 3D point cloud data of the object's surface using a 3D laser scanner;
[0040] Step 2: Smooth the point cloud to obtain a smoothed point cloud;
[0041] Step 3: Calculate the normal and curvature of the smoothed point cloud;
[0042] Step 4: Perform multi-scale constrained region growing and segmentation on the point cloud to detect defect regions on the object surface;
[0043] Step 5: Quantize the geometric parameters of the defective region.
[0044] The above technical solution, in step two, involves smoothing the point cloud to obtain a smoothed point cloud; this includes the following steps:
[0045] 1) For a target point in the point cloud, find the k nearest neighbors in its neighborhood to form a local neighborhood point set;
[0046] 2) Based on the local neighborhood point set, fit the local reference plane of the target point by minimizing the weighted squared error, and use a Gaussian function to control the decay rate of the weights;
[0047] 3) After determining the local reference plane, the local neighborhood point set is projected onto the reference plane to establish a local coordinate system. Weighted fitting is performed in the local coordinate system using a quadratic polynomial to construct a local fitted surface.
[0048] 4) Map the target point onto the local fitted surface to obtain the smoothed position of the target point and generate a smoothed point cloud.
[0049] In the above technical solution, step three, calculating the normals and curvature of the smoothed point cloud, includes the following steps:
[0050] 1) Use the kd-tree method to search for the neighborhood points of each target point p in the smoothed point cloud;
[0051] 2) Calculate the centroid of the neighborhood points. The covariance matrix C is calculated as follows:
[0052]
[0053] In the formula, k is the total number of neighborhood points, p i Let i be the i-th point in the neighborhood;
[0054] Perform singular value decomposition on the covariance matrix C to solve for its eigenvalues λ1, λ2, λ3 (satisfying λ1≤λ2≤λ3) and their corresponding eigenvectors v1, v2, v3;
[0055] 3) The eigenvector v1 corresponding to the minimum eigenvalue λ1 is used as the normal vector of the target point p. The curvature of the target point p is calculated based on the eigenvalue, and the calculation method is as follows:
[0056]
[0057] In the above technical solution, step four, which involves performing multi-scale constrained region growing and segmentation on the point cloud to detect defect regions on the object surface, includes the following steps:
[0058] 1) During the region growth initialization phase, all points in the point cloud are sorted in ascending order based on their curvature, and the point with the smallest curvature in the unlabeled point set is selected as the initial seed point p. After adding the seed point to the seed point queue, a kd-tree structure is used to quickly search the neighborhood point set N in the point cloud. p Subsequently, for each neighboring point q∈N in the neighborhood point set... pPerform geometric consistency checks sequentially;
[0059] 2) Calculate the angle between the normals of the neighboring point and the normal of the seed point. When the angle is less than a set threshold, the neighboring point and the seed point are considered to be on the same smooth surface. The constraint condition can be expressed as:
[0060] arccos(n p ·n q )<α th
[0061] In the formula, n p Let n be the normal to the seed point. q Let α be the normal to the neighboring points. th The included angle threshold;
[0062] 3) If the included angle of the normal is greater than the included angle threshold, the neighboring point is directly excluded, and the subsequent determination process for that neighboring point is terminated; if the included angle of the normal is less than the included angle threshold, it is further determined whether the curvature value of the neighboring point is less than a preset curvature threshold, wherein the constraint condition can be expressed as:
[0063] c q <c th
[0064] In the formula, c q c is the curvature value of the neighborhood point. th The curvature threshold;
[0065] 4) When the included angle of the normal and the curvature value simultaneously satisfy the corresponding threshold constraints, the neighborhood point q is included in the current region and used as a candidate point for subsequent growth;
[0066] 5) For the candidate point, calculate the overall consistency metric of its normal and the normals of its k nearest neighbors to determine whether the normal directions of the candidate point and its k nearest neighbors are consistent. The consistency metric is calculated as follows:
[0067]
[0068] In the formula, N q Let |N| be the set of nearest neighbors of the candidate point. q | represents the number of nearest neighbors, n q Let n be the normal to the candidate point. j The normal to the nearest neighbor point;
[0069] When the consistency metric D q Greater than or equal to the consistency threshold D th When the candidate point is added to the current growth region;
[0070] 6) After completing the geometric consistency determination of the neighborhood points, the current seed point p is removed from the seed queue, and a new seed point is selected from the queue. The region growth segmentation based on normal, curvature and neighborhood consistency determination is repeated until the seed point queue is empty. Then the current region growth ends, and the corresponding defect region point cloud is obtained from the original point cloud.
[0071] In the above technical solution, step five, which involves quantizing the geometric parameters of the defective region, includes the following steps:
[0072] 1) Using the point cloud of the defect neighborhood as a reference, a reference plane is fitted using the least squares method. The method for calculating the fitted plane is as follows:
[0073] ax + by + cz + d = 0
[0074] In the formula, a, b, and c are the components of the plane normal vector; d is a constant term.
[0075] 2) Based on the reference plane, calculate the perpendicular distance from each point within the defect area to the reference plane, wherein the calculation formula is:
[0076]
[0077] In the formula, h i Indicates the local depth of the i-th point;
[0078] Maximum depth value among all points Defined as the maximum depth of the defect;
[0079] 3) Project the point cloud of the defect onto the reference plane to obtain a two-dimensional projection point set. The maximum diameter of the defect is obtained by traversing the Euclidean distances between all points, where the calculation formula is:
[0080]
[0081] In the formula, q i Let q be the i-th point in the set of projection points. j Let (x′) be the j-th point in the set of projection points. i ,y′ i ,z′ i ) is q i The coordinates, (xj j ,y′ j ,z′ j ) is q j The coordinates;
[0082] The maximum Euclidean distance among all points It is defined as the maximum diameter of the defect.
Claims
1. A defect detection method based on multi-scale constraint segmentation, characterized in that, The method includes: obtaining point cloud data of an object surface using a 3D laser scanner; smoothing the point cloud to obtain a smoothed point cloud; calculating the normal and curvature of the smoothed point cloud; performing multi-scale constrained region growing segmentation on the point cloud to detect defect regions on the object surface; and quantizing the geometric parameters of the defect regions.
2. The detection method according to claim 1, characterized in that, The point cloud is smoothed to obtain a smoothed point cloud, including the following steps: Step 1: For the target point in the point cloud, find the k nearest neighbors in its neighborhood to form a local neighborhood point set. Step 2: Based on the local neighborhood point set, fit the local reference plane of the target point by minimizing the weighted squared error, and use a Gaussian function to control the decay rate of the weights. Step 3: After determining the local reference plane, project the local neighborhood point set onto the reference plane to establish a local coordinate system. Under the local coordinate system, perform weighted fitting using a quadratic polynomial to construct a local fitted surface. Step 4: Map the target point onto the local fitted surface to obtain the smoothed position of the target point and generate a smoothed point cloud.
3. The detection method according to claim 1, characterized in that, Calculating the normals and curvature of the smoothed point cloud includes the following steps: Step 1: Use the kd-tree method to search for neighborhood points of each target point p in the smoothed point cloud; Step 2: Calculate the centroid of the neighborhood points. Given the covariance matrix C, perform singular value decomposition on the covariance matrix C to solve for its eigenvalues λ1, λ2, λ3 (satisfying λ1≤λ2≤λ3) and their corresponding eigenvectors v1, v2, v3; Step 3: Take the eigenvector v1 corresponding to the minimum eigenvalue λ1 as the normal vector of the target point p. The curvature of the target point p is calculated based on the eigenvalue, wherein the calculation method is as follows:
4. The detection method according to claim 1, characterized in that, Performing multi-scale constrained region growing and segmentation on the point cloud to detect defect regions on the object surface includes the following steps: Step 1: In the region growth initialization phase, sort all points in the point cloud in ascending order based on their curvature, and select the point with the smallest curvature from the unlabeled point set as the initial seed point p. After adding the seed point to the seed point queue, use a kd-tree structure to quickly search the neighborhood point set N in the point cloud. p Subsequently, for each neighboring point q∈N in the neighborhood point set... p Perform geometric consistency checks sequentially. Step 2: Calculate the angle between the normal of the neighboring point and the normal of the seed point. When the angle between the normals is less than the set threshold, the neighboring point and the seed point are considered to be located on the same smooth surface. Step 3: If the included angle of the normal is greater than the included angle of the normal threshold, the neighboring point is directly excluded and the subsequent determination process for the neighboring point is terminated; if the included angle of the normal is less than the included angle of the normal threshold, it is further determined whether the curvature value of the neighboring point is less than the preset curvature threshold. Step 4: When the included angle of the normal and the curvature value simultaneously satisfy the corresponding threshold constraints, the neighborhood point q is included in the current region and used as a candidate point for subsequent growth; Step 5: For the candidate point, calculate the overall consistency metric between its normal and the normals of its k nearest neighbors to determine whether the normal directions of the candidate point and its k nearest neighbors are consistent. The consistency metric is calculated as follows: In the formula, N q Let |N| be the set of nearest neighbors of the candidate point. q | represents the number of nearest neighbors, n q Let n be the normal to the candidate point. j The normal to the nearest neighbor point; When the consistency metric D q Greater than or equal to the consistency threshold D th When the candidate point is added to the current growth region, it is then added to the current growth region. Step 6: After completing the geometric consistency determination of the neighborhood points, remove the current seed point p from the seed queue, select a new seed point from the queue, and continue to repeat the region growth segmentation based on normal, curvature and neighborhood consistency determination until the seed point queue is empty. Then the current region growth ends, thereby segmenting the corresponding defect region point cloud from the original point cloud.
5. The detection method according to claim 1, characterized in that, The geometric parameter quantization of the defect region includes the following steps: Step 1: Using the point cloud of the defect neighborhood as a reference, fit the reference plane using the least squares method; Step 2: Based on the reference plane, calculate the vertical distance from each point in the defect area to the reference plane, wherein the maximum value of the vertical distance is defined as the maximum depth of the defect; Step 3: Project the point cloud of the defect onto the reference plane to obtain a two-dimensional projection point set. By traversing the Euclidean distance between all points, the maximum diameter of the defect is obtained.