Industrial surface defect detection method based on three-dimensional point cloud data

Abstract

The invention discloses an industrial surface defect detection method based on three-dimensional point cloud data, and the method comprises the steps: carrying out the noise removal and point cloud downsampling of the inputted three-dimensional point cloud data, carrying out the analysis of the preprocessed point cloud data through a plurality of foreground extractors, and extracting a candidate defect point set; thirdly, performing spatial clustering on the extracted candidate defect point set through a plurality of spatial clustering devices to generate a plurality of candidate defect instance clusters; classifying each candidate defect instance cluster, calculating the probability of being a defect, and screening out candidate instances which are possibly real defects; and removing overlapped and redundant defect instance clusters, and finally outputting the detected three-dimensional surface defect instance cluster. According to the method, the accuracy of defect detection is remarkably improved, and noise and errors can be effectively processed.

Description

Technical Field

[0001] The invention relates to an industrial surface defect detection method based on three-dimensional point cloud data, belonging to the technical field of defect detection. Background Art

[0002] In the modern industrial manufacturing process, product quality control and management has always been one of the key factors to ensure product performance and safety. Especially in high-tech fields such as aerospace, automobile manufacturing, shipbuilding, electronic manufacturing, and precision machinery manufacturing, tiny defects on the product surface may have a significant impact on the performance of the final product. Traditional surface defect detection methods mainly rely on technical means such as visual inspection, ultrasonic inspection, and magnetic particle inspection. However, when faced with complex surfaces and tiny defects, these methods often have problems such as low detection accuracy, low efficiency, and strong dependence on operator experience, which makes it difficult to meet the high-precision and high-efficiency requirements of modern industrial production.

[0003] With the rapid development of computer vision technology, surface defect detection technology based on image processing has gradually become a research hotspot. However, due to the limitations of two-dimensional images, accurate defect detection on complex surfaces or three-dimensional structures still faces many challenges. Two-dimensional images can only capture the surface texture information of objects and cannot fully reflect the three-dimensional geometric shape of objects. For workpieces with complex shapes and irregular surfaces, detection methods based on two-dimensional images are prone to misjudgment or omission. In addition, factors such as lighting conditions and surface reflectivity will also significantly affect the stability and reliability of detection results.

[0004] In order to overcome the above problems, the acquisition and processing technology of 3D point cloud data has gradually attracted widespread attention in recent years. 3D point cloud data can fully describe the geometric shape and spatial structure of an object, providing richer and more accurate data information for surface defect detection. Point cloud data is acquired through laser scanning, structured light scanning, time-of-flight (TOF) sensors and other equipment, which can accurately capture the 3D shape of an object, especially when dealing with complex surfaces and tiny structures, and has obvious advantages.

[0005] Although 3D point cloud data has the advantages of high precision and high information content, the difficulty of data processing has also increased significantly. Point cloud data usually consists of millions or even hundreds of millions of points. The data volume is huge and the structure is complex, which brings challenges to subsequent data processing, feature extraction and analysis. In addition, the point cloud data itself may have problems such as noise, occlusion and incompleteness, which further increases the complexity of surface defect detection. Therefore, how to effectively process 3D point cloud data, extract meaningful geometric features from it, and combine advanced algorithms to achieve efficient and accurate surface defect detection has become an important direction of current research.

[0006] Existing surface defect detection methods based on 3D point clouds can be divided into two categories: one is traditional methods, and the other is deep learning-based methods. Traditional methods mainly use local geometric features of point clouds, such as curvature, and then use threshold segmentation to remove background points and obtain defect points. Traditional methods are relatively intuitive, but if the segmentation threshold is not properly selected, the defect segmentation performance may be relatively poor, so its robustness and applicability deserve further improvement. For deep learning-based methods, the defect segmentation task can be regarded as a 3D semantic segmentation problem or a 3D instance segmentation problem in deep learning. PointNet, PointNet++, and Point transformer are typical advanced 3D semantic segmentation networks. However, even the latest state-of-the-art semantic segmentation networks still have a segmentation mean Intersection of Union (mIoU) performance index of less than 80% on public datasets.

[0007] Although existing surface defect detection methods based on 3D point clouds have achieved certain results in certain specific scenarios, they generally have problems such as high algorithm complexity, long calculation time, and high hardware requirements, which limit their promotion in actual industrial applications. In order to improve detection efficiency and accuracy, researchers have begun to explore the combination of machine learning, deep learning and 3D point cloud processing technology to automatically extract and identify defect features through learning models. However, most existing machine learning-based detection methods rely on a large amount of training data, and have limited generalization capabilities under different workpiece types or surface textures, and still need further optimization and improvement.

[0008] In summary, the existing surface defect detection technology still has many shortcomings when facing complex industrial scenarios. There is an urgent need for an efficient, robust and easy-to-promote three-dimensional surface defect detection method to meet the needs of modern industrial manufacturing for high-precision quality inspection. Summary of the invention

[0009] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and propose an industrial surface defect detection method based on three-dimensional point cloud data, which significantly improves the detection accuracy of complex surface defects through five detection designs including data preprocessing, multi-head foreground extraction, multi-head spatial clustering, shared classification and non-maximum suppression.

[0010] The technical solution of the present invention is:

[0011] An industrial surface defect detection method based on three-dimensional point cloud data, comprising:

[0012] Preprocess the collected 3D point cloud data to remove outliers;

[0013] Perform multi-head foreground extraction on the preprocessed 3D point cloud data to obtain a set of candidate defect points;

[0014] Perform multi-head spatial clustering on the candidate defect point set to generate multiple candidate defect instance clusters;

[0015] Classify each candidate defect instance cluster, obtain the defect probability of the selected defect instance cluster, and screen out the instance clusters that may be real defects;

[0016] The overlap and redundancy of the screened instance clusters that may be real defects are removed, and finally the detected three-dimensional surface defect instance clusters are output.

[0017] Furthermore, multi-head foreground extraction is performed on the preprocessed 3D point cloud data. The specific method is as follows:

[0018] Calculate the local geometric eigenvalues ​​of each point in the 3D point cloud;

[0019] Establish multiple parallel foreground extractors, set corresponding percentage segmentation thresholds for each foreground extractor, and perform the following processing in each foreground extractor: compare the local geometric feature value of each point in the three-dimensional point cloud with the percentage segmentation threshold of the foreground extractor. If the local geometric feature value is greater than the threshold, mark the point as a candidate defect point;

[0020] The candidate defect points marked in each foreground extractor are aggregated to form a candidate defect point set.

[0021] Furthermore, the local geometric eigenvalue of each point in the three-dimensional point cloud is calculated. The local geometric eigenvalue is the normal vector change rate. i The rate of change of the normal vector is:

[0022]

[0023] Among them, ||·|| represents the Euclidean norm of the vector, n i For point p i Normal vector of point p i The normal vector of a point in the neighborhood is {n i1 ,n i2 ,…,n iK}, K is the number of neighborhood points.

[0024] Furthermore, the local geometric eigenvalue of each point in the three-dimensional point cloud is calculated, and the local geometric eigenvalue is the full variance feature. i The total variance characteristics of:

[0025] F i =GVF=tr(Σ)=λ 1 +λ 2 +λ3

[0026] Among them, λ 1 ,λ 2 ,λ 3 is the eigenvalue of the variance matrix Σ, which is obtained by i The neighborhood points are obtained by principal component analysis.

[0027] Furthermore, multiple parallel spatial clusterers with different parameter values ​​are designed, and the candidate defect point set is processed by multiple spatial clustering by each spatial clusterer;

[0028] The method for performing spatial clustering processing within each spatial clusterer is:

[0029] For each point in the candidate defect point set, calculate the number of neighboring points within the radius eps of the point. If it exceeds the set value, mark the point as a core point.

[0030] Starting from any core point, the clusterer expands all points in the neighborhood of the core point to form clusters; the cluster expansion process includes similar processing of all points in the neighborhood until the cluster expansion can no longer include new points; points with less than minpts points in any cluster are marked as noise points and do not participate in the formation of any cluster; among them, eps and minpts are hyperparameters, and different values ​​are set in different spatial clusterers.

[0031] Furthermore, each candidate defect instance cluster is classified and the defect probability of each class is calculated. The specific method is as follows:

[0032] Extracting feature vectors from each candidate defect instance cluster, wherein the feature values ​​in the feature vectors include oriented bounding box feature values, convex hull feature values, and shape feature values;

[0033] Normalize the eigenvalues ​​of the eigenvector;

[0034] A mature classification model is used to classify the standardized feature vectors and predict the probability value of the surface defect category.

[0035] Furthermore, a classification model is established by a neural network algorithm or an XGBoost classifier, taking the standardized feature vector as input and the probability value of the surface defect as output;

[0036] The standardized feature vector values ​​obtained after data preprocessing, multi-head foreground extraction, multi-head spatial clustering, feature vector extraction and standardization of the 3D point cloud historical data and the surface defect values ​​corresponding to the 3D point cloud historical data are used as training data to train the classification model;

[0037] The trained classification model is used as a mature model for prediction.

[0038] Furthermore, the eigenvector is represented as X = [X 1 ,X 2 ,…,X 8 ], X 1 ,X 2 ,X 3 are the length, width, and height of the oriented bounding box, respectively. 4 ,X 5 are the surface area and volume of the convex hull, X 6 ,X 7 ,X 8 They are normal consistency, sphericity, and flatness of shape features respectively.

[0039] Furthermore, the overlap and redundancy of the selected instance clusters are removed. The specific method is as follows:

[0040] 1) Pair the instance clusters and calculate the intersection-over-union ratio of each pair of instance clusters:

[0041]

[0042] Among them, C i ∩C j Represents instance cluster C i and C j The intersection of i ∪C j Represents instance cluster C i and C j The union of;

[0043] 2) For each pair of instance clusters, if the intersection over union (IoU) i ,C j ) is greater than the specified threshold, the two instance clusters are considered overlapping clusters;

[0044] 3) For each pair of overlapping clusters, compare the defect probabilities of the two instance clusters, retain the instance cluster with a larger defect probability, and delete the instance cluster with a smaller defect probability;

[0045] 4) Collect the remaining instance clusters and repeat steps 1) to 3) until the intersection-union ratio of any two instance clusters after pairing does not exceed the specified threshold.

[0046] Furthermore, the collected 3D point cloud data is preprocessed to remove outliers by:

[0047] Extract the neighborhood point set for each point in the 3D point cloud data, calculate the Euclidean distance between each point and all points in the corresponding neighborhood point set, and find the average;

[0048] The average distance corresponding to each point in the 3D point cloud data is counted, and the outliers are determined and removed from the 3D point cloud data based on the average value and standard deviation of the average distance.

[0049] The advantages of the present invention compared with the prior art are:

[0050] (1) The present invention removes noise in the point cloud and enhances features through a data preprocessing layer, and combines the diversified feature extraction of a multi-head foreground extraction layer with the refined clustering of a multi-head spatial clustering layer, thereby solving the problems of misjudgment or missed detection of defects caused by environmental changes or complex backgrounds in the prior art, and achieving higher detection accuracy.

[0051] (2) The present invention designs a multi-head foreground extraction layer and a multi-head spatial clustering layer, effectively utilizes computing resources to perform parallel computing, solves the problem of slow detection speed caused by single-channel processing in the prior art, achieves an improvement in detection speed, and has real-time detection capabilities.

[0052] (3) The present invention realizes the reuse of network architecture by designing a shared classification layer and combines it with the intelligent screening mechanism of the non-maximum suppression layer to solve the problem of poor applicability of the prior art when processing workpieces of different materials and shapes, and realizes that the model can still maintain high efficiency and high detection effect in a variety of complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0054] Figure 1 This is an overall flow chart of the industrial surface defect detection method according to an embodiment of the present invention;

[0055] Figure 2 A detailed flow chart of the industrial surface defect detection method according to an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of a three-dimensional surface defect case detected by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0058] The present invention proposes an industrial surface defect detection method based on three-dimensional point cloud data, which significantly improves the detection accuracy of complex industrial surface defects. Figure 1 As shown, specifically including:

[0059] S1: Data preprocessing layer

[0060] In the collection of 3D point cloud data, outliers and noise may exist in the point cloud data due to equipment errors, environmental interference, etc. These outliers will affect the subsequent surface defect detection results. Therefore, the K nearest neighbor point extractor and outlier filter are first used to remove outliers and downsample the input 3D point cloud data to ensure the accuracy and reliability of the data.

[0061] S1.1: The nearest neighbor point set is extracted for each point using the K-Nearest Neighbors (KNN) method.

[0062] The K nearest neighbor method selects the nearest neighbor points of a point in the point cloud based on the distance metric. Let the point cloud data be P = {p 1 ,p 2 ,…,p n}, where p i =(x i ,y i ,z i ) represents the i-th point in the point cloud, whose coordinates are (x i ,y i ,z i ).

[0063] For any point p i , select its K nearest neighbor points by calculating the Euclidean distance between it and other points. The formula is as follows:

[0064]

[0065] By calculating the distance between other points, select the K points closest to it as point p i The neighbor point set N i .

[0066] S1.2: Outlier Filter

[0067] After the neighbor point set is extracted, the outliers in the point cloud data are filtered using statistical methods. Specifically, for each point p i , calculate the average distance between it and all neighboring points

[0068]

[0069] By counting the average distance of all points in the point cloud and standard deviation σ d , to determine whether each point is an outlier. If the average distance of a point Beyond the overall average distance A certain threshold (usually ), then the point is considered an outlier, where α is an adjustable parameter, usually set to 2 or 3 to cover 95% or 99% of the data:

[0070]

[0071] All points p that satisfy the above conditions i are marked as outliers and removed.

[0072] Through the above preprocessing steps, the present invention removes a large number of useless outliers in the three-dimensional point cloud data preprocessing stage, ensuring the cleanliness and integrity of the data, thereby improving the reliability and accuracy of subsequent detection steps.

[0073] S2: Long Foreground Extraction Layer

[0074] The preprocessed point cloud data is analyzed by multiple parallel foreground extractors with different parameters. Each foreground extractor identifies a set of candidate defect points based on different geometric features and segmentation thresholds.

[0075] For each foreground extractor E k , the specific steps are as follows:

[0076] S2.1: Feature calculation

[0077] Calculate the local geometric features F of each point in the point cloud i ,The main geometric features include the normal vector change rate and the total variance features.

[0078] Among them, the normal variation rate (NVR) is used to measure the point p i The degree of change of the normal vector of p in its local neighborhood. i The normal vector is n i , the normal vector of the point in its local neighborhood is {n i1 ,n i2 ,…,n iK}. The normal vector change rate can be calculated by the following formula:

[0079]

[0080] where ||·|| represents the Euclidean norm of the vector and K is the number of neighborhood points. A higher rate of change of the normal vector indicates the presence of larger surface irregularities or potential defects in the point cloud.

[0081] The global variance feature (GVF) is used to describe the geometric irregularity of the local area of ​​the point cloud. By performing principal component analysis (PCA) on the local point cloud, its variance matrix Σ is calculated. The global variance feature is the trace of the variance matrix, and the formula is as follows:

[0082] F i =GVF=tr(Σ)=λ 1 +λ 2 +λ 3

[0083] Among them, λ 1 ,λ 2 ,λ 3 is the eigenvalue of the variance matrix Σ. The full variance feature reflects the degree of geometric dispersion of the local area of ​​the point cloud.

[0084] S2.2: Threshold setting: Set a specific percentage segmentation threshold T for each foreground extractor k .

[0085] S2.3: Point cloud segmentation: For each point p i , if its geometric eigenvalue F i Greater than the threshold T k , then mark the point as a candidate defect point. The judgment formula is as follows:

[0086] If F i >T k then i is a defect candidate

[0087] S2.4: Generation of candidate defect point set: Gather all points that meet the conditions to form a candidate defect point set D k :

[0088] D k ={p i ∣F i >T k}

[0089] S3: Multi-head spatial clustering layer

[0090] Multiple parallel spatial clusterers with different parameters are used to perform spatial clustering on the candidate defect point set to generate multiple independent candidate defect instance clusters.

[0091] Each spatial clusterer uses a density-based spatial clustering algorithm (DBSCAN, Density-Based Spatial Clustering of Applications with Noise) to cluster defect instances. The core of the DBSCAN algorithm is to identify cluster structures by point density and effectively handle noise points. The steps of the DBSCAN algorithm used in this step include:

[0092] S3.1: Core point determination: For each point p i , calculate the number of its neighboring points within the radius eps. If the number of neighboring points is greater than or equal to minpts, then p i are marked as core points.

[0093] S3.2: Cluster expansion: Starting from a core point, the clusterer will include all points in its neighborhood and further expand these points to form clusters. The cluster expansion process involves similarly processing all points in the neighborhood until the cluster expansion can no longer include new points.

[0094] S3.3: Noise point identification: Points with less than minpts points in any cluster are marked as noise points and do not participate in the formation of any cluster.

[0095] Each spatial clusterer C in the multi-head spatial clustering layer k All with different hyperparameters (eps k ,minpts k ). Through these different hyperparameter configurations, defect clusters of different sizes and densities can be identified. Each spatial clusterer will generate a defect cluster set S k :

[0096] S k = {clusters formed by C k}

[0097] S4: Shared classification layer

[0098] like Figure 2 As shown in the figure, all candidate defect instance clusters are classified, the probability that the candidate defect instance cluster is a defect is calculated, and the candidate instances that may be real defects are screened out. The shared classification layer includes a shared candidate defect instance cluster feature extractor, a shared normalizer, and a shared classifier, which have the same hyperparameters.

[0099] S4.1: Shared candidate defect instance cluster feature extractor

[0100] The shared candidate defect instance cluster feature extractor extracts a feature vector from the input candidate defect instance cluster. The feature values ​​in the feature vector are composed of the following types of elements:

[0101] 1) Oriented bounding box feature value:

[0102] The oriented bounding box is the minimum volume cuboid that encloses the point cluster. The length, width, and height of the bounding box are defined as X in the feature vector 1 , X 2 , X 3 .

[0103] 2) Convex hull eigenvalues:

[0104] The convex hull is the smallest convex polyhedron that contains the point cluster. Its surface area and volume are defined as the X in the eigenvector 4 and X 5

[0105] 3) Shape feature value:

[0106] Normal consistency: By calculating the average normal angle of all points in the candidate cluster, it is defined as X 6 , where θ i is the normal angle of the i-th point, and N is the number of points in the cluster:

[0107]

[0108] Sphericity: A measure of how close a point cluster is to a sphere, defined as X 7 , where r min and r max These are the radii of the minimum and maximum bounding spheres of the point cluster:

[0109]

[0110] Planarity: It measures the degree to which a point cluster is close to a plane in three-dimensional space. It can be calculated by the eigenvalue of the covariance matrix of the point cluster, defined as X 8 :

[0111]

[0112] Among them, (λ 1 ,λ 2 ,λ 3 ) is the eigenvalue of the point cluster covariance matrix and satisfies λ 1 ≥λ 2 ≥λ 3 ,λ 1 represents the expansion of the point cluster along the main direction, λ 2 and λ 3 Respectively represent the expansion in the secondary direction.

[0113] All the eigenvalues ​​extracted above are summarized into the eigenvector X = [X1 ,X 2 ,…,X n ], where n = 8 is the total number of features. The shared candidate defect instance cluster feature extractor runs multiple instances with the same parameters in parallel and receives the data output from the multi-head spatial clustering layer to improve the operation speed.

[0114] S4.2: Shared Normalizer

[0115] After obtaining the feature vectors of the candidate defect instance cluster, standardization is performed. The purpose of standardization is to eliminate the dimensional differences between different features so that their weights in the classifier are more balanced. Standardization can be achieved through the following formula:

[0116]

[0117] Among them, X′ is the standardized feature vector, μ and σ are the mean and standard deviation of each feature in the feature vector, respectively.

[0118] The shared normalizer runs multiple instances with the same parameters in parallel and receives data output from the shared candidate defect instance cluster extractor to increase the operation speed.

[0119] S4.3: Shared classifiers

[0120] The normalized feature vector X′ is input to the classifier layer for binary classification of candidate defect instance clusters. The classifier is trained using an artificial neural network (ANN) or an XGBoost classifier. The specific classification process is as follows:

[0121] 1) Model training:

[0122] Artificial Neural Network (ANN): The standardized feature vector is input into the Multilayer Perceptron (MLP). The MLP performs nonlinear mapping on the input data through multiple hidden layers and activation functions, and the output is the probability value of the candidate defect instance cluster being a surface defect.

[0123] XGBoost: Using the Gradient Boosting Tree (GBT) framework, XGBoost classifies the standardized feature vectors and outputs the probability that the candidate defect instance cluster is a surface defect.

[0124] 2) Model Reasoning:

[0125] In the inference stage, the trained classification model is used to classify the newly input candidate defect instance cluster into two categories and output the probability that it is a surface defect. Let the output of the classifier be P(D), where D represents the probability that the cluster is a surface defect, which is defined as:

[0126] P(D)=Classifier(X′)

[0127] According to the probability value output by the classifier, it can be further determined whether the candidate defect instance cluster is a real surface defect. The shared classifier runs multiple instances with the same parameters in parallel and receives the data output from the shared normalizer to improve the operation speed.

[0128] Through the above steps, the shared classification layer can accurately classify candidate defect instance clusters, providing a reliable basis for subsequent defect analysis and processing.

[0129] S5: Non-maximum suppression layer

[0130] The overlapping candidate defect instance clusters are processed, and the overlapping and redundant defect instance clusters are removed to retain the instance cluster that is most likely to be a surface defect as the final detected three-dimensional surface defect instance cluster.

[0131] S5.1: Shared filters

[0132] The shared filter performs preliminary screening on the classified candidate defect instance clusters, removes instance clusters with low classification probability, and retains only candidate defect instance clusters with high classification probability. According to the classification probability P(D) obtained in the shared classification layer, a probability threshold P is set. th . Only keep the classification probability P(D)≥P th The candidate defect instance clusters are selected, and other instance clusters with low classification probability will be removed. The filtered candidate defect instance cluster set is recorded as S filtered :

[0133] S filtered ={C i ∣P(C i )≥P th}

[0134] Among them, C i is a candidate defect instance cluster, P(C i ) is C i The classification probability of .

[0135] S5.2: Non-maximum suppressor

[0136] The main task of the non-maximum suppressor is to eliminate redundancy and overlap in candidate defect instance clusters and retain only the most representative surface defect instance clusters. The non-maximum suppressor determines the degree of overlap of instance clusters by calculating the intersection over union (IoU) between candidate defect instance clusters. The specific operation is as follows:

[0137] 1) Intersection over Union (IoU) calculation:

[0138] For two candidate defect instance clusters C i and C j, the intersection over union (IoU) is defined as:

[0139]

[0140] Among them, C i ∩C j Represents instance cluster C i and C j The intersection (i.e. the overlapping part), C i ∪C j represents the union of the two.

[0141] 2) Overlapping candidate defect instance cluster processing:

[0142] For each pair of candidate defect instance clusters C i and C j , if its intersection over union (IoU) i ,C j ) is greater than the specified threshold T IoU , then these two instance clusters are considered overlapping clusters.

[0143] 3) Non-maximum suppression:

[0144] For each pair of overlapping clusters C i and C j , compare their classification probabilities P(D i ) and P(D j ). Keep the instance clusters with higher classification probability and delete the instance clusters with lower classification probability. Repeat the non-maximum suppression operation until no instance cluster has an intersection-over-union ratio greater than T. IoU .

[0145] Retain C i ,if P(D i )>P(D j ) and IoU(C i ,C j )>T IoU

[0146] In this way, the non-maximum suppressor can effectively eliminate the redundancy in candidate defect instance clusters and only retain the instance clusters that are most likely to be real surface defects.

[0147] After being processed by the non-maximum suppressor, the output instance clusters are the final detected three-dimensional surface defects. These instance clusters no longer overlap and represent the surface defect areas with the highest classification probability, which are the final detection results of the method of the present invention.

[0148] The non-maximum suppression layer effectively optimizes the screening and retention process of candidate defect instance clusters by combining shared filters and non-maximum suppressors, thereby improving the accuracy and reliability of three-dimensional surface defect detection.

[0149] In order to further verify the beneficial effects of the present invention, the present invention is compared with several other typical 3D semantic segmentation networks that can be applied to 3D surface defect detection, including PointNet, PointNet++, KPConv and PointTransformer. The above network uses a self-collected composite material surface defect dataset to carry out training and verification on the same server. Table 1 shows the best results of each model trained with different parameters. Compared with other networks, the present invention has the best detection accuracy, and its accuracy and intersection-over-union ratio reach 0.9997 and 0.9402 respectively. Compared with the suboptimal network (i.e. PointNet), the intersection-over-union ratio of the present invention has increased by about 30%, and the effect is as shown in the attached figure. Figure 3 shown.

[0150] Table 1 Comparison of the accuracy of the present invention and other three-dimensional surface defect detection methods

[0151]

[0152] The present invention identifies and detects surface defects such as cracks, dents, scratches, etc. by collecting and analyzing three-dimensional point cloud data of the workpiece surface. It can be applied to surface quality inspection and evaluation in industries including but not limited to aerospace, automobile manufacturing, shipbuilding, electronics manufacturing and precision machinery manufacturing.

[0153] The above-described embodiments are only preferred specific implementations of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. An industrial surface defect detection method based on three-dimensional point cloud data, characterized in that: include: Preprocess the collected 3D point cloud data to remove outliers; Perform multi-head foreground extraction on the preprocessed 3D point cloud data to obtain a set of candidate defect points; Perform multi-head spatial clustering on the candidate defect point set to generate multiple candidate defect instance clusters; Classify each candidate defect instance cluster, obtain the defect probability of the selected defect instance cluster, and screen out the instance clusters that may be real defects; The overlap and redundancy of the screened instance clusters that may be real defects are removed, and finally the detected three-dimensional surface defect instance clusters are output.

2. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 1 is characterized in that: The preprocessed 3D point cloud data is subjected to multi-head foreground extraction. The specific method is as follows: Calculate the local geometric eigenvalues ​​of each point in the 3D point cloud; Establish multiple parallel foreground extractors, set corresponding percentage segmentation thresholds for each foreground extractor, and perform the following processing in each foreground extractor: compare the local geometric feature value of each point in the three-dimensional point cloud with the percentage segmentation threshold of the foreground extractor. If the local geometric feature value is greater than the threshold, mark the point as a candidate defect point; The candidate defect points marked in each foreground extractor are aggregated to form a candidate defect point set.

3. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 2 is characterized in that: Calculate the local geometric eigenvalue of each point in the three-dimensional point cloud. The local geometric eigenvalue is the rate of change of the normal vector. Calculate the point p in the three-dimensional point cloud i The rate of change of the normal vector is: Among them, ||·|| represents the Euclidean norm of the vector, n i For point p i Normal vector of point p i The normal vector of a point in the neighborhood is {n i1 ,n i2 ,…,n iK }, K is the number of neighborhood points.

4. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 2 is characterized in that: Calculate the local geometric eigenvalue of each point in the three-dimensional point cloud. The local geometric eigenvalue is the full variance feature. Calculate the point p in the three-dimensional point cloud i The total variance characteristics of: F i =GVF=tr(Σ)=λ1+λ2+λ3 Among them, λ1,λ2,λ3 are the eigenvalues ​​of the variance matrix Σ, and the variance matrix is ​​calculated by i The neighborhood points are obtained by principal component analysis.

5. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 1, characterized in that: Design multiple parallel spatial clusterers with different parameter values, and the candidate defect point set is processed by multiple spatial clusterers; The method for performing spatial clustering processing within each spatial clusterer is: For each point in the candidate defect point set, calculate the number of neighboring points within the radius eps of the point. If it exceeds the set value, mark the point as a core point. Starting from any core point, the clusterer expands all points including the core point and its neighborhood to form clusters until the cluster expansion can no longer include new points in the candidate defect point set; points with less than minpts points in any cluster are marked as noise points and do not participate in the formation of any cluster; eps and minpts are hyperparameters, and different values ​​are set in different spatial clusterers.

6. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 1, characterized in that: Classify each candidate defect instance cluster and calculate the defect probability of each category. The specific method is as follows: Extracting feature vectors from each candidate defect instance cluster, wherein the feature values ​​in the feature vectors include oriented bounding box feature values, convex hull feature values, and shape feature values; Normalize the eigenvalues ​​of the eigenvector; A mature classification model is used to classify the standardized feature vectors and predict the probability value of the surface defect category.

7. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 6 is characterized in that: Establish a classification model through a neural network algorithm or XGBoost classifier, taking the standardized feature vector as input and the probability value of surface defects as output; The standardized feature vector values ​​obtained after data preprocessing, multi-head foreground extraction, multi-head spatial clustering, feature vector extraction and standardization of the 3D point cloud historical data and the surface defect values ​​corresponding to the 3D point cloud historical data are used as training data to train the classification model; The trained classification model is used as a mature model for prediction.

8. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 6, characterized in that: The feature vector is expressed as X = [X1, X2, ..., X8], where X1, X2, X3 are the length, width, and height of the oriented bounding box, respectively; X4, X5 are the surface area and volume of the convex hull, respectively; and X6, X7, X8 are the normal consistency, sphericity, and flatness of the shape feature, respectively.

9. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 1, characterized in that: Remove overlap and redundancy from the selected instance clusters. The specific method is as follows: 1) Pair the instance clusters and calculate the intersection-and-union ratio of each pair of instance clusters: Among them, C i ∩C j Represents instance cluster C i and C j The intersection of i ∪C j Represents instance cluster C i and C j The union of; 2) For each pair of instance clusters, if the intersection over union (IoU) i ,C j ) is greater than the specified threshold, the two instance clusters are considered overlapping clusters; 3) For each pair of overlapping clusters, compare the defect probabilities of the two instance clusters, retain the instance cluster with a larger defect probability, and delete the instance cluster with a smaller defect probability; 4) Collect the remaining instance clusters and repeat steps 1) to 3) until the intersection-union ratio of any two instance clusters after pairing does not exceed the specified threshold.

10. The industrial surface defect detection method based on three-dimensional point cloud data according to claim 1, characterized in that: The method for preprocessing the collected 3D point cloud data and removing outliers is as follows: Extract the neighborhood point set for each point in the 3D point cloud data, calculate the Euclidean distance between each point and all points in the corresponding neighborhood point set, and find the average; The average distance corresponding to each point in the 3D point cloud data is counted, and the outliers are determined and removed from the 3D point cloud data based on the average value and standard deviation of the average distance.

Images