Point cloud anomaly detection method and device considering prototype score correction

Abstract

The invention belongs to the related technical field of three-dimensional detection, and discloses a point cloud anomaly detection method and device considering prototype score correction, and the method comprises the steps: (1) carrying out the simulation based on normal point cloud data, and obtaining synthetic anomaly data; (2) training a lightweight point cloud feature extractor based on the synthesized abnormal data; (3) extracting the features of the normal point cloud data and synthesizing the features of the abnormal data based on the lightweight point cloud feature extractor to respectively form a normal feature set and an abnormal feature set, and respectively recording the normal feature set and the abnormal feature set as a normal prototype set and an abnormal prototype set; and (4) calculating an abnormal score of the traditional prototype by using the minimum distance of the test data relative to the normal prototype, and correcting the abnormal score of the traditional prototype by using the minimum distance of the test data relative to the abnormal prototype so as to obtain a corrected abnormal score and realize point cloud anomaly detection. According to the invention, the accuracy of abnormal recognition is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to three-dimensional detection, and more specifically, relates to a point cloud anomaly detection method and device considering prototype score correction. Background Art

[0002] In today's industrial production, ensuring product quality is of utmost importance. Against this backdrop, the application of industrial vision inspection technology has become particularly crucial. Industrial vision inspection technology inspects products through an automated vision system to identify defective, flawed, or non-compliant products, thereby ensuring that product quality meets standards. In recent years, anomaly detection technology has been widely applied to the surface defect quality inspection of industrial products. In industrial vision inspection, through the analysis of product surface images, anomaly detection can identify various defects such as scratches, dents, impurities, etc., thus excluding unqualified products at an early stage of production. However, although image anomaly detection technology performs well in many aspects, it has limitations when dealing with certain complex scenarios. For example, for objects with indistinct surface textures, it is difficult for images to accurately capture their surface details, and image anomaly detection will fail. In contrast, point cloud data can provide three-dimensional geometric information of objects, making the detection more comprehensive and accurate.

[0003] Current point cloud anomaly detection methods mainly rely on prototype-based unsupervised detection methods, where a feature extractor maps test points into a distinguishable feature space and calculates an anomaly score based on the minimum distance (referred to as the normal prototype distance) between the test point and its nearest normal prototype. However, relying solely on the normal prototype distance is insufficient. Test points with similar normal prototype distances but different distances to the nearest abnormal prototype should ideally obtain different anomaly scores. Summary of the Invention

[0004] In view of the above deficiencies or improvement requirements of the prior art, the present invention provides a point cloud anomaly detection method and device considering prototype score correction, aiming to solve the problem of low accuracy in current point cloud anomaly recognition.

[0005] To achieve the above object, according to one aspect of the present invention, a point cloud anomaly detection method considering prototype anomaly score correction is provided. The method includes the following steps:

[0006] (1) Simulate synthetic anomaly data based on normal point cloud data;

[0007] (2) Train a lightweight point cloud feature extractor based on the synthetic anomaly data;

[0008] (3) Extract the features of the normal point cloud data and synthesize the features of the abnormal data based on the lightweight point cloud feature extractor to respectively form a set of normal features and a set of abnormal features, which are respectively denoted as the normal prototype set and the abnormal prototype set;

[0009] (4) Calculate the traditional prototype anomaly score using the minimum distance of the test data relative to the normal prototype, and then correct the traditional prototype anomaly score using the minimum distance of the test data relative to the abnormal prototype to obtain the calibrated anomaly score, realizing point cloud anomaly detection.

[0010] Furthermore, there are three ways to generate synthetic abnormal data, namely local stretching, local rotation, and local randomization.

[0011] Furthermore, the abnormal generation process includes three steps: reference point selection, neighborhood point search, and local point transformation. Let P ∈ R n×3 be a normal point cloud containing n points, randomly select a reference point p r ∈ R 3 , and then use the nearest neighbor search algorithm to find the k points closest to p r to form a neighborhood Next, transform Q to generate the synthetic abnormal Q′. Let q i represent the i-th neighborhood point of the reference point p r , and q i ′ ∈ Q′ represent the transformed point.

[0012] Furthermore, local stretching: Stretch the points in the neighborhood Q along the normal vector direction n r of the reference point p r ∈ R 3 , the total stretching length is D, and dir represents the attribute of the formed anomaly. When dir = 1, local stretching forms a bulge, and when dir = -1, local stretching forms a depression;

[0013]

[0014] Furthermore, local rotation: Use a rotation matrix R ∈ R 3×3 to transform the points in the neighborhood Q, select the angle range for the three rotation angles along the three coordinate values, and the rotation point around which the rotation transformation is performed is the reference point p r ,

[0015] q i ′ = R · (q i - p r ) + p r .

[0016] Further, local randomization: randomly replace the points within the neighborhood Q with points from a Gaussian distribution.

[0017] Further, the lightweight feature extractor includes two point transformer layers, and a segmentation head composed of a multi-layer perceptron is set after the lightweight feature extractor; the combined model is trained end-to-end using the IoU loss and the Focal loss, and the loss function is:

[0018]

[0019] where Φ represents the feature extractor and h represents the segmentation head.

[0020] Further, the calculation formula for calibrating the anomaly score is:

[0021]

[0022] where P(f t |A) represents the probability that the test data f t is an anomaly, ||d A || 2 represents the distance of the test data from the nearest anomaly prototype. The calibrated prototype score consists of two parts: the difference term (1 - λ)||d N || 2 - λ||d A || 2 , where the balance factor λ is used for weighted averaging of the two uncertainties; represents the ratio of the uncertainties, i.e., the relative uncertainty.

[0023] The present invention also provides a point cloud anomaly detection system considering prototype score correction. The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it executes the point cloud anomaly detection method considering prototype score correction as described above.

[0024] The present invention also provides a computer-readable storage medium. The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions cause the processor to implement the point cloud anomaly detection method considering prototype score correction as described above.

[0025] Generally speaking, compared with the prior art by the above technical solutions conceived by the present invention, the point cloud anomaly detection method and device considering prototype score correction provided by the present invention mainly have the following beneficial effects:

[0026] 1. Extract the features of normal point cloud data and synthesize the features of abnormal data based on the lightweight point cloud feature extractor to form a set of normal features and a set of abnormal features, denoted as the normal prototype set and the abnormal prototype set respectively. Calculate the traditional prototype anomaly score using the minimum distance of the test data relative to the normal prototype, and then use the minimum distance of the test data relative to the abnormal prototype to correct the traditional prototype anomaly score. At the same time, consider the absolute and relative distances between the test data and the normal and abnormal prototypes, improving the more accurate anomaly scoring.

[0027] 2. Simulate synthetic abnormal data based on normal point cloud data, reducing the cost of collecting abnormal points.

[0028] 3. There are three ways to generate synthetic abnormal data, namely local stretching, local rotation, and local randomization. Introduce three abnormal generation strategies to generate diverse and realistic synthetic abnormal data to support unsupervised anomaly detection methods. These synthetic abnormalities are further used for the training of the lightweight point cloud feature extractor, which is specifically used to extract local geometric information for anomaly detection.

[0029] 4. Through the training of synthetic abnormal data, encourage the feature extractor to aggregate local geometric information, enabling the segmentation head to effectively identify normal and abnormal points. Description of the Drawings

[0030] Figure 1 is a flowchart of a point cloud anomaly detection method considering prototype score correction provided by the present invention;

[0031] Figure 2 is a flowchart of abnormal data synthesis;

[0032] Figure 3 is a visualization diagram of the synthetic abnormal data;

[0033] Figure 4 is a comparison diagram before and after prototype distance correction, where (1) is for the case of only considering the normal prototype distance, and (2) is for the case of considering the normal and abnormal prototype distances. Detailed Embodiments

[0034] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] Please refer to Figure 1, the present invention provides a point cloud anomaly detection method considering prototype anomaly score correction. The detection method proposes traditional prototype anomaly score correction, improves the traditional prototype anomaly score by combining normal and abnormal prototype distances, and considers both the absolute and relative distances between test data and normal and abnormal prototypes to provide a more accurate anomaly score. In addition, since the cost of collecting real anomaly points is too high, three anomaly generation strategies are introduced to generate diverse and realistic synthetic anomaly data to support unsupervised anomaly detection methods. These synthetic anomalies are further used for training a lightweight point cloud feature extractor, which is specifically designed to extract local geometric information for anomaly detection.

[0036] In one embodiment, first, an anomaly generation strategy is proposed to generate diverse and realistic synthetic anomaly data; then, a lightweight point cloud feature extractor is trained using the synthetic anomalies; next, the features of normal training data and synthetic anomaly data are extracted by the point cloud feature extractor, and normal and abnormal prototype sets are established; finally, prototype anomaly score correction is proposed, considering both the absolute and relative distances between test data and normal and abnormal prototypes, to provide a more accurate anomaly score.

[0037] The detection method mainly includes the following steps:

[0038] Step 1, simulate synthetic anomaly data based on normal point cloud data.

[0039] Point cloud anomalies usually manifest as local changes. Therefore, in this embodiment, three anomaly generation methods are designed to introduce point cloud anomalies through local geometric perturbations: local stretching, local rotation, and local randomization. As Figure 2 shown, the anomaly generation process includes three steps: reference point selection, neighborhood point search, and local point transformation. Let P ∈ R n×3 be a normal point cloud containing n points. First, a reference point p r ∈ R 3 is randomly selected from P. Then, the k nearest points to p r are found using the k-nearest neighbor search algorithm (KNN) to form a neighborhood Next, Q is transformed to generate synthetic anomaly Q'. Let q i represent the i-th neighborhood point of the reference point p r , and q i ' ∈ Q' represent the transformed point. The transformation is defined as follows:

[0040] Local stretching: Stretch the points in the neighborhood Q along the normal vector direction n r of the reference point p r ∈ R 3Perform stretching with a total stretching length of D. dir represents the attribute of the formed anomaly. When dir = 1, local stretching forms a bulge, and when dir = -1, local stretching forms a depression.

[0041]

[0042] Local rotation: Use a rotation matrix R ∈ R 3×3 Transform the points within the neighborhood Q, and select a relatively small range for the three rotation angles along the three coordinate values, such as 0 - 30°. The rotation point around which the rotation transformation occurs is the reference point p r .

[0043] q i ′ = R · (q i - p r ) + p r (2)

[0044] Local randomization: Randomly replace the points within the neighborhood Q with points following a Gaussian distribution. Gauss is the generating function of the Gaussian distribution, with p r as the mean, and the standard deviation σ is generally selected as 1 / 3 of the maximum point spacing among the points in the neighborhood Q

[0045] q i ′ = Gauss(p r , σ 2 ) (3)

[0046] The resulting anomalous data P ′ = (P \ Q) ∪ Q ′ As Figure 3 shown.

[0047] Step 2: Train a lightweight point cloud feature extractor based on the synthesized anomalous data.

[0048] The above synthesis of anomalies introduces regions with unique shapes and point distributions compared to normal point clouds, and point-level anomaly labels y can be obtained. If the i-th point in the point cloud is an anomalous point, y j = 1, otherwise y j = 0. By flexibly selecting the reference point, this method can generate various realistic anomalies, and these anomalies can approximate the realistic anomaly distribution. Existing point cloud extractors rely on publicly available real-life data for pre-training, but these models are not optimized for anomaly detection and often lack efficiency. Therefore, in this embodiment, various realistic synthesized anomalies and the corresponding point-level anomaly labels of these generated anomalies are used to train a lightweight point cloud feature extractor specifically tailored for anomaly detection from scratch.

[0049] Local geometric features are particularly valuable in detecting anomalies in point clouds. Therefore, this embodiment introduces a lightweight feature extractor that is simpler and shallower than traditional backbones. The lightweight feature extractor contains only two point transformer layers and can focus on capturing core geometric information. A segmentation head composed of a multi-layer perceptron (MLP) is added after the feature extractor, which directly predicts the anomaly probability. The combined model is trained end-to-end using the IoU loss and the Focal loss. The loss function is as follows:

[0050]

[0051] where φ represents the feature extractor and h represents the segmentation head. Through training on synthetic anomaly data, the feature extractor is encouraged to aggregate local geometric information, enabling the segmentation head to effectively identify normal and abnormal points.

[0052] Step 3: Based on the lightweight point cloud feature extractor, extract the features of the normal point cloud data and the synthetic anomaly data to form a set of normal features and a set of anomaly features, denoted as the normal prototype set and the anomaly prototype set, respectively.

[0053] After obtaining the feature extractor, it is necessary to extract the features f i of the normal training data P i and the features f' i of the synthetic anomaly data P' i , where i represents the sequence of normal or abnormal point clouds. The normal feature f i is as follows:

[0054] f i = φ(P i ) (5)

[0055] The abnormal feature f' i is extracted from the synthetic anomaly data using the feature extractor:

[0056] f' i = φ(P' i ) (6)

[0057] f ′ i = ∪{f ′ ij}y j = 1

[0058] Then, take the union of all the extracted normal features to establish a set of normal features, denoted as the normal prototype set S N :

[0059] S N = ∪{f i}, i = 0, 1, 2, … (7)

[0060] Take the union of all the extracted abnormal features and establish a set of abnormal features, denoted as the abnormal prototype set S A :

[0061] S A = ∪{f′ i}, i = 0, 1, 2, … (8)

[0062] Step 4: Calculate the traditional prototype anomaly score using the minimum distance of the test data relative to the normal prototype, and then use the minimum distance of the test data relative to the abnormal prototype to correct the traditional prototype anomaly score to obtain the calibrated anomaly score, realizing point cloud anomaly detection.

[0063] First, calculate the anomaly score using the minimum distance between the test data and the normal prototype, which can be reinterpreted from the perspectives of probability and information theory. Specifically, the normal prototype set can be regarded as a Gaussian isotropic mixture distribution P(f t |N):

[0064]

[0065] where f t is the feature of the test data, J is the total number of normal prototypes, represents the Gaussian distribution where the i-th normal prototype is located, with μ i as the mean and Σ i as the covariance. ω i is the weight of the i-th prototype distribution's contribution to the anomaly score. Approximately only consider the distribution probability of the nearest prototype in the set:

[0066]

[0067] Define f s* as the normal prototype closest to the test data. When the independent Gaussian distribution assumption is satisfied, its mean μ s* is f s* , and its covariance can be defined as Then can be further written as:

[0068]

[0069] The anomaly score A(f t ) of the test data can be calculated using the uncertainty represented by self-information:

[0070]

[0071]

[0072] Among them and associated with is consistent with, ||d N || 2 represents the distance between the test data feature and the nearest normal prototype. Therefore, the anomaly scores of different test data only vary with ||d N || 2 Changing, so let A N be 0 and B N be 1, then there is:

[0073] A(f t ) = ||d N || 2 (13)

[0074] However, this measure of uncertainty about normality does not fully capture the likelihood of anomalies. As Figure 4 shown, p a and p B have the same minimum normal prototype distance, but because compared with p a , the anomaly prototype distance of p B is smaller, that is, p B is closer to the anomaly point. Therefore, p B should theoretically have a higher anomaly score than p a . Therefore, the traditional prototype anomaly score is calibrated by introducing the anomaly prototype distance. Intuitively, when the test point is closer to the anomaly prototype, the higher the likelihood that it is an anomaly. This results in a reduction in uncertainty in the anomaly distribution, and vice versa. Conversely, the higher the uncertainty in the normal distribution, the higher the anomaly score. Therefore, the formula for calibrating the anomaly score is:

[0075]

[0076] where P(f t |A) represents the probability that the test data f t is an anomaly, ||d A || 2 represents the distance of the test data from the nearest anomaly prototype. The calibrated prototype score consists of two parts: the difference term (1 - λ)||d N || 2 - λ||d A || 2 , where the balance factor λ is used for weighted averaging of the two uncertainties; represents the ratio of uncertainties, that is, the relative uncertainty.

[0077] The present invention also provides a point cloud anomaly detection system considering prototype score correction. The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it executes the point cloud anomaly detection method considering prototype score correction as described above.

[0078] The present invention also provides a computer-readable storage medium. The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the point cloud anomaly detection method considering prototype score correction as described above.

[0079] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A point cloud anomaly detection method considering prototype anomaly score correction, characterized in that: The method comprises the following steps: (1) Synthetic abnormal data is obtained by simulation based on normal point cloud data; (2) Training a lightweight point cloud feature extractor based on synthetic anomaly data; (3) extracting features of normal point cloud data and synthesizing features of abnormal data based on the lightweight point cloud feature extractor to form a set of normal features and a set of abnormal features, which are respectively recorded as a normal prototype set and an abnormal prototype set; (4) The minimum distance between the test data and the normal prototype is used to calculate the anomaly score of the traditional prototype, and then the minimum distance between the test data and the abnormal prototype is used to correct the anomaly score of the traditional prototype to obtain the calibrated anomaly score and realize point cloud anomaly detection.

2. The point cloud anomaly detection method considering prototype anomaly score correction according to claim 1, characterized in that: There are three ways to generate synthetic abnormal data, namely local stretching, local rotation and local randomization.

3. The point cloud anomaly detection method considering prototype anomaly score correction as claimed in claim 2, characterized in that: The anomaly generation process includes three steps: reference point selection, neighborhood point search and local point transformation. Let P∈R n×3 For a normal point cloud containing n points, randomly select a reference point p from P r ∈R 3 Then, the nearest neighbor search algorithm is used to find the distance p r The nearest k points form a neighborhood Next, transform Q to generate a synthetic anomaly Q′, and let q i Denote the reference point p r The i-th neighboring point of i ′∈Q′ represents the transformed point.

4. The point cloud anomaly detection method considering prototype anomaly score correction as claimed in claim 3, characterized in that: Local stretching: Move the points in the neighborhood Q along the reference point p r The normal direction n r ∈R 3 Stretch, the total stretching length is D, dir represents the properties of the formed anomaly, when dir = 1, local stretching forms a bulge, when dir = -1, local stretching forms a depression; 5. The point cloud anomaly detection method considering prototype anomaly score correction as claimed in claim 3, characterized in that: Local rotation: Use a rotation matrix R∈R 3×3 Transform the points in the neighborhood Q, select the angle range along the three rotation angles of the three coordinate values, and the rotation point around which the rotation transformation is performed is the reference point p r , q i ′ =R·(q i -p r )+p r 。 6. The point cloud anomaly detection method considering prototype anomaly score correction as claimed in claim 3, characterized in that: Local randomization: Randomly replace points in the neighborhood Q with Gaussian distributed points.

7. The point cloud anomaly detection method considering prototype anomaly score correction according to any one of claims 1 to 6, characterized in that: The lightweight feature extractor consists of two point transformer layers, followed by a segmentation head consisting of a multi-layer perceptron. The combined model is trained end-to-end using IoU loss and Focal loss, with a loss function of for: Where φ represents the feature extractor and h represents the segmentation head.

8. The point cloud anomaly detection method considering prototype anomaly score correction according to any one of claims 1 to 6, characterized in that: The calculation formula for the calibration anomaly score is: Where P(f t |A) represents the test data f t is the probability of abnormality, ||d A || 2 Represents the distance between the test data and the nearest abnormal prototype. The calibration prototype score consists of two parts: the difference term (1-λ)||d N || 2 -λ||d A || 2 , where the balancing factor λ is used to weighted average the two uncertainties; Represents the ratio of uncertainty, that is, relative uncertainty.

9. A point cloud anomaly detection system considering prototype score correction, characterized in that: The system includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the point cloud anomaly detection method considering prototype score correction according to any one of claims 1 to 8 is executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the point cloud anomaly detection method considering prototype score correction as described in any one of claims 1-8.

Images