A multi-category point cloud anomaly detection method and system considering multi-level features

Abstract

The present invention belongs to the technical field related to three-dimensional detection, and discloses a multi-category point cloud anomaly detection method and system considering multi-level features, the method comprising: extracting multi-dimensional features of training point cloud data to obtain local features and global features; clustering global features to obtain multiple cluster centers to form a category memory library; obtaining the cluster center closest to the global features of each training point cloud data in the category memory library as the target cluster center, extracting the core set of the set as a sub-memory library; using the global features of the point cloud data to be tested to obtain the corresponding target sub-memory library, and then using the local features of the point cloud data to be tested to find the closest target features in the target sub-memory library; calculating the distance between the local features of the point cloud data to be tested and the target features to obtain the anomaly score, and judging the anomaly of the point cloud data to be tested based on the anomaly score. The present application solves the technical problem of feature confusion caused by the similarity of multiple categories of data.

Description

Technical Field

[0001] The present invention belongs to the technical field related to three-dimensional detection, and more specifically, relates to a multi-category point cloud anomaly detection method and system considering multi-level features. Background Art

[0002] In today's industrial production, it is crucial to ensure product quality. In this context, the application of industrial visual inspection technology has become particularly important. Industrial visual inspection technology uses an automated visual system to inspect products to identify defects, flaws or products that do not meet specifications, thereby ensuring that product quality meets standards. In recent years, anomaly detection technology has been widely used in surface defect quality inspection of industrial products. In industrial visual inspection, by analyzing the surface image of the product, anomaly detection can identify a variety of defects such as scratches, dents, impurities, etc., so that defective products can be eliminated in the early stages of production. However, although image anomaly detection technology performs well in many aspects, it has limitations when dealing with certain complex scenes. For example, for objects with unclear surface textures, it is difficult for images to accurately capture their surface details, and image anomaly detection will fail. In contrast, point cloud data can provide three-dimensional geometric information of objects, making detection more comprehensive and accurate.

[0003] Current point cloud anomaly detection methods use an unsupervised approach, using only normal data for training to detect anomalies, but require training a dedicated detection model for each type of data. However, as the number of classes increases, the time and storage space required to train and deploy dedicated models becomes enormous. Therefore, it is necessary to study methods that can be used for multi-class data anomaly detection.

[0004] Existing multi-class anomaly detection methods aim to obtain tight boundaries of features from multi-class training data, but due to the similarities between normal and abnormal features of different categories, it is often difficult to accurately obtain tight boundaries, resulting in feature confusion. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a multi-category point cloud anomaly detection method and system taking into account multi-level features, thereby solving the technical problem of feature confusion caused by the similarity of multiple categories of data.

[0006] To achieve the above object, according to one aspect of the present invention, a multi-category point cloud anomaly detection method considering multi-level features is provided, comprising: S1: extracting multi-dimensional features of training point cloud data using a point cloud feature extractor; S2: sequentially obtaining local features and global features based on the multi-dimensional features; S3: clustering the global features to obtain multiple cluster centers, and all cluster centers form a category memory library M cls ; S4: In the category memory M clsThe cluster center closest to the global feature of each training point cloud data is obtained as the target cluster center of the training point cloud data, all local features of the target cluster center are put into a set, and the core set of the set is extracted as a sub-memory library; S5: local features and global features of the point cloud data to be tested are obtained, and the corresponding target sub-memory library is obtained by using the global features of the point cloud data to be tested, and then the local features of the point cloud data to be tested are used to find the target feature F closest to the target sub-memory library. * ; S6: Calculate the local features of the point cloud data to be measured and the target features F * The abnormality score is obtained by calculating the distance between the points and the abnormality of the point cloud data to be tested.

[0007] Preferably, step S6 comprises: calculating the local features of the point cloud data to be measured and the target features F * The point-level anomaly score is obtained by calculating the distance between the two points; the maximum value of the point-level anomaly score is obtained as the target anomaly score; the target anomaly score is compared with the preset threshold to obtain the anomaly of the point cloud data to be tested.

[0008] Preferably, the point cloud feature extractor includes multiple transformer layers, and multi-dimensional features are obtained through multiple transformer layers.

[0009] Preferably, in step S2, the tensor of the multi-dimensional features is concatenated along the feature dimension to obtain local features.

[0010] Preferably, the local features are averaged along the feature dimension to obtain the global features.

[0011] Preferably, in step S3, clustering is performed using a K-means or DBSCAN clustering method.

[0012] Preferably, the calculation formula of the core set in step S4 is:

[0013]

[0014] Among them, M* is the core set, M c is any subset of M, M is a set, s is any element in M, t is M c Any element in .

[0015] Preferably, in step S5, the subscript idx' of the corresponding target sub-memory library is obtained by using the global features of the point cloud data to be measured, and the target sub-memory library is obtained according to the subscript, wherein:

[0016]

[0017] Among them, Fck is the kth cluster center, is the global feature of the point cloud data to be tested.

[0018] The second aspect of the present application provides a multi-category point cloud anomaly detection system considering multi-level features, including: a feature extraction module: used to extract multi-dimensional features of training point cloud data using a point cloud feature extractor; a local and global feature acquisition module: used to sequentially obtain local features and global features based on the multi-dimensional features; a distance module: used to cluster the global features to obtain multiple cluster centers, and all cluster centers form a category memory library M cls ; Sub-memory building block: used in the category memory M cls The cluster center closest to the global feature of each training point cloud data is obtained as the target cluster center of the training point cloud data, all local features of the target cluster center are put into a set, and the core set of the set is extracted as a sub-memory library; the target feature acquisition module is used to obtain the local features and global features of the point cloud data to be tested, obtain the corresponding target sub-memory library using the global features of the point cloud data to be tested, and then use the local features of the point cloud data to be tested to find the target feature F closest to the target sub-memory library * ; Abnormal judgment module: used to calculate the local features of the point cloud data to be tested and the target features F * The abnormality score is obtained by calculating the distance between the points and the abnormality of the point cloud data to be tested.

[0019] In general, compared with the prior art, the above technical solution conceived by the present invention provides a multi-category point cloud anomaly detection method and system considering multi-level features, which mainly has the following beneficial effects:

[0020] 1. This application first extracts the global features of the training point cloud data for clustering, and then builds a sub-memory library for each cluster center. The sub-memory library is built using the local features of the training point cloud data. According to the distance between the global features of the test data and the cluster center, the test data is transmitted to the corresponding sub-repository for anomaly detection. This application uses global features and local features for multi-level detection, avoiding the problem of feature confusion.

[0021] 2. This application has multi-dimensional feature recognition capabilities, so it can be used for multi-category data anomaly detection, greatly reducing computing and storage costs.

[0022] 3. This application takes the maximum value of the point-level anomaly score to obtain the target anomaly score, which can maximize anomaly detection and greatly improve detection sensitivity and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of the steps of a multi-category point cloud anomaly detection method considering multi-level features according to an embodiment of the present application;

[0024] Figure 2 It is a flowchart of a multi-category point cloud anomaly detection method considering multi-level features according to an embodiment of the present application;

[0025] Figure 3 This is a schematic diagram of the testing process of the multi-category point cloud anomaly detection method considering multi-level features in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0027] The first aspect of the present application provides a multi-category point cloud anomaly detection method considering multi-level features, such as Figure 1 and Figure 2 As shown, the method includes S1 to S6.

[0028] S1: Use the point cloud feature extractor to extract multi-dimensional features of the training point cloud data.

[0029] A point cloud feature extractor f of a model such as PointMAE or PointBert can be used to extract multidimensional features F of the training point cloud data 1 , F 2 , …, F m .

[0030] S2: Obtain local features and global features in sequence according to the multi-dimensional features.

[0031] f is preferably N k Transformer layers, set F i is the output of the i-th transformer layer of f, then the local feature F of the input training point cloud data l for:

[0032] F l =cat([F 1 , F 2 , …, F m ])

[0033] Among them, the cat() function concatenates the input tensor along the feature dimension. For example, the feature with a feature dimension of [256, 768, 50] and the feature with a feature dimension of [256, 768, 70] are concatenated along the last dimension into a new feature with a feature dimension of [256, 768, 120]. m is the number of layers of the selected features.

[0034] Global feature F g By performing global average pooling on the local features of the point cloud, we can obtain:

[0035] F g =AvePooling(F l )

[0036] Among them, the AvePooling() function takes the average of the input tensor along the feature dimension. For example, the feature with a feature dimension of [256, 768, 120] is averaged along the last dimension to obtain a new feature with a feature dimension of [256, 768, 1].

[0037] According to the above method, the global features F of all training point cloud data are extracted gi and local features F li , where i represents the i-th data in the training point cloud data, satisfies 1≤i≤N, and N is the total number of training point cloud data.

[0038] S3: Clustering the global features to obtain multiple cluster centers, all cluster centers form a category memory library M cls .

[0039] The clustering center F of all global features is calculated by clustering algorithm cj , 1≤j≤K, K is the total number of cluster centers.

[0040] F c1 , F c2 , …, F cK =Cluster(F g1 , F g2 , …, F gN )

[0041] Cluster can be any clustering method, such as K-means and DBSCAN. K-means is more suitable for the situation where the global features of the point cloud are distributed in clusters, while DBSCAN is more suitable for the situation where the global features of the point cloud are distributed continuously.

[0042] Store all cluster centers in a set to form a category memory library M cls .

[0043] S4: In the category memory Mcls The cluster center closest to the global feature of each training point cloud data is obtained as the target cluster center of the training point cloud data, all local features of the target cluster center are put into a set, and the core set of the set is extracted as a sub-memory library.

[0044] Category Memory M cls Each cluster center in represents a potential category. Next, a sub-memory library is established for each cluster center to store the feature distribution of the training point cloud data.

[0045] The sub-memory corresponding to the K cluster centers is denoted as M j , where 1≤j≤K. For any training point cloud data, for example, the i-th data in the training point cloud data, its global feature is F gi , whose local feature is F li First, in M cls Find the global feature F gi The closest cluster center is assumed to be the idx-th cluster center:

[0046]

[0047] Then the local feature F li Will be used to construct the sub-memory M idx .

[0048] The construction process of the sub-memory library is to first put all the local features of the cluster center into a set, and then extract the core set of the set to achieve it, where the core set M of any set M is * The calculation method is:

[0049]

[0050] Among them, M* is the core set, M c is any subset of M, M is a set, s is any element in M, t is M c Any element in .

[0051] Core sets can represent the distribution of original features and reduce unnecessary computational costs.

[0052] S5: Obtain the local features and global features of the point cloud data to be tested, use the global features of the point cloud data to be tested to obtain the corresponding target sub-memory library, and then use the local features of the point cloud data to be tested to find the nearest target feature F in the target sub-memory library * .

[0053] like Figure 3 As shown, during the test, for any input point cloud data P test, obtain the corresponding local features and global features according to the method in step S2.

[0054]

[0055] The subscript idx' of the corresponding target sub-memory library is obtained by using the global features of the point cloud data to be tested, and the target sub-memory library M is obtained according to the subscript idx′ ,in:

[0056]

[0057] Among them, F ck is the kth cluster center, is the global feature of the point cloud data to be tested.

[0058] Then use In M idx′ Find the closest target feature F in * :

[0059]

[0060] S6: Calculate the local features of the point cloud data to be measured and the target features F * The abnormality score is obtained by calculating the distance between the points and the abnormality of the point cloud data to be tested.

[0061] Step S6 specifically includes:

[0062] S61: Calculate the local features of the point cloud data to be measured and the target features F * The distance between them is used to obtain the point-level anomaly score A;

[0063]

[0064] S62: Obtain the maximum value of the point-level anomaly scores as the target anomaly score ξ.

[0065] ξ=max(A)

[0066] S63: Compare the target anomaly score with a preset threshold to obtain an abnormality of the point cloud data to be tested.

[0067] The second aspect of the present application provides a multi-category point cloud anomaly detection system considering multi-layer machine features, including:

[0068] Feature extraction module: the above step S1 may be executed to extract multi-dimensional features of training point cloud data using a point cloud feature extractor;

[0069] A local and global feature acquisition module: the above step S2 may be executed to sequentially obtain local features and global features according to the multi-dimensional features;

[0070] Distance module: The above step S3 can be executed to cluster the global features to obtain multiple cluster centers, and all cluster centers constitute a category memory library M cls ;

[0071] Sub-memory library construction module: The above step S4 can be executed to construct a sub-memory library in the category memory library M cls The cluster center closest to the global feature of each training point cloud data is obtained as the target cluster center of the training point cloud data, all local features of the target cluster center are put into a set, and the core set of the set is extracted as a sub-memory library;

[0072] Target feature acquisition module: The above step S5 can be executed to obtain the local features and global features of the point cloud data to be tested, use the global features of the point cloud data to be tested to obtain the corresponding target sub-memory library, and then use the local features of the point cloud data to be tested to find the nearest target feature F in the target sub-memory library. * ;

[0073] Abnormal judgment module: can execute the above step S6 to calculate the local features of the point cloud data to be tested and the target features F * The abnormality score is obtained by calculating the distance between the points and the abnormality of the point cloud data to be tested.

[0074] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-category point cloud anomaly detection method considering multi-level features, characterized in that: include: S1: Use point cloud feature extractor to extract multi-dimensional features of training point cloud data; S2: Obtaining local features and global features in sequence according to the multidimensional features; S3: Clustering the global features to obtain multiple cluster centers, all cluster centers form a category memory library M cls ; S4: In the category memory M cls The cluster center closest to the global feature of each training point cloud data is obtained as the target cluster center of the training point cloud data, all local features of the target cluster center are put into a set, and the core set of the set is extracted as a sub-memory library; S5: Obtain the local features and global features of the point cloud data to be tested, use the global features of the point cloud data to be tested to obtain the corresponding target sub-memory library, and then use the local features of the point cloud data to be tested to find the nearest target feature F in the target sub-memory library * ; S6: Calculate the local features of the point cloud data to be measured and the target features F * The distance between the two points is used to obtain an abnormality score, and the abnormality of the point cloud data to be tested is determined according to the abnormality score; The construction process of the sub-memory library first puts all the local features of the cluster center into a set, and then extracts the core set of the set to achieve it. The core set M of any set M is * The calculation method is: Among them, M* is the core set, M c is any subset of M, M is a set, s is any element in M, t is M c Any element in .

2. The multi-category point cloud anomaly detection method according to claim 1, characterized in that: Step S6 includes: Calculate the local features of the point cloud data to be measured and the target features F * The distance between them obtains the point-level anomaly score; Get the maximum value of the point-level anomaly score as the target anomaly score; The target anomaly score is compared with the preset threshold to obtain the anomaly of the point cloud data to be tested.

3. The multi-category point cloud anomaly detection method according to claim 1, characterized in that: The point cloud feature extractor includes multiple transformer layers, and multi-dimensional features are obtained through multiple transformer layers.

4. The multi-category point cloud anomaly detection method according to claim 1 or 3, characterized in that: In step S2, the multi-dimensional feature tensor is concatenated along the feature dimension to obtain local features.

5. The multi-category point cloud anomaly detection method according to claim 4, characterized in that: The local features are averaged along the feature dimension to obtain the global features.

6. The multi-category point cloud anomaly detection method according to claim 1, characterized in that: In step S3, clustering is performed using K-means or DBSCAN clustering method.

7. The multi-category point cloud anomaly detection method according to claim 1, characterized in that: In step S5, the subscript idx' of the corresponding target sub-memory library is obtained by using the global features of the point cloud data to be measured, and the target sub-memory library is obtained according to the subscript, wherein: Among them, F ck is the kth cluster center, is the global feature of the point cloud data to be tested.

8. A multi-category point cloud anomaly detection system considering multi-level features, characterized in that: include: Feature extraction module: used to extract multi-dimensional features of training point cloud data using a point cloud feature extractor; Local and global feature acquisition module: used to obtain local features and global features in sequence according to the multi-dimensional features; Distance module: used to cluster the global features to obtain multiple cluster centers, and all cluster centers constitute a category memory library M cls ; Sub-memory building blocks: used in the category memory M cls The cluster center closest to the global feature of each training point cloud data is obtained as the target cluster center of the training point cloud data, all local features of the target cluster center are put into a set, and the core set of the set is extracted as a sub-memory library; Target feature acquisition module: used to obtain the local features and global features of the point cloud data to be tested, use the global features of the point cloud data to be tested to obtain the corresponding target sub-memory library, and then use the local features of the point cloud data to be tested to find the nearest target feature F in the target sub-memory library * ; Abnormal judgment module: used to calculate the local features of the point cloud data to be tested and the target features F * The distance between the two points is used to obtain an abnormality score, and the abnormality of the point cloud data to be tested is determined according to the abnormality score; The construction process of the sub-memory library first puts all the local features of the cluster center into a set, and then extracts the core set of the set to achieve it. The core set M of any set M is * The calculation method is: Among them, M* is the core set, M c is any subset of M, M is a set, s is any element in M, t is M c Any element in .

Images