Three-dimensional point cloud anomaly detection method and system

Abstract

The application relates to a three-dimensional point cloud anomaly detection method and system, wherein the three-dimensional point cloud anomaly detection method comprises the following steps: extracting curvature enhancement features and three-dimensional coordinate features of a point cloud to be detected; performing nearest neighbor matching on the curvature enhancement features and a curvature enhancement feature library to obtain a first anomaly score; performing nearest neighbor matching on the three-dimensional coordinate features and a three-dimensional coordinate feature library to obtain a second anomaly score; generating a boundary mask; suppressing the first anomaly score of each point in the boundary mask to obtain a third anomaly score; fusing the third anomaly score, the first anomaly score and the second anomaly score to obtain a fourth anomaly score; and determining that the three-dimensional point cloud is abnormal when the maximum value of the fourth anomaly score is greater than a sample level threshold. The method can detect anomalies by using the curvature enhancement features of the point cloud to be detected, improves the sensitivity to subtle geometric anomalies, and thus improves the accuracy of local subtle geometric anomaly detection.

Description

Technical Field

[0001] This application relates to the field of 3D vision inspection technology, and in particular to 3D point cloud anomaly detection methods and systems. Background Technology

[0002] In applications such as precision manufacturing, workpiece appearance inspection, and 3D scanning quality inspection, product defects often manifest as localized protrusions, depressions, cuts, missing parts, edge damage, or minute structural disturbances. These defects primarily alter the local geometry of an object's surface without necessarily accompanied by significant color or texture changes. Therefore, methods relying solely on 2D images, color textures, or single-view depth maps are prone to missing localized geometric defects or failing to accurately locate defective areas. 3D point clouds can directly represent the spatial structure of an object's surface, making them suitable for full-surface inspection of high-precision workpieces.

[0003] Existing 3D point cloud anomaly detection methods are mainly divided into feature matching, reconstruction, and pseudo-anomaly learning methods. Feature matching methods typically construct a normal feature library and determine whether the test point cloud deviates from the normal distribution using nearest neighbor distance. Reconstruction methods train a reconstruction model using normal samples and then locate anomalies based on the difference between the input and the reconstruction result. Pseudo-anomaly learning methods improve the model's sensitivity to anomalies by simulating or predicting anomalous perturbations. All three types of methods suffer from low accuracy in detecting subtle local geometric anomalies.

[0004] There is currently no effective solution to the problem of low detection accuracy for local minor geometric anomalies in related technologies. Summary of the Invention

[0005] This embodiment provides a three-dimensional point cloud anomaly detection method and system to solve the problem of low detection accuracy for local minor geometric anomalies in related technologies.

[0006] Firstly, this embodiment provides a three-dimensional point cloud anomaly detection method, including:

[0007] A 3D point cloud of the object to be detected is obtained, and the 3D point cloud is registered to obtain the point cloud to be detected.

[0008] The curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected are extracted; the curvature enhancement features characterize the local curvature of the point cloud to be detected.

[0009] The curvature enhancement features are matched with the pre-constructed curvature enhancement feature library to obtain the first anomaly score of each point in the point cloud to be detected.

[0010] The three-dimensional coordinate features are matched with the nearest neighbor of the pre-constructed three-dimensional coordinate feature library to obtain the second anomaly score of each point in the point cloud to be detected.

[0011] Local boundary detection is performed on each point in the point cloud to be detected to obtain the boundary points in the point cloud to be detected; the neighborhood of the boundary points is expanded to generate a boundary mask;

[0012] The first anomaly score of each point within the boundary mask is suppressed to obtain the third anomaly score of each point within the boundary mask;

[0013] The third anomaly score of each point within the boundary mask, the first anomaly score of each point outside the boundary mask, and the second anomaly score of each point in the point cloud to be detected are fused to obtain the fourth anomaly score of each point in the point cloud to be detected.

[0014] The maximum value of the fourth anomaly score of each point in the point cloud to be detected is extracted to obtain the sample-level anomaly score of the point cloud to be detected; when the sample-level anomaly score is greater than the preset sample-level threshold, the three-dimensional point cloud is determined to be abnormal.

[0015] In some embodiments, the extraction of curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected includes:

[0016] The farthest point is sampled from the point cloud to be detected to obtain multiple local point sets of the point cloud to be detected;

[0017] Extract the curvature enhancement features of each of the local point sets;

[0018] Extract the three-dimensional coordinate features of each point in the point cloud to be detected.

[0019] In some embodiments, the step of sampling the farthest point of the point cloud to be detected to obtain multiple local point sets of the point cloud to be detected includes:

[0020] The farthest point is sampled from the point cloud to be detected to obtain multiple point block centers;

[0021] Perform a K-nearest neighbor search on each of the point block centers to obtain the local point set corresponding to each of the point block centers.

[0022] In some embodiments, extracting the curvature enhancement features of each of the local point sets includes:

[0023] Calculate the curvature of the local point set; obtain the weighted curvature based on the curvature and a preset curvature weight;

[0024] The depth features of the local point set are extracted using a point cloud mask autoencoder.

[0025] The depth feature and the weighted curvature are concatenated to obtain the curvature enhancement feature of the local point set.

[0026] In some embodiments, the step of performing nearest neighbor matching between the curvature enhancement features and a pre-built curvature enhancement feature library to obtain a first anomaly score for each point in the point cloud to be detected includes:

[0027] The curvature enhancement features are matched with the nearest neighbor of the pre-constructed curvature enhancement feature library to obtain the point block level anomaly score of the local point set;

[0028] Based on the point-block level anomaly scores of the local point set, the first anomaly score of each point in the point cloud to be detected is obtained.

[0029] In some embodiments, before performing nearest neighbor matching between the curvature enhancement features and a pre-built curvature enhancement feature library to obtain the point-block level anomaly score of the local point set, the method further includes:

[0030] Construct the curvature enhancement feature library.

[0031] In some embodiments, constructing the curvature enhancement feature library includes:

[0032] Extract the curvature enhancement features of each normal point cloud in the normal point cloud set;

[0033] The normal point cloud is segmented to obtain multiple local point clouds;

[0034] Extract the curvature enhancement features from the local point cloud;

[0035] The curvature enhancement features of each normal point cloud and the curvature enhancement features of each local point cloud are merged to obtain a curvature enhancement feature candidate library.

[0036] Samples are collected from the candidate library of curvature enhancement features to obtain the curvature enhancement feature library.

[0037] In some embodiments, before performing nearest neighbor matching between the three-dimensional coordinate features and a pre-built three-dimensional coordinate feature library to obtain a second anomaly score for each point in the point cloud to be detected, the method further includes:

[0038] Construct the three-dimensional coordinate feature library.

[0039] In some embodiments, constructing the three-dimensional coordinate feature library includes:

[0040] Extract the three-dimensional coordinate features of each normal point cloud in the normal point cloud set;

[0041] The normal point cloud is segmented to obtain multiple local point clouds;

[0042] Extract the three-dimensional coordinate features of the local point cloud;

[0043] The three-dimensional coordinate features of each normal point cloud and the three-dimensional coordinate features of each local point cloud are merged to obtain a candidate library of three-dimensional coordinate features.

[0044] Samples are collected from the candidate library of three-dimensional coordinate features to obtain the three-dimensional coordinate feature library.

[0045] Secondly, this embodiment provides a 3D point cloud anomaly detection system, including a preprocessing module, a feature extraction module, an anomaly score calculation module, an anomaly score suppression module, an anomaly score fusion module, and an anomaly determination module, wherein:

[0046] The preprocessing module is used to acquire the three-dimensional point cloud of the object to be detected and to perform registration processing on the three-dimensional point cloud to obtain the point cloud to be detected.

[0047] The feature extraction module is used to extract the curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected; the curvature enhancement features characterize the local curvature of the point cloud to be detected.

[0048] The anomaly score calculation module is used to perform nearest neighbor matching between the curvature enhancement features and a pre-built curvature enhancement feature library to obtain the first anomaly score of each point in the point cloud to be detected; and to perform nearest neighbor matching between the three-dimensional coordinate features and a pre-built three-dimensional coordinate feature library to obtain the second anomaly score of each point in the point cloud to be detected.

[0049] The anomaly score suppression module is used to perform local boundary detection on each point in the point cloud to be detected, to obtain the boundary points in the point cloud to be detected; to perform neighborhood expansion on the boundary points to generate a boundary mask; and to suppress the first anomaly score of each point within the boundary mask to obtain the third anomaly score of each point within the boundary mask.

[0050] The anomaly score fusion module is used to fuse the third anomaly score of each point within the boundary mask, the first anomaly score of each point outside the boundary mask, and the second anomaly score of each point in the point cloud to be detected, to obtain the fourth anomaly score of each point in the point cloud to be detected.

[0051] The anomaly determination module is used to extract the maximum value of the fourth anomaly score of each point in the point cloud to be detected, and obtain the sample-level anomaly score of the point cloud to be detected; when the sample-level anomaly score is greater than the preset sample-level threshold, the three-dimensional point cloud is determined to be abnormal.

[0052] Compared with related technologies, the 3D point cloud anomaly detection method provided in this embodiment obtains the 3D point cloud of the object to be detected and performs registration processing on the 3D point cloud to obtain the point cloud to be detected; extracts the curvature enhancement features and 3D coordinate features of the point cloud to be detected; the curvature enhancement features characterize the local curvature of the point cloud to be detected; performs nearest neighbor matching between the curvature enhancement features and a pre-built curvature enhancement feature library to obtain the first anomaly score of each point in the point cloud to be detected; performs nearest neighbor matching between the 3D coordinate features and a pre-built 3D coordinate feature library to obtain the second anomaly score of each point in the point cloud to be detected; and performs local boundary mapping on each point in the point cloud to be detected. The detection process involves identifying boundary points in the point cloud to be detected; expanding the neighborhood of these boundary points to generate a boundary mask; suppressing the first anomaly score of each point within the boundary mask to obtain a third anomaly score; fusing the third anomaly scores of each point within the boundary mask, the first anomaly scores of each point outside the boundary mask, and the second anomaly scores of each point in the point cloud to obtain a fourth anomaly score; extracting the maximum value of the fourth anomaly score of each point in the point cloud to obtain a sample-level anomaly score; and determining an anomaly in the 3D point cloud when the sample-level anomaly score exceeds a preset sample-level threshold. This method utilizes the curvature enhancement features of the point cloud to perform anomaly detection, improving sensitivity to subtle geometric anomalies and thus enhancing the accuracy of detecting local subtle geometric anomalies.

[0053] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0054] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0055] Figure 1 This is a hardware structure block diagram of the terminal of the three-dimensional point cloud anomaly detection method in this embodiment;

[0056] Figure 2 This is a flowchart of the three-dimensional point cloud anomaly detection method in this embodiment;

[0057] Figure 3 This is a structural block diagram of the three-dimensional point cloud anomaly detection system in this embodiment. Detailed Implementation

[0058] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0059] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0060] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the 3D point cloud anomaly detection method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0061] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the 3D point cloud anomaly detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0062] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0063] This embodiment provides a method for detecting anomalies in three-dimensional point clouds. Figure 2 This is a flowchart of the three-dimensional point cloud anomaly detection method in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0064] Step S201: Obtain the 3D point cloud of the object to be detected, and perform registration processing on the 3D point cloud to obtain the point cloud to be detected.

[0065] Specifically, a 3D point cloud of the object to be inspected is acquired using acquisition devices such as 3D scanners, structured light equipment, or laser scanners. This 3D point cloud includes at least the 3D spatial coordinate information of each point, and may also include normal vectors, color, or other additional attributes. The 3D point cloud is then preprocessed, including centering correction, scale normalization, downsampling, or denoising. The preprocessed point cloud is then registered to the normal template coordinate system of its respective category to obtain the point cloud to be inspected.

[0066] Step S202: Extract the curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected.

[0067] Specifically, the curvature enhancement feature characterizes the local curvature of the point cloud to be detected. The three-dimensional coordinate feature represents the three-dimensional spatial coordinates of each point in the point cloud. The farthest point is sampled from the point cloud to be detected, resulting in multiple point patch centers. A K-nearest neighbor search is performed on each point patch center to obtain the local point set corresponding to each point patch center. The mean of the local point set is calculated, and this mean is expressed as:

[0068] ;

[0069] in Let represent the local point set corresponding to the center of the i-th point block, K represent the total number of points in the local point set corresponding to the center of the i-th point block, and p represent the three-dimensional spatial coordinates of the points in the local point set corresponding to the center of the i-th point block. Calculate the covariance matrix of the local point set, which is expressed as:

[0070] ;

[0071] Eigenvalue decomposition of the covariance matrix yields... , , Calculate the curvature of the local point set, which is expressed as:

[0072] ;

[0073] in This represents a minimum constant. The minimum eigenvalue is the smallest eigenvalue when the local point set approximates a plane. Smaller, curvature Approaching 0; when a local point set exhibits bulges, depressions, cuts, or irregular disturbances, the minimum eigenvalue is... Increase, curvature This also increases accordingly. The curvature of each local point set is standardized using the following formula:

[0074] ;

[0075] in and Let represent the mean and standard deviation of the curvature of a local point set in the point cloud to be detected, respectively. The curvature enhancement feature of the local point set is represented as:

[0076] ;

[0077] in This represents the depth feature of the local point set corresponding to the center of the i-th point block, which is extracted by the point cloud mask autoencoder. This indicates the preset curvature weight.

[0078] Step S203: Perform nearest neighbor matching between the curvature enhancement features and the pre-built curvature enhancement feature library to obtain the first anomaly score of each point in the point cloud to be detected; perform nearest neighbor matching between the three-dimensional coordinate features and the pre-built three-dimensional coordinate feature library to obtain the second anomaly score of each point in the point cloud to be detected.

[0079] Specifically, a curvature enhancement feature library is constructed. Curvature enhancement features are extracted from each normal point cloud in the normal point cloud set according to step S202. The normal point cloud is then segmented to obtain multiple local point clouds. For example, the q-th segmentation plane yields two local point clouds, represented as follows:

[0080] ;

[0081] ;

[0082] in Indicates the anchor point. The normal vector is represented. Curvature enhancement features of each local point cloud are extracted according to step S202. The curvature enhancement features of each normal point cloud and each local point cloud are merged to obtain a curvature enhancement feature candidate library. Samples are collected from the curvature enhancement feature candidate library to obtain the curvature enhancement feature library. The curvature enhancement features of the point cloud to be detected are matched with the pre-constructed curvature enhancement feature library using nearest neighbor matching, and the nearest neighbor distance is used as the block-level anomaly score of the point cloud to be detected. The block-level anomaly score of the point cloud to be detected is assigned to the first anomaly score of each point in the corresponding local point set, thereby obtaining the first anomaly score of each point in the point cloud to be detected.

[0083] Construct a 3D coordinate feature library. Extract the 3D coordinate features of each normal point cloud from the normal point cloud set according to step S202. Segment the normal point clouds to obtain multiple local point clouds. For example, the q-th segmentation plane can yield two local point clouds, represented as follows:

[0084] ;

[0085] ;

[0086] in Indicates the anchor point. The normal vector is represented. Following step S202, the 3D coordinate features of each local point cloud are extracted. The 3D coordinate features of each normal point cloud and each local point cloud are merged to obtain a 3D coordinate feature candidate library. Samples are collected from the 3D coordinate feature candidate library to obtain the 3D coordinate feature library. The 3D coordinate features of the point cloud to be detected are matched with the pre-constructed 3D coordinate feature library using nearest neighbor matching, and the nearest neighbor distance is used as the second anomaly score for each point in the point cloud to be detected.

[0087] Step S204: Perform local boundary detection on each point in the point cloud to be detected to obtain the boundary points in the point cloud to be detected; expand the neighborhood of the boundary points to generate a boundary mask; suppress the first anomaly score of each point in the boundary mask to obtain the third anomaly score of each point in the boundary mask.

[0088] Specifically, a local tangent plane is constructed within the neighborhood of a point in the point cloud to be detected. The neighborhood points are then projected onto two principal directions, and a polar angle sequence is calculated. If the maximum gap between adjacent polar angles is... Greater than the preset angle threshold If the condition is met, the point is determined to be a boundary point. A neighborhood expansion is performed on the boundary point to generate a boundary mask. The first anomaly score of each point within the boundary mask is suppressed to obtain the third anomaly score of each point within the boundary mask, expressed as:

[0089] ;

[0090] in Indicates the intensity of inhibition. Indicates boundary mask, Represents the first outlier score of a point, when When the first anomaly score of a point decreases, that is, the first anomaly score of each point within the boundary mask is suppressed, resulting in the third anomaly score of each point within the boundary mask; when At that time, the first outlier score of the point remains unchanged.

[0091] Step S205: Fuse the third anomaly score of each point within the boundary mask, the first anomaly score of each point outside the boundary mask, and the second anomaly score of each point in the point cloud to be detected to obtain the fourth anomaly score of each point in the point cloud to be detected.

[0092] Specifically, the fourth anomaly score of each point in the point cloud to be detected is represented as follows:

[0093] ;

[0094] in This represents the third anomaly score for each point within the boundary mask and the first anomaly score for each point outside the boundary mask. Let N represent the second anomaly score of each point in the point cloud to be detected, and let N represent the normalization function, expressed as:

[0095] ;

[0096] Step S206: Extract the maximum value of the fourth anomaly score of each point in the point cloud to be detected to obtain the sample-level anomaly score of the point cloud to be detected; when the sample-level anomaly score is greater than the preset sample-level threshold, the three-dimensional point cloud is determined to be abnormal.

[0097] Specifically, the sample-level anomaly score of the point cloud to be detected is represented as follows:

[0098] ;

[0099] in This represents the fourth anomaly score for each point in the point cloud to be detected. When this sample-level anomaly score is greater than a preset sample-level threshold, the 3D point cloud is determined to be abnormal. An anomaly heatmap can be generated based on the fourth anomaly score of each point in the point cloud to be detected, which is used to locate abnormal regions.

[0100] In this embodiment, a three-dimensional point cloud of the object to be detected is acquired and registered to obtain the point cloud to be detected. Curvature enhancement features and three-dimensional coordinate features are extracted from the point cloud. The curvature enhancement features characterize the local curvature of the point cloud. The curvature enhancement features are matched with a pre-built curvature enhancement feature library to obtain the first anomaly score for each point in the point cloud. The three-dimensional coordinate features are matched with a pre-built three-dimensional coordinate feature library to obtain the second anomaly score for each point in the point cloud. Local boundary detection is performed on each point in the point cloud to obtain the second anomaly score for each point in the point cloud. The method involves identifying boundary points; expanding the neighborhood of these boundary points to generate a boundary mask; suppressing the first anomaly score of each point within the boundary mask to obtain a third anomaly score; fusing the third anomaly score of each point within the boundary mask, the first anomaly score of each point outside the boundary mask, and the second anomaly score of each point in the point cloud to be detected to obtain a fourth anomaly score of each point in the point cloud to be detected; extracting the maximum value of the fourth anomaly score of each point in the point cloud to obtain a sample-level anomaly score of the point cloud to be detected; and determining an anomaly in the 3D point cloud when the sample-level anomaly score is greater than a preset sample-level threshold. This method utilizes the curvature enhancement features of the point cloud to be detected for anomaly detection, improving sensitivity to subtle geometric anomalies and thus increasing the accuracy of detecting local subtle geometric anomalies.

[0101] In some embodiments, the curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected are extracted, including:

[0102] The farthest point of the point cloud to be detected is sampled to obtain multiple local point sets; the curvature enhancement features of each local point set are extracted; and the three-dimensional coordinate features of each point in the point cloud to be detected are extracted.

[0103] Specifically, the curvature enhancement feature characterizes the local curvature of the point cloud to be detected. The three-dimensional coordinate feature represents the three-dimensional spatial coordinates of each point in the point cloud. The farthest point is sampled from the point cloud to be detected, resulting in multiple point patch centers. A K-nearest neighbor search is performed on each point patch center to obtain the local point set corresponding to each point patch center. The mean of the local point set is calculated, and this mean is expressed as:

[0104] ;

[0105] in Let represent the local point set corresponding to the center of the i-th point block, K represent the total number of points in the local point set corresponding to the center of the i-th point block, and p represent the three-dimensional spatial coordinates of the points in the local point set corresponding to the center of the i-th point block. Calculate the covariance matrix of the local point set, which is expressed as:

[0106] ;

[0107] Eigenvalue decomposition of the covariance matrix yields... , , Calculate the curvature of the local point set, which is expressed as:

[0108] ;

[0109] in This represents a minimum constant. The minimum eigenvalue is the smallest eigenvalue when the local point set approximates a plane. Smaller, curvature Approaching 0; when a local point set exhibits bulges, depressions, cuts, or irregular disturbances, the minimum eigenvalue is... Increase, curvature This also increases accordingly. The curvature of each local point set is standardized using the following formula:

[0110] ;

[0111] in and Let represent the mean and standard deviation of the curvature of a local point set in the point cloud to be detected, respectively. The curvature enhancement feature of the local point set is represented as:

[0112] ;

[0113] in This represents the depth feature of the local point set corresponding to the center of the i-th point block, which is extracted by the point cloud mask autoencoder. This indicates the preset curvature weight.

[0114] In some embodiments, the farthest point of the point cloud to be detected is sampled to obtain multiple local point sets of the point cloud to be detected, including:

[0115] The farthest point of the point cloud to be detected is sampled to obtain multiple point block centers; K-nearest neighbor search is performed on each point block center to obtain the local point set corresponding to each point block center.

[0116] Specifically, the farthest point sampling involves selecting uniformly distributed sampling centers from the point cloud. By performing farthest point sampling on the point cloud to be detected, multiple uniformly distributed point patch centers can be obtained. A K-nearest neighbor search is then performed on each point patch center to obtain the local point set corresponding to each point patch center.

[0117] In some of these embodiments, curvature enhancement features are extracted from each local point set, including:

[0118] Calculate the curvature of the local point set; obtain the weighted curvature based on the curvature and preset weights; extract the depth features of the local point set using a point cloud mask autoencoder; concatenate the depth features and the weighted curvature to obtain the curvature enhancement features of the local point set.

[0119] Specifically, the mean of the local point set is calculated, and this mean is expressed as:

[0120] ;

[0121] in Let represent the local point set corresponding to the center of the i-th point block, K represent the total number of points in the local point set corresponding to the center of the i-th point block, and p represent the three-dimensional spatial coordinates of the points in the local point set corresponding to the center of the i-th point block. Calculate the covariance matrix of the local point set, which is expressed as:

[0122] ;

[0123] Eigenvalue decomposition of the covariance matrix yields... , , Calculate the curvature of the local point set, which is expressed as:

[0124] ;

[0125] in This represents a minimum constant. The minimum eigenvalue is the smallest eigenvalue when the local point set approximates a plane. Smaller, curvature Approaching 0; when a local point set exhibits bulges, depressions, cuts, or irregular disturbances, the minimum eigenvalue is... Increase, curvature This also increases accordingly. The curvature of each local point set is standardized using the following formula:

[0126] ;

[0127] in and Let represent the mean and standard deviation of the curvature of a local point set in the point cloud to be detected, respectively. The curvature enhancement feature of the local point set is represented as:

[0128] ;

[0129] in This represents the depth feature of the local point set corresponding to the center of the i-th point block, which is extracted by the point cloud mask autoencoder. This indicates the preset curvature weight.

[0130] In some embodiments, the curvature enhancement features are matched with a pre-built curvature enhancement feature library to obtain the first anomaly score for each point in the point cloud to be detected, including:

[0131] The curvature enhancement features are matched with the nearest neighbor of the pre-built curvature enhancement feature library to obtain the point block level anomaly score of the local point set; based on the point block level anomaly score of the local point set, the first anomaly score of each point in the point cloud to be detected is obtained.

[0132] Specifically, the curvature enhancement features of the point cloud to be detected are matched with the pre-constructed curvature enhancement feature library using nearest neighbor matching, and the nearest neighbor distance is used as the block-level anomaly score of the point cloud to be detected. The block-level anomaly score of the point cloud to be detected is assigned to the first anomaly score of each point in the corresponding local point set, thereby obtaining the first anomaly score of each point in the point cloud to be detected.

[0133] In some embodiments, before performing nearest neighbor matching between the curvature enhancement features and a pre-built curvature enhancement feature library to obtain the point-block level anomaly scores of the local point set, the method further includes:

[0134] Construct a curvature enhancement feature library.

[0135] Specifically, a curvature enhancement feature library is constructed. Curvature enhancement features are extracted from each normal point cloud in the normal point cloud set according to step S202. The normal point cloud is then segmented to obtain multiple local point clouds. For example, the q-th segmentation plane yields two local point clouds, represented as follows:

[0136] ;

[0137] ;

[0138] in Indicates the anchor point. The normal vector is represented. Curvature enhancement features of each local point cloud are extracted according to step S202. The curvature enhancement features of each normal point cloud and each local point cloud are merged to obtain a curvature enhancement feature candidate library. Samples are collected from the curvature enhancement feature candidate library to obtain the curvature enhancement feature library.

[0139] In some of these embodiments, a curvature enhancement feature library is constructed, including:

[0140] Curvature enhancement features are extracted from each normal point cloud in the normal point cloud set; the normal point cloud is segmented to obtain multiple local point clouds; curvature enhancement features are extracted from the local point clouds; the curvature enhancement features of each normal point cloud and each local point cloud are merged to obtain a curvature enhancement feature candidate library; samples are collected from the curvature enhancement feature candidate library to obtain the curvature enhancement feature library.

[0141] Specifically, a curvature enhancement feature library is constructed. Curvature enhancement features are extracted from each normal point cloud in the normal point cloud set according to step S202. The normal point cloud is then segmented to obtain multiple local point clouds. For example, the q-th segmentation plane yields two local point clouds, represented as follows:

[0142] ;

[0143] ;

[0144] in Indicates the anchor point. The normal vector is represented. Curvature enhancement features of each local point cloud are extracted according to step S202. The curvature enhancement features of each normal point cloud and each local point cloud are merged to obtain a curvature enhancement feature candidate library. Samples are collected from the curvature enhancement feature candidate library to obtain the curvature enhancement feature library.

[0145] In some embodiments, before performing nearest neighbor matching between the 3D coordinate features and a pre-built 3D coordinate feature library to obtain a second anomaly score for each point in the point cloud to be detected, the method further includes:

[0146] Construct a 3D coordinate feature library.

[0147] Specifically, a 3D coordinate feature library is constructed. Following step S202, the 3D coordinate features of each normal point cloud in the normal point cloud set are extracted. The normal point clouds are then segmented to obtain multiple local point clouds. For example, the q-th segmentation plane can yield two local point clouds, represented as follows:

[0148] ;

[0149] ;

[0150] in Indicates the anchor point. The normal vector is represented. Following step S202, the 3D coordinate features of each local point cloud are extracted. The 3D coordinate features of each normal point cloud and each local point cloud are merged to obtain a 3D coordinate feature candidate library. Samples are then collected from the 3D coordinate feature candidate library to obtain the 3D coordinate feature database.

[0151] In some of these embodiments, a three-dimensional coordinate feature library is constructed, including:

[0152] Extract the 3D coordinate features of each normal point cloud in the normal point cloud set; segment the normal point cloud to obtain multiple local point clouds; extract the 3D coordinate features of the local point clouds; merge the 3D coordinate features of each normal point cloud and each local point cloud to obtain a 3D coordinate feature candidate library; sample the 3D coordinate feature candidate library to obtain a 3D coordinate feature library.

[0153] Specifically, a 3D coordinate feature library is constructed. Following step S202, the 3D coordinate features of each normal point cloud in the normal point cloud set are extracted. The normal point clouds are then segmented to obtain multiple local point clouds. For example, the q-th segmentation plane can yield two local point clouds, represented as follows:

[0154] ;

[0155] ;

[0156] in Indicates the anchor point. The normal vector is represented. Following step S202, the 3D coordinate features of each local point cloud are extracted. The 3D coordinate features of each normal point cloud and each local point cloud are merged to obtain a 3D coordinate feature candidate library. Samples are then collected from the 3D coordinate feature candidate library to obtain the 3D coordinate feature database.

[0157] This embodiment also provides a 3D point cloud anomaly detection system, which is used to implement the above embodiments; details already described will not be repeated. The terms "module," "unit," and "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0158] Figure 3 This is a structural block diagram of the 3D point cloud anomaly detection system of this embodiment, as shown below. Figure 3 As shown, the 3D point cloud anomaly detection system 30 includes: a preprocessing module 301, a feature extraction module 302, an anomaly score calculation module 303, an anomaly score suppression module 304, an anomaly score fusion module 305, and an anomaly determination module 306, wherein:

[0159] The preprocessing module 301 is used to acquire the three-dimensional point cloud of the object to be detected and to perform registration processing on the three-dimensional point cloud to obtain the point cloud to be detected.

[0160] The feature extraction module 302 is used to extract the curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected; the curvature enhancement features characterize the local curvature of the point cloud to be detected.

[0161] The anomaly score calculation module 303 is used to perform nearest neighbor matching between the curvature enhancement features and the pre-built curvature enhancement feature library to obtain the first anomaly score of each point in the point cloud to be detected; and to perform nearest neighbor matching between the three-dimensional coordinate features and the pre-built three-dimensional coordinate feature library to obtain the second anomaly score of each point in the point cloud to be detected.

[0162] The anomaly score suppression module 304 is used to perform local boundary detection on each point in the point cloud to be detected, to obtain the boundary points in the point cloud to be detected; to perform neighborhood expansion on the boundary points to generate a boundary mask; and to suppress the first anomaly score of each point within the boundary mask to obtain the third anomaly score of each point within the boundary mask.

[0163] Anomaly score fusion module 305 is used to fuse the third anomaly score of each point within the boundary mask, the first anomaly score of each point outside the boundary mask, and the second anomaly score of each point in the point cloud to be detected, to obtain the fourth anomaly score of each point in the point cloud to be detected.

[0164] The anomaly detection module 306 is used to extract the maximum value of the fourth anomaly score of each point in the point cloud to be detected, and obtain the sample-level anomaly score of the point cloud to be detected; when the sample-level anomaly score is greater than the preset sample-level threshold, the three-dimensional point cloud is determined to be abnormal.

[0165] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0166] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0168] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0169] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0170] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for anomaly detection in three-dimensional point clouds, characterized in that, include: A 3D point cloud of the object to be detected is obtained, and the 3D point cloud is registered to obtain the point cloud to be detected. Extract the curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected; The curvature enhancement feature characterizes the degree of local curvature of the point cloud to be detected; The curvature enhancement features are matched with the pre-constructed curvature enhancement feature library to obtain the first anomaly score of each point in the point cloud to be detected. The three-dimensional coordinate features are matched with the nearest neighbor of the pre-constructed three-dimensional coordinate feature library to obtain the second anomaly score of each point in the point cloud to be detected. Local boundary detection is performed on each point in the point cloud to be detected to obtain the boundary points in the point cloud to be detected. The boundary points are expanded to their neighborhoods to generate a boundary mask; The first anomaly score of each point within the boundary mask is suppressed to obtain the third anomaly score of each point within the boundary mask; The third anomaly score of each point within the boundary mask, the first anomaly score of each point outside the boundary mask, and the second anomaly score of each point in the point cloud to be detected are fused to obtain the fourth anomaly score of each point in the point cloud to be detected. Extract the maximum value of the fourth anomaly score of each point in the point cloud to be detected to obtain the sample-level anomaly score of the point cloud to be detected. When the sample-level anomaly score is greater than a preset sample-level threshold, the 3D point cloud is determined to be abnormal.

2. The three-dimensional point cloud anomaly detection method according to claim 1, characterized in that, The extraction of curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected includes: The farthest point is sampled from the point cloud to be detected to obtain multiple local point sets of the point cloud to be detected; Extract the curvature enhancement features of each of the local point sets; Extract the three-dimensional coordinate features of each point in the point cloud to be detected.

3. The three-dimensional point cloud anomaly detection method according to claim 2, characterized in that, The step of sampling the farthest point of the point cloud to be detected to obtain multiple local point sets of the point cloud to be detected includes: The farthest point is sampled from the point cloud to be detected to obtain multiple point block centers; Perform a K-nearest neighbor search on each of the point block centers to obtain the local point set corresponding to each of the point block centers.

4. The three-dimensional point cloud anomaly detection method according to claim 2, characterized in that, The extraction of curvature enhancement features from each of the local point sets includes: Calculate the curvature of the local point set; obtain the weighted curvature based on the curvature and a preset curvature weight; The depth features of the local point set are extracted using a point cloud mask autoencoder. The depth feature and the weighted curvature are concatenated to obtain the curvature enhancement feature of the local point set.

5. The three-dimensional point cloud anomaly detection method according to claim 2, characterized in that, The step of performing nearest neighbor matching between the curvature enhancement features and a pre-constructed curvature enhancement feature library to obtain the first anomaly score for each point in the point cloud to be detected includes: The curvature enhancement features are matched with the nearest neighbor of the pre-constructed curvature enhancement feature library to obtain the point block level anomaly score of the local point set; Based on the point-block level anomaly scores of the local point set, the first anomaly score of each point in the point cloud to be detected is obtained.

6. The three-dimensional point cloud anomaly detection method according to claim 5, characterized in that, Before performing nearest neighbor matching between the curvature enhancement features and a pre-constructed curvature enhancement feature library to obtain the point-block level anomaly score of the local point set, the method further includes: Construct the curvature enhancement feature library.

7. The three-dimensional point cloud anomaly detection method according to claim 6, characterized in that, The construction of the curvature enhancement feature library includes: Extract the curvature enhancement features of each normal point cloud in the normal point cloud set; The normal point cloud is segmented to obtain multiple local point clouds; Extract the curvature enhancement features from the local point cloud; The curvature enhancement features of each normal point cloud and the curvature enhancement features of each local point cloud are merged to obtain a curvature enhancement feature candidate library. Samples are collected from the candidate library of curvature enhancement features to obtain the curvature enhancement feature library.

8. The three-dimensional point cloud anomaly detection method according to claim 1, characterized in that, Before performing nearest neighbor matching between the three-dimensional coordinate features and a pre-constructed three-dimensional coordinate feature library to obtain the second anomaly score for each point in the point cloud to be detected, the method further includes: Construct the three-dimensional coordinate feature library.

9. The three-dimensional point cloud anomaly detection method according to claim 8, characterized in that, The construction of the three-dimensional coordinate feature library includes: Extract the three-dimensional coordinate features of each normal point cloud in the normal point cloud set; The normal point cloud is segmented to obtain multiple local point clouds; Extract the three-dimensional coordinate features of the local point cloud; The three-dimensional coordinate features of each normal point cloud and the three-dimensional coordinate features of each local point cloud are merged to obtain a candidate library of three-dimensional coordinate features. Samples are collected from the candidate library of three-dimensional coordinate features to obtain the three-dimensional coordinate feature library.

10. A three-dimensional point cloud anomaly detection system, characterized in that, It includes a preprocessing module, a feature extraction module, an anomaly score calculation module, an anomaly score suppression module, an anomaly score fusion module, and an anomaly detection module, wherein: The preprocessing module is used to acquire the three-dimensional point cloud of the object to be detected and to perform registration processing on the three-dimensional point cloud to obtain the point cloud to be detected. The feature extraction module is used to extract the curvature enhancement features and three-dimensional coordinate features of the point cloud to be detected; the curvature enhancement features characterize the local curvature of the point cloud to be detected. The anomaly score calculation module is used to perform nearest neighbor matching between the curvature enhancement features and a pre-built curvature enhancement feature library to obtain the first anomaly score of each point in the point cloud to be detected; and to perform nearest neighbor matching between the three-dimensional coordinate features and a pre-built three-dimensional coordinate feature library to obtain the second anomaly score of each point in the point cloud to be detected. The anomaly score suppression module is used to perform local boundary detection on each point in the point cloud to be detected, to obtain the boundary points in the point cloud to be detected; to perform neighborhood expansion on the boundary points to generate a boundary mask; and to suppress the first anomaly score of each point within the boundary mask to obtain the third anomaly score of each point within the boundary mask. The anomaly score fusion module is used to fuse the third anomaly score of each point within the boundary mask, the first anomaly score of each point outside the boundary mask, and the second anomaly score of each point in the point cloud to be detected, to obtain the fourth anomaly score of each point in the point cloud to be detected. The anomaly determination module is used to extract the maximum value of the fourth anomaly score of each point in the point cloud to be detected, and obtain the sample-level anomaly score of the point cloud to be detected; when the sample-level anomaly score is greater than the preset sample-level threshold, the three-dimensional point cloud is determined to be abnormal.

Images