Method and system for improving point cloud classification accuracy based on atlas domain

Abstract

The present invention discloses a method and system for improving the accuracy of point cloud classification in the atlas domain, and relates to the field of point cloud classification. The method comprises the following steps: obtaining original point cloud data of a 3D object; constructing a KNN graph on the original point cloud to represent the geometric structure information of the original point cloud; using GFT, the KNN graph converts the original point cloud data from the data domain to the atlas domain; constructing an atlas filter to filter the spectral features of the data transformed to the atlas domain to generate a perturbed spectral signal; using GFT, the perturbed spectral signal is converted back to the data domain to obtain adversarial point cloud data; constructing samples based on the original point cloud data and adversarial samples based on the adversarial point cloud data as training data, which are input into a point cloud classification model for classification training; and using the trained point cloud classification model to classify the original point cloud data of the 3D object to be processed, and outputting the classification results. The present invention can improve the accuracy of the model's classification and recognition of point clouds.

Description

Technical Field

[0001] The present invention relates to the field of point cloud classification, and specifically to a method and system for improving the classification accuracy of a three-dimensional point cloud classification model by generating adversarial samples based on a graph domain. Background Art

[0002] By acquiring key point information of a 3D object with three-dimensional structural characteristics, 3D point cloud data is formed. This point cloud data is then recognized using a point cloud classification model to identify / classify the 3D object represented by the point cloud data. The performance of existing point cloud classification models is limited by manually annotated datasets. Due to the limited size of the dataset, point cloud classification models can often only fit the approximate data distribution and are unable to capture the actual data space. To this end, existing work uses the method of generating adversarial samples to learn samples that may interfere with the point cloud classification model. These samples are then fed into the model as difficult sample data to further assist model training, thereby improving model classification accuracy.

[0003] At present, the existing adversarial sample generation methods for 3D point cloud models can be mainly divided into two categories: (1) Methods based on adding / discarding points: Xiang et al. generated adversarial point clouds by adding a limited number of interference points / point clusters / objects composed of point clusters to the original point cloud, and confirmed its effectiveness on the PointNet point cloud classification model. At the same time, there are other works that use gradient-based generation methods to identify key points in point clouds for modification, addition, and deletion. The goal of these adversarial point cloud generation methods is to change the classification score of the network model for the overall point cloud label by adding or deleting individual key points of the identified 3D objects to achieve misclassification. (2) Adversarial point cloud generation methods based on point-by-point perturbation: The initial point-by-point perturbation method usually adopts the C&W algorithm framework based on Chamfer and Hausdorff distance constraints to learn and find the perturbation size for each point in the point cloud, thereby changing its corresponding 3D xyz coordinates to achieve adversarial point cloud generation. Subsequently, some recent works further adopted iterative gradient methods for the C&W algorithm strategy to generate more fine-grained adversarial perturbations, thereby improving the performance of adversarial perturbations. In order to maintain the geometric features of three-dimensional objects as much as possible, Liu et al. proposed not to perturb each point along the three-dimensional xyz direction, but to perturb each point along the normal direction of the section within a limited and strict boundary width to maintain the geometric smoothness between adjacent points.

[0004] Although both of the above-mentioned adversarial point cloud generation methods have achieved significant success rates, they both achieve perturbation generation through coordinate offset in the data space, and have the following two bottlenecks: (1) Existing point cloud perturbation methods all achieve noise interference by directly changing the coordinates. This perturbation strategy can easily lead to the presence of outliers in the generated adversarial point cloud and uneven distribution of points in local areas. 2) Existing point cloud perturbation methods do not consider preserving the geometric properties of the original 3D point cloud, such as piecewise smoothness, which exhibits a slowly changing underlying surface separated by sharp edges. However, whether it is a method that modifies local points or a method that perturbs the entire point cloud along the xyz direction, they will directly deform the geometric structure of the original point cloud, thereby losing the geometric properties of the original point cloud. Summary of the Invention

[0005] In order to solve the problem of low classification accuracy of existing point cloud classification models, the present invention proposes a method and system for improving the classification accuracy of point cloud classification models by generating adversarial samples based on the atlas domain. By capturing and retaining the geometric characteristics of 3D objects, imperceptible noise disturbances are added to the three-dimensional point cloud to generate an adversarial point cloud for the point cloud classification model, which is input into the point cloud classification model to assist in training, thereby further improving the classification accuracy of the model, that is, improving the accuracy of the model in identifying 3D objects based on point cloud data classification.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for improving point cloud classification accuracy based on the atlas domain includes the following steps:

[0008] Get the original point cloud data of 3D objects;

[0009] A KNN graph (k-Nearest Neighbour Graph) is constructed on the original point cloud to represent the geometric structure information of the original point cloud. The KNN graph converts the original point cloud data from the data domain to the graph domain through the graph Fourier transform (GFT).

[0010] Construct a spectrum filter to filter the spectrum features of the data transformed into the spectrum domain to generate a disturbed spectrum signal;

[0011] Through the inverse image Fourier transform (IGFT), the perturbed spectrum signal is converted back to the data domain to obtain the adversarial point cloud data.

[0012] Construct samples based on original point cloud data and adversarial samples based on adversarial point cloud data as training data, and input them into the point cloud classification model for classification training;

[0013] Use the trained point cloud classification model to classify the original point cloud data of the 3D object to be processed and output the classification results.

[0014] Preferably, the graph spectrum filter is a polynomial function of the spectrum eigenvalues, which perturbs the data transformed into the graph spectrum domain in a learnable manner, and generates a perturbed spectrum signal by minimizing the adversarial loss.

[0015] Preferably, the adversarial loss consists of a cross entropy loss function, a regularization term, and a low-frequency constraint term used to promote adversarial point cloud misclassification.

[0016] A system for improving the accuracy of point cloud classification based on the atlas domain includes a memory and a processor. A computer program is stored on the memory, and the processor implements the steps of the above method when executing the program.

[0017] Compared with the prior art, the advantages of the present invention are as follows:

[0018] Usually, signals can represent geometric features in the data domain in the atlas domain, and this feature is very important for point cloud classification tasks. Therefore, the present invention conducts in-depth research on how to effectively analyze and preserve the geometric information of 3D objects, and proposes to generate adversarial samples based on the atlas domain from a new perspective, and then train the point cloud classification model to improve its classification accuracy. Unlike previous point cloud perturbation methods in data space, the innovation of the present invention lies in the first use of the features representing the geometric structure of three-dimensional objects in the atlas domain, and the design of graph filters to disturb the GFT coefficients, thereby explicitly retaining the main geometric structure. In other words, each graph frequency represents certain structural features of the point cloud, corresponding to the corresponding global or local geometric context information. Therefore, it is crucial to study frequency components and their correlation with geometric information in the point cloud. Once the appropriate frequency band is perturbed, the corresponding geometric structure changes reflected in the data domain may be very small, making it difficult for the point cloud classification network to detect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the method for improving point cloud classification accuracy based on the atlas domain proposed in the present invention.

[0020] Figure 2 To study the atlas distribution map of 3D objects.

[0021] Figure 3 This is the atlas domain analysis diagram of 3D point cloud.

[0022] Figure 4 This is a visualization result diagram of the adversarial point cloud generated by the method of the present invention.

[0023] Figure 5Comparison of visualization results of adversarial point clouds generated by the method of the present invention and the existing method GeoA in spatial and frequency domains.

[0024] Figure 6 This is a comparison chart of the visualization results of the adversarial sample generation method proposed in this invention with and without low-frequency constraints (with: w / .LFC, without: w / o.LFC). DETAILED DESCRIPTION

[0025] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0026] The embodiment of the present invention specifically proposes a method for improving the classification accuracy of a point cloud classification model, which improves the classification accuracy of a point cloud classification model based on adversarial sample generation in the atlas domain. The processing process is as follows: Figure 1 The overall processing steps of this method include:

[0027] First, obtain clean point cloud data of the 3D object;

[0028] Then, a KNN graph structure is constructed to transform the point cloud into the graph domain through graph Fourier transform (GFT);

[0029] Then, a specially designed graph filter is used to perturb the original GFT coefficients in a learnable way;

[0030] Then, the perturbed spectrum signal is converted back to the data domain through the inverse image Fourier transform (IGFT) to construct the adversarial point cloud sample data;

[0031] Finally, the adversarial point cloud sample data is used to train a point cloud classification model to improve the classification accuracy of the target 3D object based on the point cloud data of the target 3D object.

[0032] In this method, GFT and IGFT belong to graph signal processing methods, which are described in detail as follows:

[0033] This method converts the image signal Represented by a diagram in is the set of vertices on the graph, ε is the set of edges. Is the adjacency matrix, which is a real symmetric matrix, and the matrix element a i,j In the transformation operator in graph signal processing, considering that the point cloud is piecewise smooth, this method focuses more on the graph Laplacian matrix L:=DA, where D is a diagonal matrix, d i,irepresents the degree of each node. Given real edges and non-negative edge weights in an undirected graph, L is real, symmetric, and positive semidefinite. Therefore, it can be characterized by L = UΛU T , where U is an orthogonal matrix containing eigenvectors, A=diag(λ1...λ n ) by the eigenvalue {λ1=0≤λ2≤...≤λ n In graph signal processing theory, the above eigenvalues ​​are called graph frequencies / spectra, where smaller eigenvalues ​​correspond to lower graph frequencies.

[0034] For any graph signal Its graph Fourier transform (GFT) can be defined as:

[0035]

[0036] Its inverse graphic Fourier transform (IGFT) can be defined as:

[0037]

[0038] If a graph that captures the signal structure well is constructed appropriately on the given input data, its GFT transform will lead to a compact representation of the graph signal in the spectral domain. Since U is an orthogonal matrix, both GFT and IGFT operations are lossless.

[0039] Let h(λ i ), (i=1,2,...,n) represents the frequency domain response of a spectrum domain filter, then its filtering operation can be defined as:

[0040]

[0041] The filter first transforms the data x into the GFT domain U T Then, each eigenvalue (i.e., graph frequency / spectrum) is filtered and finally projected back to the spatial domain / data domain through IGFT transformation to obtain the filtered output x′.

[0042] For example, an intuitive implementation of low-pass spectrum filtering is to eliminate all spectrum frequencies above a given bandwidth b while keeping frequencies below the given bandwidth b unchanged:

[0043]

[0044] In this method, the spectral domain and atlas filters use the atlas domain characteristics of the 3D point cloud. The atlas domain of the 3D point cloud is described below:

[0045] Typically, when constructing a suitable graph to capture the geometric structure of a point cloud well, the low-frequency components of the corresponding GFT coefficients primarily characterize the rough shape of the point cloud, while the high-frequency components represent fine details or noise (i.e., large shape variations, such as geometric contours) of the 3D object. This is because the eigenvectors of the graph Laplacian matrix vary gradually from low to high frequencies, capturing increasingly detailed 3D object structures. For example, all values ​​in the first eigenvector are 1, indicating a smooth surface, while the values ​​in the last eigenvector alternate between positive and negative, indicating fine details.

[0046] In order to deeply analyze and understand the spectral characteristics of the 3D point cloud, that is, how each frequency band contributes to the geometric structure of the point cloud in the data domain, this embodiment Figure 2 An intuitive example is provided in the spectral distribution diagram for studying 3D objects. Specifically, each point cloud is converted to the spectral domain, and their corresponding GFT coefficients and energy are plotted in the diagram. It can be seen that the GFT coefficients in the spectral domain have larger amplitudes in the low-frequency components and smaller amplitudes in the high-frequency components, indicating that most of the information is concentrated in the low-frequency components. The entire spectral domain is divided into three frequency bands by the energy distribution - the sum of the squares of the transform coefficients. Figure 2 As shown in Figure 2, the point cloud has almost 75% of its energy in the lowest 100 frequencies and almost 90% of its energy in the lowest 400 frequencies. Based on this observation, this embodiment sets three frequency bands for the point cloud in the ModelNet40 dataset: low frequency band (frequency [0, 100)), mid frequency band (frequency [100, 400)), and high frequency band (frequency [400, 1024)).

[0047] In addition, this embodiment also studies the effect of each frequency band on the geometric structure by setting some GFT coefficients to zero through spectrum filtering. Figure 3 Figure 2 shows a spectral domain analysis of a 3D point cloud. Taking a 3D aircraft as an example, the effects of different frequency bands are studied by removing certain frequency components in the spectral domain. (a) shows the original point cloud and spectral information; (b) shows the removal of mid- and high-frequency components; (c) shows the removal of low- and high-frequency components; (d) shows the removal of low- and mid-frequency components; (e) shows the removal of mid-frequency components; and (f) shows the removal of high-frequency components. When the GFT coefficients in the mid- and high-frequency bands are set to zero, the point cloud reconstructed using only the low-frequency band can only show the rough shape of the original object, lacking fine details such as the edge of a wing. Figure 3 Figures (c) and (d) show the point clouds reconstructed using only the mid-frequency band or the high-frequency band, respectively, further demonstrating the importance of the low-frequency band for constructing the shape of three-dimensional objects. Figure 3 By adding more information from the mid-frequency or high-frequency parts in (b), the reconstructed point cloud has richer local context, such as Figure 3As shown in Figures (e) and (f), it still lacks fine-grained details, such as the aircraft's engine. In summary, each frequency band represents different aspects of the point cloud's geometric characteristics. In particular, the low-frequency band represents the most basic contour geometry of the 3D point cloud, while the high-frequency band represents the local fine details and noise of the object.

[0048] Based on the aforementioned graph signal processing and spectral domain characteristics, this method generates adversarial examples in the spectral domain to train a point cloud classification model. This method selectively perturbs some frequencies in the spectral domain of the point cloud to optimally preserve the essential geometric information of the original point cloud. Specifically, this method learns a specific distribution of the spectral domain (i.e., the eigenvalues ​​of the graph Laplacian matrix) to apply perturbations in order to transform the potential features of the original point cloud into features of another different category.

[0049] In this method, we first input the clean point cloud of the 3D object Each point Is a vector containing coordinates (x, y, z). A trained 3D point cloud classification model f(·) can predict the accurate label that best represents the category of the point cloud object. Where c is the number of categories of three-dimensional objects. The goal of generating adversarial samples for classification tasks is to transform the point cloud P into the adversarial sample P′, so that f(P′) = y′ (targeted adversarial point cloud generation) or f(P′) ≠ y (untargeted adversarial point cloud generation), where But y'≠y.

[0050] The goal of this method's atlas-based adversarial sample generation is to learn destructive but imperceptible noise perturbations in the atlas domain of a 3D point cloud to generate an adversarial point cloud. This method learns perturbations in the atlas domain, while constraining the distance between the GFT coefficients before and after the perturbation, to minimize the adversarial loss while preserving the geometric features of the original 3D object. This point cloud perturbation method is implemented through atlas filtering in the GFT domain. Formally, the adversarial sample generation of this method can be formulated as the following optimization problem:

[0051]

[0052] where P′=φ IGFT (Δ(φ GFT (P)),

[0053]

[0054] Among them, L adv(P′, P, y) is the adversarial loss, Δ is the learnable perturbation in the spectral domain, and p represents the point cloud data. In the spectral domain constraint, ∈ is a threshold that limits the perturbation size in the spectral domain, which is used to preserve the original spectral domain features as much as possible. Therefore, the generated adversarial point cloud P′ is visually indistinguishable from its original sample P. where P′ represents the adversarial point cloud generation. To achieve perturbation in the spectral domain, this method uses a learnable distributed graph filter to design the noise perturbation Δ. Specifically, each GFT coefficient is filtered, where the graph filter is a polynomial function of the eigenvalue to fit the ideal spectral distribution. Used to learn the contribution distribution of each frequency component, It is used to learn the importance of each frequency signal to the final geometric characteristics. i and k correspond to the i-th and k-th contribution scores and importance parameters respectively, and n and K represent the total number of samples.

[0055] In order to optimize the learning of the best perturbation by back-propagating the gradient, the adversarial loss L adv (P′,P,y) is defined as follows:

[0056]

[0057] Among them, L class (P′,y) is used to promote the misclassification of point cloud P′ and is usually defined by cross entropy:

[0058]

[0059] Here p(·) is the softmax function applied to the output of the point cloud classification model to calculate the probability that the adversarial point cloud is classified as the adversarial category y′. By minimizing this loss function, this method will optimize the designed spectral perturbation Δ to mislead the point cloud classification model f(·). In addition, L reg (P′,P) is a regularization term that guides perturbation learning in the graph domain by minimizing the spatial distance between P′ and P. It is an additional low-frequency constraint term proposed in this method to preserve the basic geometric shape of the 3D point cloud and limit the perceptible noise. are the point clouds reconstructed based only on the low-frequency components of P′ and P, respectively.

[0060] In order to preserve the basic geometric structure of the original 3D object, this method further proposes a low-frequency constraint to limit more perturbations to be added to imperceptible high-frequency components. Although this method adds and optimizes perturbations in the atlas domain, these perturbations may interfere with certain spectral features, resulting in perceptible structural changes in local areas of the mapped spatial domain data. On the other hand, traditional constrained optimization may cause perturbations to be randomly distributed across the entire spectrum. Therefore, this method uses a new constraint method to limit the generation of perturbations so that they are imperceptible after acting on the original point cloud. Since low-frequency components mainly affect the contour shape of 3D objects, this method imposes constraints on the spectral perturbations on these components to guide the perturbations to focus more on high-frequency components representing fine details.

[0061] Specifically, in order to preserve the significant geometric structure characteristics of the original object, this embodiment transforms the original point cloud P and its adversarial sample P ′ The high-frequency components of are set to zero and a new point cloud is reconstructed using only their low frequencies as follows:

[0062]

[0063]

[0064] where h(λ i ) is a low-pass filter:

[0065]

[0066] Here, b is the upper bound of the low-frequency bandwidth, which is set to 400 in the experiment of this embodiment. Therefore, the low-frequency constraint between the original point cloud and its adversarial sample in this method can be defined as:

[0067]

[0068] Finally, this embodiment sends the generated high-quality adversarial samples to the point cloud classification model for retraining to help it further improve the classification accuracy.

[0069] like Figure 4 As shown in Figure 3, the adversarial samples generated by this method are imperceptible.

[0070] Figure 5 The distortion levels of adversarial samples generated by different methods are shown, and it can be seen that the spectral domain distortion of the proposed method is the lowest.

[0071] Figure 6 The effectiveness of the low-frequency constraints proposed in this paper is demonstrated, and higher quality adversarial samples can be generated.

[0072] In addition to 3D point cloud data, this method can also be used on 3D mesh data and other similar data.

[0073] Table 1 shows the adversarial performance results of different point cloud classification networks on the ModelNet40 dataset, where the smaller the perturbation size, the better.

[0074] Table 1

[0075]

[0076] Note: GSDA++ in Table 1 represents the method of the present invention.

[0077] Table 2 compares the robustness of different adversarial sample generation algorithms against different point cloud defense strategies for the PointNet classification network.

[0078] Table 2

[0079]

[0080] Note: GSDA++ in Table 2 represents the method of the present invention.

[0081] Table 3 compares the robustness of different adversarial sample generation algorithms to different data augmentation strategies for the PointNet classification network.

[0082] Table 3

[0083]

[0084] Note: GSDA++ in Table 3 represents the method of the present invention.

[0085] Table 4 shows the contribution of the adversarial samples generated by the present invention to the improvement of the accuracy of the point cloud classification model.

[0086] Table 4

[0087]

[0088] Note: * in Table 4 indicates the point cloud classification model that was retrained using the adversarial examples we generated.

[0089] Although the present invention has been disclosed as above by way of embodiments, they are not intended to limit the present invention. Any appropriate modification or equivalent substitution of the technical solution of the present invention by a person skilled in the art should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on that defined in the claims.

Claims

1. A method for improving point cloud classification accuracy based on the atlas domain, characterized in that: The following steps are involved: Get the original point cloud data of 3D objects; Constructing a KNN graph on the original point cloud to represent the geometric structure information of the original point cloud, and converting the original point cloud data from the data domain to the atlas domain through the graph Fourier transform (GFT); Construct a spectrum filter to filter the spectrum features of the data transformed into the spectrum domain to generate a disturbed spectrum signal; Through the inverse image Fourier transform (IGFT), the perturbed spectrum signal is converted back to the data domain to obtain the adversarial point cloud data. Construct samples based on original point cloud data and adversarial samples based on adversarial point cloud data as training data, and input them into the point cloud classification model for classification training; Use the trained point cloud classification model to classify the original point cloud data of the 3D object to be processed and output the classification results.

2. The method according to claim 1, wherein The graph filter is a polynomial function of the spectral eigenvalues, which uses a learnable method to perturb the data transformed into the graph domain and generates a perturbed spectral signal by minimizing the adversarial loss.

3. The method according to claim 2, wherein The expression for minimizing the adversarial loss is as follows: where P′=φ IGFT (Δ(φ GFT (P)), Among them, L adv (P′,P,y) represents the adversarial loss, P represents the original sample, P′ represents the adversarial point cloud, φ GFT () represents the Fourier transform of the graph, φ IGFT () represents the inverse Fourier transform, y represents the point cloud object category, p represents the point cloud data, Δ represents the learnable perturbation in the spectral domain, ∈ represents the threshold that limits the perturbation size in the spectral domain, where P′ represents adversarial point cloud generation, Represents learning the contribution distribution of each frequency component, Indicates the importance of learning each frequency signal to the final geometric characteristics, i and k represent the corresponding i-th and k-th contribution scores and importance parameters, respectively, and n and K represent the total number of samples.

4. The method according to claim 3, wherein The adversarial loss L adv (P′,P,y) is determined by the cross entropy loss function L used to promote adversarial point cloud misclassification class (P′,y), regularization term L reg (P′,P) and low-frequency constraints The composition is expressed as follows: in, Represent the point clouds reconstructed based only on the low-frequency components of P′ and P, respectively.

5. The method according to claim 4, wherein Among them, p() represents the softmax function acting on the output of the point cloud classification model, y represents the point cloud object category, and y′ represents the adversarial category.

6. The method according to claim 4, wherein L reg (P ′ ,P) by minimizing P ′ and P to guide perturbation learning in the graph domain.

7. The method according to claim 4, wherein in, h(λ i ) is a low-pass filter: b is the upper bound that defines the low-frequency bandwidth.

8. A system for improving point cloud classification accuracy based on the atlas domain, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images