A method and system for extracting feature points from point clouds based on self-attention mechanism

Abstract

This invention discloses a method and system for extracting feature points from point clouds based on a self-attention mechanism. The method includes: acquiring point cloud slices of a point cloud model; inputting the point cloud slices into a neural network to obtain multi-channel feature neighborhoods; performing MLP calculations on the multi-channel feature neighborhoods and then applying a self-attention mechanism to the calculation results to obtain global features; and sequentially performing max pooling, MLP, and FNN calculations on the global features to obtain the probability that the center point of the point cloud slice is a feature point. This invention obtains multi-channel feature neighborhoods from point cloud slices. In addition to the spatial location information of the point cloud, the multi-channel feature neighborhoods also include Euclidean distance information and center point neighborhood information, thus obtaining more semantic information. Combined with self-attention mechanism calculations and post-processing, the dimensionality of the output is reduced, decreasing the computational load of subsequent feature mapping; this allows for convenient and efficient acquisition of feature points.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud feature point extraction, and particularly relates to a point cloud feature point extraction method and system based on a self-attention mechanism. BACKGROUND

[0002] In recent years, with the wide application of three-dimensional laser scanners in reverse engineering, medical visualization, aerial survey, building model construction and other fields, three-dimensional laser point clouds play a very important role, and various industries have put forward higher requirements for the geometric processing method of point cloud data. Feature extraction technology of three-dimensional point cloud is one of the key technologies of point cloud data application, and has important application in simplification, visualization, surface reconstruction, surface smoothing, point cloud registration and other fields.

[0003] The feature points of three-dimensional point cloud refer to the points with high stability and representativeness in three-dimensional point cloud. These points can express the point set of various geometric shapes and surface concave-convex features of the model, and are often used as the basis for feature extraction in the processing of three-dimensional point cloud, and can be used for simplification, segmentation, description, matching and tracking of point cloud.

[0004] In order to extract feature points in three-dimensional point cloud, the prior art mostly fits the local plane of scattered point cloud, calculates the curvature or normal vector of the local plane, sets the normal vector or curvature change threshold, and extracts point cloud feature points by judging the threshold. This extraction method is complicated, has large amount of calculation, consumes much time, has low efficiency, and the feature extraction method of three-dimensional point cloud has high spatial complexity, so that its application in many production and life fields is less.

[0005] With the gradual development of three-dimensional point cloud technology, the feature extraction algorithm of three-dimensional point cloud shows more and more importance and wide application prospect. Feature point extraction technology is an important research topic of three-dimensional point cloud, and quickly and accurately extracting feature points from scattered point cloud plays an important role in realizing accurate reconstruction and accurate expression of the appearance and structure of geometric model. SUMMARY

[0006] The present application aims to provide a point cloud feature point extraction method and system based on a self-attention mechanism, to solve the problems of complicated process and low efficiency of point cloud feature point extraction in the prior art.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical scheme:

[0008] In a first aspect, the present application discloses a point cloud feature point extraction method based on a self-attention mechanism, comprising:

[0009] obtaining a point cloud slice of a point cloud model;

[0010] inputting the point cloud slice into a neural network to obtain a multi-channel feature neighborhood;

[0011] performing MLP calculation on the multi-channel feature neighborhood and performing self-attention mechanism calculation on the calculation result to obtain a global feature;

[0012] sequentially performing maximum pooling calculation, MLP calculation and FNN calculation on the global feature to obtain a probability that the point cloud slice center point is a feature point.

[0013] Further, the obtaining of the multi-channel feature neighborhood comprises:

[0014] performing farthest point sampling on the point cloud slice to select a neighborhood center point, and obtaining a plurality of sampling point neighborhoods centered on the neighborhood center point;

[0015] respectively calculating the Euclidean distances from the center points of the plurality of sampling point neighborhoods to the center point of the point cloud slice to obtain a feature neighborhood;

[0016] the feature neighborhood and the center point neighborhood jointly form the multi-channel feature neighborhood; wherein the center point neighborhood comprises the sampling point neighborhood and the point cloud slice.

[0017] Further, it further comprises feature mapping the probability that the point cloud slice center point is a feature point, calculating a loss function of the neural network, and correcting the multi-channel feature neighborhood output by the neural network according to the calculation result.

[0018] The calculation formula of the loss function is:

[0019]

[0020] wherein y i represents the label value of sample i, the positive class is 1 and the negative class is 0, p i represents the probability that sample i is predicted as a positive class, and N represents the number of input point clouds.

[0021] Further, the obtaining process of the point cloud slice is: randomly selecting equal proportions of feature points and non-feature points in the point cloud model, and selecting neighborhood data centered on the selected feature points and non-feature points as the point cloud slice.

[0022] Further, the method for obtaining the feature points and non-feature points comprises:

[0023] establishing a KDTree for the point cloud model, traversing each feature point in the model, obtaining a neighborhood, and obtaining a feature point data set

[0024] selecting non-feature points in the non-feature point region in equal proportion to the number of feature points, obtaining a neighborhood, and obtaining a non-feature point data set.

[0025] Further, the screening method of the feature points and non-feature points is: curvature calculation is performed on the point cloud model, points with curvature change greater than a set threshold in the point cloud model are taken as the feature points, and points with curvature change not greater than the set threshold in the point cloud model are taken as the non-feature points.

[0026] Further, the method further comprises adding different degrees of Gaussian noise to the point cloud model before the curvature calculation is performed on the point cloud model.

[0027] Further, the selection process of the point cloud model is: selecting point cloud models of different categories in equal proportions from free-form surface models and models with sharp features.

[0028] In the second aspect, the application discloses a point cloud feature point extraction system based on a self-attention mechanism, comprising a processor and a storage medium.

[0029] The storage medium is used for storing instructions.

[0030] The processor is used for operating according to the instructions to perform the steps of the method of any one of the first aspect.

[0031] In the third aspect, the application discloses a computer-readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps of the method of any one of the first aspect.

[0032] According to the above technical solution, the application has the following beneficial effects: the application obtains a multi-channel feature neighborhood through point cloud slicing, the multi-channel feature neighborhood comprises Euclidean distance information and center point neighborhood information in addition to spatial position information of the point cloud, more semantic information is obtained, the dimension of the output is reduced through self-attention mechanism calculation and post-processing, and the calculation amount of subsequent feature mapping is reduced; the feature points can be conveniently and efficiently obtained through the setting of the neural network and the self-attention mechanism. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 It is a whole flowchart of the feature point extraction method of the application.

[0034] Figure 2 It is a flowchart of the acquisition of the multi-channel feature neighborhood in the application.

[0035] Figure 3 It is a flowchart of the acquisition of the point cloud slice in the application.

[0036] Figure 4 It is a whole network schematic diagram of the feature point extraction method of the application.

[0037] Figure 5 It is an explanatory diagram of the feature expansion mechanism of the feature point extraction method of the application.

[0038] Figure 6 For the illustration of the self-attention mechanism in the application;

[0039] Figure 7 For the illustration of the data set making process in the application;

[0040] Figure 8 For the experimental comparison chart of the feature point extraction method of the application. DETAILED DESCRIPTION

[0041] In order to make the technical means, creative features, purposes and effects achieved by the application easy to understand, the application will be further described below in combination with specific embodiments.

[0042] The application discloses a neural network SA-EFNet capable of extracting point cloud feature points, which classifies each point of the input point cloud. Specifically, in order to obtain more point cloud feature semantic information, in addition to the existing three-dimensional spatial position information of the center point neighborhood, the position information of the neighboring point and the Euclidean distance information of the center point and the neighborhood point are introduced, and the probability of the center point being a feature point is judged by mixing the multi-dimensional feature relationship of the center point and the neighborhood point and the relative relationship of the center point feature and the neighborhood feature. In addition, the application also introduces a self-attention mechanism to extract more feature information of the network input, and introduces a skip-connection in the feature extraction of the self-attention mechanism. In order to speed up the prediction speed of the network, the network model is simplified and the calculation parameter amount of the model is reduced. The application provides a new research idea for point cloud data processing, which can accurately extract feature points in point cloud data, and the extraction effect is not easily affected by noise and has good robustness.

[0043] Embodiment 1

[0044] The application will be described below through specific embodiments. As shown in Figures 1 to 8 The application discloses a point cloud feature point extraction method based on a self-attention mechanism, which comprises the following steps: step 100, obtaining a point cloud slice of a point cloud model; step 200, inputting the point cloud slice into a neural network to obtain a multi-channel feature neighborhood; step 300, performing MLP calculation on the multi-channel feature neighborhood, and performing self-attention mechanism calculation on the calculation result to obtain global features; and step 400, sequentially performing maximum pooling calculation, MLP calculation and FNN calculation on the global features to obtain the probability of the center point of the point cloud slice being a feature point.

[0045] The application obtains a multi-channel feature neighborhood through point cloud slicing, the multi-channel feature neighborhood includes Euclidean distance information and center point neighborhood information in addition to spatial position information of the point cloud, more semantic information is obtained, the dimension of the output is reduced through self-attention mechanism calculation and post-processing, and the calculation amount of subsequent feature mapping is reduced; the feature points can be conveniently and efficiently obtained through the setting of the neural network and the self-attention mechanism.

[0046] As shown in Figure 3 Step 100 includes: step 101, selecting different types of point cloud models in equal proportions in a free-form surface model and a model with sharp features; step 102, randomly selecting equal proportions of feature points and non-feature points in the point cloud model, and selecting neighborhood data as the center of the selected feature points and non-feature points as point cloud slices.

[0047] In some further embodiments, the method for obtaining feature points and non-feature points includes: establishing a KDTree for the point cloud model, traversing each feature point in the model, obtaining a neighborhood, and obtaining a feature point data set In the non-feature point area, select non-feature points in equal proportions to the number of feature points, obtain a neighborhood, and obtain a non-feature point data set.

[0048] Further, the feature point discrimination method is: curvature calculation is performed on the point cloud model, points with a curvature change greater than a set threshold value in the point cloud model are taken as feature points, and points with a curvature change not greater than the set threshold value in the point cloud model are taken as non-feature points. In this process, for different models, a suitable curvature threshold value is selected, and points greater than the threshold value are set as feature points.

[0049] In some further embodiments, before the curvature calculation is performed on the point cloud model, different degrees of Gaussian noise are added to the point cloud model. Through this design, the generalization ability of the network model and the robustness of the algorithm can be improved.

[0050] As shown in Figure 2 Obtaining the multi-channel feature neighborhood includes: step 201, farthest point sampling is performed on the point cloud slice, a neighborhood center point is selected, and a plurality of sampling point neighborhoods are obtained with the neighborhood center point as the center.

[0051] Step 202, respectively calculating the Euclidean distances of the center points of the plurality of sampling point neighborhoods to the center point of the point cloud slice, and obtaining a feature neighborhood.

[0052] Step 203, the feature neighborhood and the center point neighborhood jointly form a multi-channel feature neighborhood; wherein the center point neighborhood includes the sampling point neighborhood and the point cloud slice.

[0053] In step 201, in the selected point cloud slice, the farthest point sampling is performed, and when sampling, the center point is taken as the first sampling point, so that the neighborhood points near the center point can be taken more evenly.

[0054] The multi-channel feature neighborhood is only MLP calculated, and then the obtained result is taken as the feature input of the self-attention mechanism.

[0055] The weight calculated by the self-attention mechanism is normalized by softmax, and the feature after the self-attention is obtained by combining the residual structure skip-connection structure. Then, the MLP calculation is used again, the output dimension is unified, the dimension of the output is reduced, and the calculation amount of the subsequent feature mapping is reduced. Then, the global feature after the feature extraction is subjected to max-pooling calculation, and the feature obtained is subjected to MLP calculation again, followed by two-layer full connection operation, to obtain the probability value of the center point as the feature point.

[0056] In the present application, step 500 is also included, and the probability values of all feature points are obtained by sequentially performing operations. A suitable global probability threshold is selected, and the feature points with probability values greater than the global probability threshold are output as global feature points.

[0057] In some further embodiments, the neural network is also modified to ensure efficient and accurate acquisition of the multi-channel feature neighborhood. Specifically, the probability of the point cloud slice center point as the feature point is subjected to feature mapping, the loss function of the neural network is calculated, and the multi-channel feature neighborhood output by the neural network is modified according to the calculation result.

[0058] The calculation formula of the loss function is:

[0059]

[0060] Wherein, y i represents the label value of sample i, the positive class is 1, and the negative class is 0, p i represents the probability that sample i is predicted as a positive class, and N represents the number of input point clouds. The network feature mapping output can more accurately obtain the probability size of the center point as the feature point.

[0061] In summary, the generalization ability of the model is used to realize the extraction of the feature point. In order to obtain more semantic information, the peripheral feature expansion structure is proposed in the present application. In addition to the displayed spatial position information, the Euclidean distance information between the slice point cloud center point and the neighborhood point cloud and the associated semantic information of the center point feature relative to the neighborhood feature are increased. At the same time, in order to better extract the implicit information of the center point and the neighborhood feature from the feature information, the standard transformer without induction bias is applied to 3D point cloud learning. In order to speed up the prediction speed of the model, the network model size of the transformer is simplified.

[0062] Example 2

[0063] Those skilled in the art will appreciate that embodiments of the application can be readily used as a method, a system or a computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code thereon.

[0064] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.

[0065] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.

[0066] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.

[0067] Within the scope of the application, various modifications can be made to the embodiments described above. Therefore, the above disclosed embodiments are to be considered illustrative only and not restrictive on the scope of the application. All modifications falling within the scope of the application or within the scope of equivalents of the application are intended to be embraced therein.

Claims

1. A method for extracting feature points of a point cloud based on a self-attention mechanism, characterized in that, The method comprises the following steps: Step 1, obtaining a point cloud slice of a point cloud model; The obtaining process of the point cloud slice is as follows: randomly selecting equal proportions of feature points and non-feature points in the point cloud model, and selecting neighborhood data centered on the selected feature points and non-feature points as the point cloud slice; wherein the method for obtaining the feature points and non-feature points comprises: A KDTree is established for the point cloud model, each feature point in the model is traversed, a neighborhood is obtained, and a feature point data set is obtained ; selecting equal proportions of non-feature points in the non-feature point region, obtaining a neighborhood, and obtaining a non-feature point data set; wherein the screening method for the feature points and non-feature points comprises: performing curvature calculation on the point cloud model, and taking points with a curvature change greater than a set threshold in the point cloud model as feature points; taking points with a curvature change not greater than a set threshold in the point cloud model as non-feature points; wherein the method further comprises adding different degrees of Gaussian noise to the point cloud model before performing the curvature calculation on the point cloud model; Step 2, inputting the point cloud slice into a neural network to obtain a multi-channel feature neighborhood; wherein the method for obtaining the multi-channel feature neighborhood comprises: performing farthest point sampling on the point cloud slice, selecting a neighborhood center point, and obtaining a plurality of sampling point neighborhoods centered on the neighborhood center point; calculating the Euclidean distance from the center points of the plurality of sampling point neighborhoods to the center point of the point cloud slice respectively to obtain a feature neighborhood; the feature neighborhood and the center point neighborhood jointly form a multi-channel feature neighborhood; wherein the center point neighborhood comprises the sampling point neighborhood and the point cloud slice; Step 3, performing MLP calculation on the multi-channel feature neighborhood, and performing self-attention mechanism calculation on the calculation result to obtain global features; Step 4, sequentially performing maximum pooling calculation, MLP calculation and FNN calculation on the global features to obtain the probability that the center point of the point cloud slice is a feature point; Step 5, performing feature mapping on the probability that the center point of the point cloud slice is a feature point, calculating a loss function of the neural network, and correcting the multi-channel feature neighborhood output by the neural network according to the calculation result; The calculation formula of the loss function is: ； wherein, denotes a label value of a sample , positive class is 1, negative class is 0, denotes a label value of a sample predicted as a positive class, denotes the number of input point clouds.

2. The point cloud feature point extraction method based on self-attention mechanism according to claim 1, characterized in that, The selection process of the point cloud model is as follows: selecting equal proportions of different types of point cloud models in free-form surface models and models with sharp features.

3. A point cloud feature point extraction system based on a self-attention mechanism, characterized in that, The device comprises a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to perform the steps of the method according to any one of claims 1-2.

4. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method according to any one of claims 1-2.

Images