An adaptive three-dimensional point cloud local sampling method based on maximum pooling guidance

Abstract

The application provides a kind of adaptive three-dimensional point cloud local sampling method based on maximum pooling guidance, comprising, obtaining the point cloud data of three-dimensional image;Using depth vision module obtains the feature of each point in point cloud data and the coordinate corresponding to each point, sampling point cloud data and each point after sampling as center point;Using local sampling method, M neighborhood points are sampled for center point;Center point feature and neighborhood point and the coordinate difference of center point are input into adaptive neighborhood scoring module, and the score of M neighborhood points is obtained, and the highest N neighborhood points are selected as the final neighborhood points.The application designs a more effective local sampling method, selects more effective neighborhood points through adaptive neighborhood point scoring module, and reduces the number of redundant points.

Description

Technical Field

[0001] This invention belongs to the field of computer point cloud analysis and feature learning technology. Background Technology

[0002] With the development of 3D sensors, point cloud analysis has gained increasing attention. Unlike 2D deep learning, point cloud data is sparse and unordered, meaning that successful 2D models cannot be directly applied to it. PointNet and PointNet++ were proposed to address this issue, extracting local geometric features from point clouds through local sampling methods and aggregation layers, significantly improving point cloud analysis performance. Subsequent works have built upon PointNet++, further improving performance by designing more efficient architectures for modeling local geometric features. However, no work has yet directly analyzed and addressed the limitations of existing local sampling methods.

[0003] Max pooling is the most commonly used aggregation method in point cloud models. This invention observes that max pooling can adaptively select effective points in the neighborhood because points not captured by max pooling do not provide effective features during neighborhood feature aggregation; therefore, these points can be simply considered redundant. Based on this, this invention finds that while local sampling methods such as kNN and ball query used by PointNet++ and its variants can effectively extract local neighborhoods, they also sample a large number of redundant points that offer no benefit to the target task. For example, in the point cloud segmentation task on the S3DIS dataset using PointNet++, the proportion of redundant points is as high as 65%, meaning that only 35% of the neighborhood points provide effective features. Therefore, the effectiveness of existing local sampling methods still has significant room for improvement. The purpose of this invention is to design a new local sampling method to increase the number of effective points in existing local sampling methods, helping the model better aggregate local features and improve model performance. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the purpose of this invention is to propose an adaptive 3D point cloud local sampling method based on max pooling guidance, which uses an adaptive neighborhood point scoring module to select more effective neighborhood points and reduce the number of redundant points.

[0006] To achieve the above objectives, a first aspect of the present invention proposes an adaptive 3D point cloud local sampling method based on max pooling guidance, comprising:

[0007] Acquire point cloud data of a 3D image;

[0008] The feature of each point in the point cloud data and the coordinates of each point are obtained using a depth vision module. The point cloud data is sampled and each sampled point is used as the center point.

[0009] M neighboring points of the center point are sampled using a local sampling method;

[0010] The center point features and the coordinate differences between the neighboring points and the center point are input into the adaptive neighborhood scoring module to obtain the scores of the M neighboring points. The N neighboring points with the highest scores are selected as the final neighboring points.

[0011] In addition, the adaptive 3D point cloud local sampling method based on max pooling guidance according to the above embodiments of the present invention may also have the following additional technical features:

[0012] Furthermore, in one embodiment of the present invention, the depth vision module includes PointNet++ and PointMLP, and the local sampling method includes kNN and ball query.

[0013] Furthermore, in one embodiment of the present invention, the adaptive neighborhood scoring module consists of a three-layer MLP, and the input is the features of the center point and the coordinate difference between the neighboring points and the center point.

[0014] Furthermore, in one embodiment of the present invention, the center point features and the coordinate differences between the neighboring points and the center point are input into the adaptive neighborhood scoring module, including:

[0015] The first layer of MLP is used to process the two inputs, and the outputs of the two inputs are multiplied and then passed through the last layer of MLP to obtain the scores of M neighborhood points; it also includes setting the output of the last layer of MLP to low dimension.

[0016] Furthermore, in one embodiment of the present invention, inputting the center point features and the coordinate differences between the neighboring points and the center point into the adaptive neighborhood scoring module includes:

[0017] The first layer of MLP is used to process the two inputs, and the outputs of the two inputs are multiplied and then passed through the last layer of MLP to obtain the scores of M neighborhood points; it also includes setting the output of the last layer of MLP to low dimension.

[0018] Furthermore, in one embodiment of the present invention, it further includes:

[0019] Max pooling is used to select valid points from M neighborhood points, and these valid points are used as positive samples and redundant points as negative samples. Then, a binary classification task is used to train an adaptive neighborhood scoring module.

[0020] To achieve the above objectives, a second aspect of the present invention provides an adaptive 3D point cloud local sampling device based on max pooling guidance, comprising the following modules:

[0021] The acquisition module is used to acquire point cloud data of 3D images;

[0022] The preprocessing module is used to obtain the features of each point in the point cloud data and the coordinates corresponding to each point using the depth vision module, to sample the point cloud data and to take each sampled point as the center point.

[0023] The sampling module is used to sample M neighboring points of the center point using a local sampling method;

[0024] The output module is used to input the center point features and the coordinate differences between the neighboring points and the center point into the adaptive neighborhood scoring module to obtain the scores of the M neighboring points, and select the N neighboring points with the highest scores as the final neighboring points.

[0025] To achieve the above objectives, a third aspect of the present invention provides a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements an adaptive 3D point cloud local sampling method based on max pooling guidance as described above.

[0026] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements an adaptive 3D point cloud local sampling method based on max pooling guidance as described above.

[0027] The adaptive 3D point cloud local sampling method based on max pooling guidance proposed in this invention first introduces a new metric for measuring neighborhood effectiveness. Based on this new metric, an adaptive lightweight neighborhood point scoring module is designed to score neighborhood points and select more effective ones. Furthermore, for training the neighborhood scoring module, this invention does not require additional supervision signals; instead, it uses a self-reinforcing learning framework to provide labels for the scoring module. Attached Figure Description

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0029] Figure 1 This is a flowchart illustrating an adaptive 3D point cloud local sampling method based on max pooling guidance provided in an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of a maximum pooling method for screening effective neighborhood points, provided in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of a self-reinforcing learning framework provided in an embodiment of the present invention.

[0032] Figure 4 This is a schematic diagram of a local sampling device for adaptive 3D point clouds based on max pooling guidance, provided in an embodiment of the present invention. Detailed Implementation

[0033] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0034] The following describes an embodiment of the present invention, an adaptive 3D point cloud local sampling method based on max pooling guidance, with reference to the accompanying drawings.

[0035] Figure 1 This is a flowchart illustrating an adaptive 3D point cloud local sampling method based on max pooling guidance provided in an embodiment of the present invention.

[0036] like Figure 1 As shown, the adaptive 3D point cloud local sampling method based on max pooling guidance includes the following steps:

[0037] S101: Acquire point cloud data of a 3D image;

[0038] S102: Use the depth vision module to obtain the features of each point in the point cloud data and the coordinates of each point, sample the point cloud data and take each sampled point as the center point;

[0039] S103: Use a local sampling method to sample M neighboring points of the center point;

[0040] S104: Input the center point features and the coordinate differences between the neighboring points and the center point into the adaptive neighborhood scoring module to obtain the scores of M neighboring points, and select the N neighboring points with the highest scores as the final neighboring points.

[0041] like Figure 2As shown, max pooling has the advantage of easily selecting effective neighborhood points. Max pooling selects a neighborhood point with the maximum value for each feature channel. Typically, the number of feature channels is much larger than the number of neighborhood points. Therefore, ideally, every neighborhood point can be captured by max pooling, and all features can be effectively utilized. This invention considers these points as effective points, in which case there is no redundant information in the neighborhood. However, in reality, many points are not captured by max pooling. Therefore, the effectiveness of the neighborhood is evaluated by calculating the proportion of points captured by max pooling to the total number of neighborhood points. The calculation of this new metric is very simple and efficient, and requires no additional parameters.

[0042] Furthermore, in one embodiment of the present invention, the depth vision module includes PointNet++ and PointMLP, and the local sampling method includes kNN and ball query.

[0043] The neighborhood scoring module consists of a three-layer MLP. Its inputs are the features of the center point and the coordinate differences between neighboring points and the center point. This invention uses a separate MLP layer to process each input, and then multiplies the two outputs before passing them through the final MLP layer to obtain the scores for the M neighboring points. By setting the MLP output to a lower dimension, the computational load is significantly reduced, allowing this invention to greatly improve model performance with minimal increase in computational cost. Finally, the N neighboring points with the highest scores are selected to form a more effective neighborhood.

[0044] Furthermore, in one embodiment of the present invention, the adaptive neighborhood scoring module consists of a three-layer MLP, and the input is the features of the center point and the coordinate difference between the neighboring points and the center point.

[0045] Furthermore, in one embodiment of the present invention, the center point features and the coordinate differences between the neighboring points and the center point are input into the adaptive neighborhood scoring module, including:

[0046] The first layer of MLP processes the two inputs, and the outputs of the two inputs are multiplied and then passed through the last layer of MLP to obtain the scores of M neighborhood points; this also includes setting the output of the last layer of MLP to low dimension.

[0047] To train the neighborhood scoring module to accurately evaluate the effectiveness of neighborhood points, this invention uses a self-reinforcing learning framework, the overall process of which is as follows: Figure 3As shown. Because the shallow layers of the model lack high-level semantic information, its aggregation process struggles to determine the validity of neighboring points and cannot provide accurate labels for training the neighborhood scoring module. Therefore, this invention employs an additional task-oriented branch. In this branch, max pooling is used to simply filter valid points from M neighbors, treating valid points as positive samples and redundant points as negative samples. A binary classification task is then used to train the neighborhood scoring module. This invention aims to enable the neighborhood scoring module to output higher scores for valid points through this training method. Simultaneously, to encourage this branch to learn high-level semantic information, an auxiliary classifier is added to predict the target task. The proposed learning framework requires no additional supervision signals and can generate its own labels for training the neighborhood scoring module, which can be discarded during inference.

[0048] Furthermore, in one embodiment of the present invention, it further includes:

[0049] Max pooling is used to select valid points from M neighborhood points, and valid points are used as positive samples and redundant points are used as negative samples. Then, a binary classification task is used to train the adaptive neighborhood scoring module.

[0050] The adaptive 3D point cloud local sampling method based on max pooling guidance proposed in this invention first introduces a new metric for measuring neighborhood effectiveness. Based on this new metric, an adaptive lightweight neighborhood point scoring module is designed to score neighborhood points and select more effective ones. Furthermore, for training the neighborhood scoring module, this invention does not require additional supervision signals; instead, it uses a self-reinforcing learning framework to provide labels for the scoring module.

[0051] To achieve the above embodiments, the present invention also proposes an adaptive 3D point cloud local sampling device based on max pooling guidance.

[0052] Figure 4 This is a schematic diagram of an adaptive 3D point cloud local sampling device based on max pooling guidance, provided in an embodiment of the present invention.

[0053] like Figure 4 As shown, the adaptive 3D point cloud local sampling device based on max pooling guidance includes: an acquisition module 100, a preprocessing module 200, a sampling module 300, and an output module 400, wherein...

[0054] The acquisition module is used to acquire point cloud data of 3D images;

[0055] The preprocessing module is used to obtain the features of each point in the point cloud data and the coordinates of each point using the depth vision module, to sample the point cloud data and to use each sampled point as the center point.

[0056] The sampling module is used to sample M neighboring points of the center point using a local sampling method.

[0057] The output module is used to input the center point features and the coordinate differences between the neighboring points and the center point into the adaptive neighborhood scoring module to obtain the scores of M neighboring points, and select the N neighboring points with the highest scores as the final neighboring points.

[0058] Furthermore, in one embodiment of the present invention, the output module is also used for:

[0059] The first layer of MLP processes the two inputs, and the outputs of the two inputs are multiplied and then passed through the last layer of MLP to obtain the scores of K neighborhood points; this also includes setting the output of the last layer of MLP to low dimension.

[0060] Furthermore, in one embodiment of the present invention, a training module is further included, for:

[0061] Max pooling is used to select valid points from M neighborhood points, and valid points are used as positive samples and redundant points are used as negative samples. Then, a binary classification task is used to train the adaptive neighborhood scoring module.

[0062] To achieve the above objectives, a third aspect of the present invention provides a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the adaptive 3D point cloud local sampling method based on max pooling guidance as described above.

[0063] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the adaptive 3D point cloud local sampling method based on max pooling guidance as described above.

[0064] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0065] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0066] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An adaptive 3D point cloud local sampling method based on max pooling guidance, characterized in that, Includes the following steps: Acquire point cloud data of a 3D image; The feature of each point in the point cloud data and the coordinates of each point are obtained using a depth vision module. The point cloud data is sampled and each sampled point is used as the center point. M neighboring points of the center point are sampled using a local sampling method; The center point features and the coordinate differences between the neighboring points and the center point are input into the adaptive neighborhood scoring module to obtain the scores of the M neighboring points. The N neighboring points with the highest scores are selected as the final neighboring points. During the training phase, max pooling is used to select effective points from the M neighborhood points, and these effective points are used as positive samples, while the remaining points are used as negative samples. The adaptive neighborhood scoring module is trained using a binary classification task. An auxiliary classifier is also added during the training phase to predict the target task, guiding the adaptive neighborhood scoring module to learn advanced semantic information. No additional supervision signal is required during the training process, and the adaptive neighborhood scoring module is discarded during the inference phase.

2. The method according to claim 1, characterized in that, The depth vision module includes PointNet++ and PointMLP, and the local sampling methods include kNN and ball query.

3. The method according to claim 1, characterized in that, The adaptive neighborhood scoring module consists of a three-layer MLP.

4. The method according to claim 1, characterized in that, The step of inputting the center point features and the coordinate differences between the neighboring points and the center point into the adaptive neighborhood scoring module includes: The first layer of MLP is used to process the two inputs, and the outputs of the two inputs are multiplied and then passed through the last layer of MLP to obtain the scores of M neighborhood points; it also includes setting the output of the last layer of MLP to low dimension.

5. An adaptive 3D point cloud local sampling device based on max pooling guidance, characterized in that, Includes the following modules: The acquisition module is used to acquire point cloud data of 3D images; The preprocessing module is used to obtain the features of each point in the point cloud data and the coordinates corresponding to each point using the depth vision module, to sample the point cloud data and to take each sampled point as the center point. The sampling module is used to sample M neighboring points of the center point using a local sampling method; The output module is used to input the center point features and the coordinate difference between the neighboring points and the center point into the adaptive neighborhood scoring module to obtain the scores of the M neighboring points, and select the N neighboring points with the highest scores as the final neighboring points. During the training phase, max pooling is used to select effective points from the M neighborhood points, and these effective points are used as positive samples, while the remaining points are used as negative samples. The adaptive neighborhood scoring module is trained using a binary classification task. An auxiliary classifier is also added during the training phase to predict the target task, guiding the adaptive neighborhood scoring module to learn advanced semantic information. No additional supervision signal is required during the training process, and the adaptive neighborhood scoring module is discarded during the inference phase.

6. The apparatus according to claim 5, characterized in that, The output module is also used for: The first layer of MLP is used to process the two inputs, and the outputs of the two inputs are multiplied and then passed through the last layer of MLP to obtain the scores of M neighborhood points; it also includes setting the output of the last layer of MLP to low dimension.

7. A computer device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the adaptive 3D point cloud local sampling method based on max pooling guidance as described in any one of claims 1-4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the adaptive 3D point cloud local sampling method based on max pooling guidance as described in any one of claims 1-4.

Images