Feature extraction method, classification and segmentation method and system for three-dimensional point cloud data
By employing electrostatic field principles to set source and probe charges for 3D point cloud data, the method enhances feature extraction reliability and accuracy, addressing noise issues and capturing geometric structure, thus improving classification and segmentation performance.
Patent Information
- Application Number
- CN202411394057.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-08
- Publication Date
- 2024-12-10
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing deep learning-based methods for feature extraction from 3D point cloud data struggle with noise and lack of geometric structure, leading to reduced reliability and accuracy, especially in real-world datasets like ScanObjectNN and S3DIS.
A feature extraction method utilizing the concept of physical electrostatic fields, where source charges are set at the vertices of a minimum bounding box, and probe point charges are constructed to calculate point geometry Coulomb force, combined with deep learning networks for robust feature extraction.
The method achieves high reliability and accuracy in feature extraction, classification, and segmentation of 3D point cloud data, being robust to noise and effectively capturing geometric structure, outperforming existing methods on real-world datasets.
Smart Images
Figure CN118887416B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of digital image processing, and in particular relates to a feature extraction method, a classification and segmentation method and a system for three-dimensional point cloud data. Background Art
[0002] 3D point cloud data is one of the most basic representations of 3D data. With the rapid development of deep learning technology, deep learning-based point cloud analysis has become a prominent topic, attracting widespread attention from researchers. However, due to the irregularity and disorder of 3D point cloud data, how to effectively characterize the input 3D point cloud data in deep learning has become a key research focus.
[0003] Currently, the vast majority of research approaches focus on extracting feature information from 3D point cloud data using max pooling as a feature aggregation function. However, this feature extraction approach has significant drawbacks. While these approaches consistently achieve superior results on laboratory datasets (such as the ModelNet40 dataset), their performance often degrades dramatically when processing real-world scan datasets (such as the ScanObjectNN dataset). This is because real-world scan datasets often contain significant amounts of noise, such as local data loss, irrelevant additions, or jitter. These additional high-frequency features, which are not inherent to the original model, propagate from local features to global features through the max pooling aggregation function, impacting the overall performance of the framework. Furthermore, because 3D point cloud data only contains 3D coordinates and lacks any geometric structure or connectivity information, relying solely on implicit features learned by the network is difficult to distinguish these additional high-frequency features that are not inherent to the model. These issues significantly reduce the reliability and accuracy of existing feature extraction approaches for 3D point cloud data, leading to failure in subsequent classification and segmentation tasks. Summary of the Invention
[0004] One of the purposes of the present invention is to provide a feature extraction method for three-dimensional point cloud data with high reliability, good accuracy and good applicability.
[0005] A second object of the present invention is to provide a classification and segmentation method that includes the feature extraction method of the three-dimensional point cloud data.
[0006] A third object of the present invention is to provide a system for implementing the feature extraction method of the three-dimensional point cloud data.
[0007] The feature extraction method of the three-dimensional point cloud data provided by the present invention comprises the following steps:
[0008] S1. Obtain a known 3D point cloud dataset;
[0009] S2 preprocesses the three-dimensional point cloud dataset obtained in step S1;
[0010] S3. Based on the concept of physical electrostatic fields, source charges are set at the vertices of the minimum bounding box of the 3D point cloud dataset. Probe point charges are then constructed for each point in the 3D point cloud dataset based on the set source charges.
[0011] S4. Based on the source charge and probe point charge set in step S3 and the physical Coulomb's law, the point geometric Coulomb force of each point in the three-dimensional point cloud dataset is calculated;
[0012] S5. The point geometry Coulomb force of each point obtained in step S4 is combined with the corresponding three-dimensional coordinates to obtain a training data set, and the deep learning network is trained to obtain a feature extraction network;
[0013] S6. Using the feature extraction network obtained in step S5, complete feature extraction of the target 3D point cloud data.
[0014] The step S1 specifically includes the following steps:
[0015] Acquire a known three-dimensional point cloud dataset; the three-dimensional point cloud dataset includes a ScanObjectNN dataset and an S3DIS dataset.
[0016] The step S2 specifically includes the following steps:
[0017] For the ScanObjectNN dataset, the 3D point cloud data in the dataset is randomly sampled to a set number of points. The height of the 3D point cloud data is then calculated based on the y-axis. The 3D point cloud data is then scaled according to a set multiple range and randomly rotated along the set range.
[0018] For the S3DIS dataset, the height of the 3D point cloud data in the dataset is calculated according to the y-axis, and then the input color information is randomly discarded, and finally the color automatic contrast alignment is performed.
[0019] Setting source charges at the vertices of the minimum bounding box of the three-dimensional point cloud data set in step S3 includes the following steps:
[0020] Obtain the minimum bounding box of the preprocessed three-dimensional point cloud data obtained in step S2;
[0021] Set the source charge at the vertices of the minimum bounding box of the 3D point cloud dataset, and the set source charge meets the following requirements:
[0022] Requirement 1: The position and charge of the source charge are fixed values;
[0023] Requirement 2: The source charge can encode the structural information of the 3D point cloud data according to the set requirements;
[0024] Requirement 3: The electric field strength generated by the source charge must be robust to local noise in the 3D point cloud data.
[0025] The step of setting source charges at the vertices of the minimum bounding box of the three-dimensional point cloud data set specifically includes the following steps:
[0026] Get the minimum bounding box of the preprocessed 3D point cloud data obtained in step S2:
[0027] In the pre-processed 3D point cloud data obtained in step S2, the X-axis maximum value of each point in the 3D point cloud data is obtained. , Y-axis maximum value , Z-axis maximum value , X-axis minimum value , Y-axis minimum value and Z-axis minimum ;
[0028] Get the 8 vertices of the minimum bounding box, which are the first vertex , second vertex , the third vertex , the fourth vertex , the fifth vertex , the sixth vertex , the seventh vertex and the eighth vertex ;
[0029] Set the source charges at the vertices of the minimum bounding box of a 3D point cloud dataset:
[0030] Set a source charge at each of the 8 vertices of the minimum bounding box , s is the label of the source charge and its value range is 1~8;
[0031] For the minimum bounding box, divide the minimum bounding box into 8 sub-boxes along the midpoint of each edge, and set the sth sub-box to include only the source charge The generated field strength ;
[0032] Divide the 8 vertices of the minimum bounding box into two groups, namely the first group and the second set , and the first set and the second set are located on opposite sides of the bounding box of the minimum bounding box, the starting point and share the same side of the bounding box;
[0033] The first set and the second set form two directional rings in sequence, and mark the vertices according to the order of each vertex in the directional ring;
[0034] For each vertex in each set , the vertex The next vertex in the corresponding direction ring Connect to get the first vector for ; The first vector and the center point of the minimum bounding box Connect to get the second vector for ; Calculate the source charge for .
[0035] The step S3 of constructing the probe point charge of each point in the three-dimensional point cloud data set based on the set source charge specifically includes the following steps:
[0036] Each point in the 3D point cloud data set is used as a probe point charge placed in the electrostatic field generated by the source charge;
[0037] It is assumed that the probe point charges have no influence on each other, and the smaller the distance between the probe point charges, the higher the similarity;
[0038] For the 8 vertices of the minimum bounding box , for the i-th point in the s-th subframe ,Will and Concatenate to get the third vector for ;Will and Connect to get the fourth vector for ,in Vertex The previous node in the corresponding direction loop; calculate the probe point charge for .
[0039] The step S4 specifically includes the following steps:
[0040] The following formula is used to calculate the center point of the three-dimensional point cloud dataset: Coulomb force at point geometry : Where k is the electrostatic force constant; r is the source charge with dot The distance between them.
[0041] The deep learning network described in step S5 specifically includes the following steps:
[0042] The deep learning network comprises eleven layers connected in series;
[0043] The first layer consists of a one-dimensional convolutional layer;
[0044] The second layer consists of two one-dimensional convolutional layers, two batch normalization layers, and an activation function layer connected in series;
[0045] The third layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0046] The fourth layer consists of three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0047] The fifth layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0048] The sixth layer consists of three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0049] The seventh layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0050] The eighth layer consists of three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0051] The ninth layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0052] The tenth layer consists of two one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0053] The eleventh layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0054] The deep learning network is used to extract feature information of the input three-dimensional point cloud data layer by layer.
[0055] The step S6 specifically includes the following steps:
[0056] For the target 3D point cloud data, preprocessing is first performed;
[0057] For the pre-processed target 3D point cloud data, set the corresponding source charge and probe point charge, and calculate the point geometry Coulomb force;
[0058] Merge the point geometry Coulomb force with the corresponding 3D coordinates in the target 3D point cloud data;
[0059] The merged data is input into the feature extraction network obtained in step S5, and the output of the feature extraction network is the features of the extracted target three-dimensional point cloud data.
[0060] The present invention also provides a classification and segmentation method including the feature extraction method of the three-dimensional point cloud data, further comprising the following steps:
[0061] S7. Determine the classification and segmentation requirements for the target 3D point cloud data;
[0062] S8. If the classification requirement is met, the features of the target three-dimensional point cloud data extracted in step S6 are input into the corresponding classification network to complete the classification of the target three-dimensional point cloud data;
[0063] If segmentation is required, the features of the target three-dimensional point cloud data extracted in step S6 are input into the corresponding segmentation network to complete the segmentation of the target three-dimensional point cloud data.
[0064] The present invention also provides a system for implementing the feature extraction method of the three-dimensional point cloud data, comprising a data acquisition module, a data processing module, a charge setting module, a Coulomb force calculation module, a network training module and a feature extraction module; the data acquisition module, the data processing module, the charge setting module, the Coulomb force calculation module, the network training module and the feature extraction module are connected in series in sequence; the data acquisition module is used to acquire a known three-dimensional point cloud data set and upload the data information to the data processing module; the data processing module is used to pre-process the acquired three-dimensional point cloud data set according to the received data information and upload the data information to the charge setting module; the charge setting module is used to set source electric charge at the vertices of the minimum bounding box of the three-dimensional point cloud data set based on the concept of physical electrostatic field according to the received data information. Charge, and construct the probe point charge of each point in the three-dimensional point cloud dataset based on the set source charge, and upload the data information to the Coulomb force calculation module; the Coulomb force calculation module is used to calculate the point geometry Coulomb force of each point in the three-dimensional point cloud dataset based on the set source charge and probe point charge and physical Coulomb's law according to the received data information, and upload the data information to the network training module; the network training module is used to merge the obtained point geometry Coulomb force of each point with the corresponding three-dimensional coordinates according to the received data information to obtain a training dataset, and train the deep learning network to obtain a feature extraction network, and upload the data information to the feature extraction module; the feature extraction module is used to complete the feature extraction of the target three-dimensional point cloud data using the obtained feature extraction network according to the received data information.
[0065] The feature extraction method, classification and segmentation method and system for three-dimensional point cloud data provided by the present invention achieve the description of three-dimensional point cloud data and the analysis and extraction of intrinsic information based on physical electrical concepts by setting source charges and probe point charges in the three-dimensional point cloud data. Therefore, the present invention has low-dimensional and explicit characteristics and is more robust to real data. It can not only realize feature extraction of three-dimensional point cloud data, but also has higher reliability, better accuracy and better applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 Schematic diagram of the feature extraction method of the present invention.
[0067] Figure 2 Schematic diagram of the construction of the virtual field source charge and probe point charge in the feature extraction method of the present invention.
[0068] Figure 3 Schematic diagram of the vertex direction ring of the minimum bounding box of the feature extraction method of the present invention, where Figure 3 (a) is a schematic diagram of the direction ring of the first set. Figure 3 (b) is a schematic diagram of the direction ring of the second set.
[0069] Figure 4 This is a schematic diagram of the qualitative comparison results of the point cloud segmentation task based on the feature extraction method of the present invention, where Figure 4 (a) is a schematic diagram of the original input point cloud data. Figure 4 (b) is a schematic diagram of the segmentation results of the comparison scheme. Figure 4 (c) is a schematic diagram of the segmentation result of the solution of the present invention. Figure 4 (d) is a schematic diagram of the actual segmentation result.
[0070] Figure 5 Schematic diagram of the process flow of the classification and segmentation method of the present invention.
[0071] Figure 6 Schematic diagram of the functional modules of the system of the present invention. DETAILED DESCRIPTION
[0072] like Figure 1 The figure shows a flow chart of the feature extraction method of the present invention. The feature extraction method of the three-dimensional point cloud data provided by the present invention comprises the following steps:
[0073] S1. Obtain a known 3D point cloud dataset; specifically, the following steps are included:
[0074] Acquire a known three-dimensional point cloud dataset; the three-dimensional point cloud dataset includes a ScanObjectNN dataset and an S3DIS dataset;
[0075] The ScanObjectNN dataset is a real scan dataset for classification tasks. It consists of 15 classes and 15,000 real objects. This dataset introduces the presence of high-frequency noise, occlusion, and background noise, which brings additional challenges. The S3DIS dataset is a dataset for segmentation tasks. It is an indoor semantic segmentation dataset consisting of 271 scenes in 6 large-scale real indoor areas. Each point in the dataset is labeled as one of 13 categories.
[0076] S2. Preprocessing the 3D point cloud dataset obtained in step S1; specifically comprising the following steps:
[0077] For the ScanObjectNN dataset, randomly sample the 3D point cloud data in the dataset to a set number of points (preferably 1024 points), then calculate the height of the 3D point cloud data according to the y-axis, and then multiply the 3D point cloud data according to the set multiple range (preferably times) and finally zoom along the set range (preferably Random rotation of
[0078] For the S3DIS dataset, the height of the 3D point cloud data in the dataset is calculated according to the y-axis, and then the input color information is randomly discarded, and finally the color automatic contrast alignment is performed;
[0079] S3. Based on the concept of physical electrostatic field, source charges are set at the vertices of the minimum bounding box of the 3D point cloud dataset, and probe point charges of each point in the 3D point cloud dataset are constructed based on the set source charges (e.g. Figure 2 shown);
[0080] The step of setting the source charge at the vertex of the minimum bounding box of the three-dimensional point cloud data set comprises the following steps:
[0081] Obtain the minimum bounding box of the preprocessed three-dimensional point cloud data obtained in step S2;
[0082] Setting source charges at the vertices of the minimum bounding box of the 3D point cloud dataset: To construct the source charges required for point cloud data analysis, we first deeply explore the properties of source charges. In an electrostatic field, the position of the source charge relative to the observer is fixed, and the electric field intensity E generated by the source charge remains unchanged as the probe point charge changes. Based on these observations, we hypothesize that a set of virtual source charges is constructed for the point cloud data. These source charges should meet the following requirements:
[0083] Requirement 1: Once the input point cloud is given, the position and charge of the source charge are fixed values;
[0084] Requirement 2: The source charge can encode the structural information of the 3D point cloud data according to the set requirements;
[0085] Requirement 3: The electric field strength generated by the source charge must be robust to local noise within the 3D point cloud data;
[0086] The specific implementation includes the following steps:
[0087] Get the minimum bounding box of the preprocessed 3D point cloud data obtained in step S2:
[0088] In the pre-processed 3D point cloud data obtained in step S2, the X-axis maximum value of each point in the 3D point cloud data is obtained. , Y-axis maximum value , Z-axis maximum value , X-axis minimum value , Y-axis minimum value and Z-axis minimum ;
[0089] Get the 8 vertices of the minimum bounding box, which are the first vertex , second vertex , the third vertex , the fourth vertex , the fifth vertex , the sixth vertex , the seventh vertex and the eighth vertex ;
[0090] Set the source charges at the vertices of the minimum bounding box of a 3D point cloud dataset:
[0091] Set a source charge at each of the 8 vertices of the minimum bounding box , s is the label of the source charge and its value ranges from 1 to 8; the motivation of this setting is that the structural information of the cloud point can be encoded from multiple directions, and it also meets the above requirement 1;
[0092] Theoretically, each source charge can generate its own electric field strength ; However, since the bounding box has eight vertices Symmetry, all 8 The generated joint electric field strength will gradually decrease from the boundary to the center, and the value at the center is close to 0; this will greatly weaken its characterization performance of the model; in order to avoid this, for the minimum bounding box, the minimum bounding box is divided into 8 sub-boxes along the midpoint of each edge, and the sth sub-box is set to include only the source charge The generated field strength ;
[0093] Divide the 8 vertices of the minimum bounding box into two groups, namely the first group and the second set (like Figure 3 As shown), the first set and the second set are located on opposite sides of the bounding box of the minimum bounding box, and the starting point and share the same side of the bounding box;
[0094] The first set and the second set form two directional rings in sequence, and mark the vertices according to the order of each vertex in the directional ring;
[0095] For each vertex in each set , the vertex The next vertex in the corresponding direction ring Connect to get the first vector for ; The first vector and the center point of the minimum bounding box Connect to get the second vector for ; Calculate the source charge for ;
[0096] It is very robust to local noise. First, when the point cloud P is given, the source charge is fixed; each source charge The calculation depends on the vertex , and the center point of the point cloud ; For a given input point cloud P, they are all fixed; secondly, if the local noise Do not make any point in the input point cloud If the minimum bounding box is exceeded, any The value of remains unchanged; therefore, we can say It is robust to local noise; in addition, Has geometric structure properties; in fact, 8 source charges The position and charge of the point cloud encode the different spatial direction information of the point cloud; each input point cloud can be To perceive its relative position in the entire shape; these structural information will help to better characterize the features of the point;
[0097] In addition, the construction of the probe point charge of each point in the three-dimensional point cloud data set based on the set source charge specifically includes the following steps:
[0098] Point cloud data is composed of points. The volume of each vertex does not need to be considered. It is reasonable to regard each vertex as a probe point charge placed in the electrostatic field generated by the source charge constructed above. Therefore, each point in the three-dimensional point cloud data set is regarded as a probe point charge placed in the electrostatic field generated by the source charge.
[0099] It is assumed that the probe point charges have no influence on each other, and the smaller the distance (preferably Euclidean distance) between the probe point charges, the higher the similarity;
[0100] For the 8 vertices of the minimum bounding box , for the i-th point in the s-th subframe ,Will and Concatenate to get the third vector for ;Will and Connect to get the fourth vector for ,in Vertex The previous node in the corresponding direction loop; calculate the probe point charge for ;
[0101] It has geometric characteristics and is robust to local noise; it is easy to prove that the constructed Robust to noise; on the one hand, the point of The calculation depends only on , bounding box and The two vertices of , without involving other vertices in the point cloud; therefore, even if other points in the neighborhood are changed by local noise, remains unchanged; on the other hand, when It is itself affected by local noise When the interference is ,in express The neighborhood radius, represents the Euclidean distance between two points, then There will be no significant change; this is because, in the same subframe s, the point and is fixed, allowing points A small movement of the two vectors will only and and their cross product bring little change; thus, it is confirmed that Robust to local noise;
[0102] S4. Based on the source charge and probe point charge set in step S3 and the physical Coulomb's law, calculate the point geometric Coulomb force for each point in the three-dimensional point cloud dataset; specifically, the steps include:
[0103] The following formula is used to calculate the center point of the three-dimensional point cloud dataset: Coulomb force at point geometry : Where k is the electrostatic force constant; r is the source charge with dot The distance between (preferably Euclidean distance);
[0104] PGCF is an efficient and robust explicit geometric descriptor; PGCF has many desirable geometric properties: First, it allows encoding the underlying geometric structure of the point cloud because it considers the multi-directional relative position relationship of the points in space. If the points are located in different sub-boxes, then different source charges The resulting change in field E ensures that their PGCF values are also different; however, if they are in the same subframe, since each vertex The PGCF value of depends only on the three-dimensional coordinates of the vertices. When the vertices belong to the same neighborhood, their coordinate values are similar, which leads to similar PGCF values. and can be calculated explicitly and are robust to local noise. Therefore, according to the above formula, PGCF is also an explicit geometric embedding and is robust to local noise. Finally, the calculation of PGCF only involves The computational complexity is , where n is the number of vertices of the input point cloud model, and its calculation process is efficient;
[0105] S5. The point geometry Coulomb force of each point obtained in step S4 is combined with the corresponding three-dimensional coordinates to obtain a training data set, and the deep learning network is trained to obtain a feature extraction network;
[0106] The deep learning network specifically includes the following steps:
[0107] The deep learning network comprises eleven layers connected in series;
[0108] The first layer consists of a one-dimensional convolutional layer;
[0109] The second layer consists of two one-dimensional convolutional layers, two batch normalization layers, and an activation function layer connected in series;
[0110] The third layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0111] The fourth layer consists of three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0112] The fifth layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0113] The sixth layer consists of three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0114] The seventh layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0115] The eighth layer consists of three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0116] The ninth layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0117] The tenth layer consists of two one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series;
[0118] The eleventh layer consists of three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series;
[0119] The deep learning network is used to extract feature information of the input three-dimensional point cloud data layer by layer;
[0120] S6. Using the feature extraction network obtained in step S5, feature extraction of the target three-dimensional point cloud data is completed; specifically comprising the following steps:
[0121] For the target 3D point cloud data, preprocessing is first performed;
[0122] For the pre-processed target 3D point cloud data, set the corresponding source charge and probe point charge, and calculate the point geometry Coulomb force;
[0123] Merge the point geometry Coulomb force with the corresponding 3D coordinates in the target 3D point cloud data;
[0124] The merged data is input into the feature extraction network obtained in step S5, and the output of the feature extraction network is the features of the extracted target three-dimensional point cloud data.
[0125] The feature extraction method of the present invention is further described below with reference to an embodiment:
[0126] This example uses Ubuntu 20.04 x64 with CUDA 11.3 and Python 3.9, and an NVIDIA GeForce GTX 4090 GPU (24GB). In the experiment, this method uses 3D point cloud coordinates as input, and the Adam optimizer has an initial learning rate of 0.001 for all training.
[0127] Corresponding performance: For classification tasks, this method uses the real scan dataset ScanObjectNN; for segmentation tasks, this method uses the S3DIS dataset.
[0128] Table 1 shows the comparison results of classification on the ScanObjectNN dataset:
[0129] Table 1 Schematic table of classification results
[0130]
[0131] In Table 1, the indicator OA is the percentage indicator of the overall model classification accuracy, and mAcc is the percentage indicator of the average classification accuracy of each class; PointNet is the classification method proposed by Charles R Qi at the Conference on Computer Vision and Pattern Recognition in 2017; PointNet++ is the classification method proposed by Charles R Qi at the Neural Information Processing Systems in 2017; RepSurf-U is the classification method proposed by HaoxiRan at the Conference on Computer Vision and Pattern Recognition in 2022; PointNeXt-S is the classification method proposed by Guocheng Qian at the Neural Information Processing Systems in 2022; PointMetaBase-S is the classification method proposed by Haojia Lin at the Conference on Computer Vision and Pattern Recognition in 2023;
[0132] As can be seen from Table 1, the classification method based on the feature extraction method of the present invention has better performance than the existing solution, which shows that the feature extraction method of the present invention can achieve better results in the classification task;
[0133] Table 2 shows the comparison results of segmentation on the S3DIS dataset:
[0134] Table 2 Schematic diagram of segmentation results
[0135]
[0136] In Table 2, mIoU is the percentage indicator of average segmentation accuracy; KPConv is the classification method proposed by Hugues Thomas at the International Conference on Computer Vision in 2019; PointNeXt-L is the classification method proposed by Guocheng Qian at Neural Information Processing Systems in 2022; PointMetaBase-L is the classification method proposed by Haojia Lin at the Conference on Computer Vision and Pattern Recognition in 2023;
[0137] As can be seen from Table 2, the segmentation method based on the feature extraction method of the present invention has better performance than the existing solution, indicating that the feature extraction method of the present invention can also achieve good results in segmentation tasks;
[0138] like Figure 4 The figure shows a qualitative diagram of the segmentation results of the segmentation method of the present invention and the comparison scheme on the S3DIS dataset; wherein the comparison scheme is the PointMetaBase scheme, which is a classification method proposed by Haojia Lin at the Conference on Computer Vision and Pattern Recognition in 2023;
[0139] Figure 4 The circled position in FIG shows the part where the PointMetaBase solution segmented incorrectly but the segmentation solution based on the feature extraction method of the present invention segmented correctly; Figure 4 It can be seen that the feature extraction method of the present invention can indeed achieve good results in segmentation tasks.
[0140] Finally, the feature extraction method of the present invention is particularly suitable for the field of autonomous driving. When applied to this field, the steps of the feature extraction method of the present invention are specifically as follows:
[0141] S1. Obtain input 3D point cloud data; specifically, the steps include:
[0142] On vehicles equipped with autonomous driving systems, the surrounding environment is scanned using devices such as lidar to obtain point cloud scene data and mark it;
[0143] S2 preprocesses the three-dimensional point cloud dataset obtained in step S1;
[0144] S3. Based on the concept of physical electrostatic fields, source charges are set at the vertices of the minimum bounding box of the 3D point cloud dataset. Probe point charges are then constructed for each point in the 3D point cloud dataset based on the set source charges.
[0145] S4. Based on the source charge and probe point charge set in step S3 and the physical Coulomb's law, the point geometric Coulomb force of each point in the three-dimensional point cloud dataset is calculated;
[0146] S5. The point geometry Coulomb force of each point obtained in step S4 is combined with the corresponding three-dimensional coordinates to obtain a training data set, and the deep learning network is trained to obtain a feature extraction network;
[0147] S6. Using the feature extraction network obtained in step S5, feature extraction of the target three-dimensional point cloud data is completed;
[0148] S7. The features of the three-dimensional point cloud data obtained in step S6 are fed into the segmenter to complete the segmentation, and then the segmented target is fed into the classifier for classification to complete the recognition of the three-dimensional point cloud scene data;
[0149] At the same time, the feature extraction scheme of the present invention is also applicable to industrial parts recognition, robot vision, virtual display and restoration of cultural relics, high-precision map construction, real-time positioning, path planning, terrain surveying and mapping, forest monitoring, simulation digital scene modeling, virtual reality (VR) and augmented reality (AR) and other fields.
[0150] like Figure 5 The figure shows a flow chart of the classification and segmentation method of the present invention: The classification and segmentation method disclosed in the present invention, which includes the feature extraction method of the three-dimensional point cloud data, comprises the following steps:
[0151] S1. Obtain a known 3D point cloud dataset;
[0152] S2 preprocesses the three-dimensional point cloud dataset obtained in step S1;
[0153] S3. Based on the concept of physical electrostatic fields, source charges are set at the vertices of the minimum bounding box of the 3D point cloud dataset. Probe point charges are then constructed for each point in the 3D point cloud dataset based on the set source charges.
[0154] S4. Based on the source charge and probe point charge set in step S3 and the physical Coulomb's law, the point geometric Coulomb force of each point in the three-dimensional point cloud dataset is calculated;
[0155] S5. The point geometry Coulomb force of each point obtained in step S4 is combined with the corresponding three-dimensional coordinates to obtain a training data set, and the deep learning network is trained to obtain a feature extraction network;
[0156] S6. Using the feature extraction network obtained in step S5, feature extraction of the target three-dimensional point cloud data is completed;
[0157] S7. Determine the classification and segmentation requirements for the target 3D point cloud data;
[0158] S8. If the classification requirement is met, the features of the target three-dimensional point cloud data extracted in step S6 are input into the corresponding classification network to complete the classification of the target three-dimensional point cloud data;
[0159] If segmentation is required, the features of the target three-dimensional point cloud data extracted in step S6 are input into the corresponding segmentation network to complete the segmentation of the target three-dimensional point cloud data.
[0160] like Figure 6 The figure shows a schematic diagram of the functional modules of the system of the present invention: the system disclosed in the present invention for realizing the feature extraction method of the three-dimensional point cloud data includes a data acquisition module, a data processing module, a charge setting module, a Coulomb force calculation module, a network training module and a feature extraction module; the data acquisition module, the data processing module, the charge setting module, the Coulomb force calculation module, the network training module and the feature extraction module are connected in series in sequence; the data acquisition module is used to acquire a known three-dimensional point cloud data set and upload the data information to the data processing module; the data processing module is used to pre-process the acquired three-dimensional point cloud data set according to the received data information and upload the data information to the charge setting module; the charge setting module is used to, based on the concept of physical electrostatic field, pre-process the minimum bounding box of the three-dimensional point cloud data set according to the received data information. The source charge is set at the vertex of the , and the probe point charge of each point in the three-dimensional point cloud dataset is constructed based on the set source charge, and the data information is uploaded to the Coulomb force calculation module; the Coulomb force calculation module is used to calculate the point geometry Coulomb force of each point in the three-dimensional point cloud dataset based on the set source charge and probe point charge and the physical Coulomb law according to the received data information, and upload the data information to the network training module; the network training module is used to merge the obtained point geometry Coulomb force of each point with the corresponding three-dimensional coordinates according to the received data information to obtain a training dataset, and train the deep learning network to obtain a feature extraction network, and upload the data information to the feature extraction module; the feature extraction module is used to complete the feature extraction of the target three-dimensional point cloud data using the obtained feature extraction network according to the received data information.
Claims
1. A feature extraction method for three-dimensional point cloud data, characterized in that The steps include: S1. Obtain a known 3D point cloud dataset; specifically, the steps include: Acquire a known three-dimensional point cloud dataset; the three-dimensional point cloud dataset includes a ScanObjectNN dataset and an S3DIS dataset; S2. Preprocessing the three-dimensional point cloud data set obtained in step S1; specifically comprising the following steps: For the ScanObjectNN dataset, the 3D point cloud data in the dataset is randomly sampled to a set number of points, and then the height of the 3D point cloud data is calculated according to the y-axis. The 3D point cloud data is then scaled according to a set multiple range, and finally randomly rotated along the set range. For the S3DIS dataset, the height of the 3D point cloud data in the dataset is calculated according to the y-axis, and then the input color information is randomly discarded, and finally the color automatic contrast alignment is performed; S3. Based on the concept of physical electrostatic field, source charges are set at the vertices of the minimum bounding box of the three-dimensional point cloud data set, and probe point charges of each point in the three-dimensional point cloud data set are constructed based on the set source charges; comprising the following steps: Obtain the minimum bounding box of the preprocessed three-dimensional point cloud data obtained in step S2; The source charge is set at the vertices of the minimum bounding box of the 3D point cloud dataset, and the set source charge meets the following requirements: Requirement 1: The position and charge amount of the source charge are fixed values; Requirement 2: The source charge can encode the structural information of the 3D point cloud data according to the set requirements; Requirement 3: The electric field strength generated by the source charge must be robust to local noise in the 3D point cloud data; S4. Based on the source charge and probe point charge set in step S3 and the physical Coulomb's law, the point geometry Coulomb force of each point in the three-dimensional point cloud data set is calculated; S5. The point geometry Coulomb force of each point obtained in step S4 is combined with the corresponding three-dimensional coordinates to obtain a training data set, and the deep learning network is trained to obtain a feature extraction network; S6. Using the feature extraction network obtained in step S5, complete the feature extraction of the target three-dimensional point cloud data.
2. The feature extraction method of three-dimensional point cloud data according to claim 1, characterized in that The step of setting the source charge at the vertex of the minimum bounding box of the three-dimensional point cloud data set specifically includes the following steps: Get the minimum bounding box of the preprocessed 3D point cloud data obtained in step S2: In the preprocessed three-dimensional point cloud data obtained in step S2, the maximum value of the X axis of each point in the three-dimensional point cloud data is obtained. , Y-axis maximum value , Z axis maximum value , X-axis minimum value , Y axis minimum value and Z-axis minimum ; Get the 8 vertices of the minimum bounding box, which are the first vertex , second vertex , the third vertex , the fourth vertex , the fifth vertex , the sixth vertex , the seventh vertex and the eighth vertex ; Set the source charges at the vertices of the minimum bounding box of a 3D point cloud dataset: At each of the 8 vertices of the minimum bounding box, set a source charge , s is the label of the source charge and its value range is 1~8; For the minimum bounding box, divide the minimum bounding box into 8 sub-boxes along the midpoint of each edge, and set the sth sub-box to include only the source charge The generated field strength ; Divide the 8 vertices of the minimum bounding box into two groups, namely the first group And the second set , and the first set and the second set are located on opposite sides of the bounding box of the minimum bounding box, the starting point and share the same side of the bounding box; The first set and the second set form two direction rings in order, and mark the vertices according to the order of each vertex in the direction ring; For each vertex in each set , the vertex The next vertex in the corresponding direction ring Connect to get the first vector for ; The first vector and the center point of the minimum bounding box Connect to get the second vector for ; Calculate the source charge for .
3. The feature extraction method of three-dimensional point cloud data according to claim 2, characterized in that The step S3 of constructing the probe point charge of each point in the three-dimensional point cloud data set based on the set source charge specifically includes the following steps: Each point in the three-dimensional point cloud data set is used as a probe point charge placed in the electrostatic field generated by the source charge; It is assumed that the probe point charges have no influence on each other, and the smaller the distance between the probe point charges, the higher the similarity; For the 8 vertices of the minimum bounding box , for the i-th point in the s-th subframe ,Will and Concatenate to get the third vector for ;Will and Connect to get the fourth vector for ,in Vertex The previous node in the corresponding direction loop; Calculate the probe point charge for .
4. The feature extraction method of three-dimensional point cloud data according to claim 3, characterized in that The step S4 specifically comprises the following steps: The following formula is used to calculate the center point of the 3D point cloud dataset: Coulomb force at point geometry : Where k is the electrostatic force constant; r is the source charge With point The distance between.
5. The feature extraction method of three-dimensional point cloud data according to claim 4, characterized in that The deep learning network described in step S5 specifically includes the following steps: The deep learning network comprises eleven layers connected in series; The first layer consists of a one-dimensional convolutional layer; The second layer includes two one-dimensional convolutional layers, two batch normalization layers, and an activation function layer connected in series; The third layer includes three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series; The fourth layer includes three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series; The fifth layer includes three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series; The sixth layer includes three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series; The seventh layer includes three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series; The eighth layer includes three one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series; The ninth layer includes three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series; The tenth layer includes two one-dimensional convolutional layers, two batch normalization layers, and two activation function layers connected in series; The eleventh layer includes three one-dimensional convolutional layers, three batch normalization layers, and three activation function layers connected in series; The deep learning network is used to extract feature information of the input three-dimensional point cloud data layer by layer; The step S6 specifically includes the following steps: For the target 3D point cloud data, preprocessing is first performed; For the preprocessed target 3D point cloud data, set the corresponding source charge and probe point charge, and calculate the point geometry Coulomb force; Merge the point geometry Coulomb force with the corresponding 3D coordinates in the target 3D point cloud data; The merged data is input into the feature extraction network obtained in step S5, and the output of the feature extraction network is the features of the extracted target three-dimensional point cloud data.
6. A classification and segmentation method comprising the feature extraction method of three-dimensional point cloud data according to any one of claims 1 to 5, characterized in that The following steps are also included: S7. Determine the classification requirements and segmentation requirements of the target three-dimensional point cloud data; S8. If the classification requirement is met, the features of the target three-dimensional point cloud data extracted in step S6 are input into the corresponding classification network to complete the classification of the target three-dimensional point cloud data; If there is a segmentation requirement, the features of the target three-dimensional point cloud data extracted in step S6 are input into the corresponding segmentation network to complete the segmentation of the target three-dimensional point cloud data.
7. A system for implementing the feature extraction method of three-dimensional point cloud data according to any one of claims 1 to 5, characterized in that It includes a data acquisition module, a data processing module, a charge setting module, a Coulomb force calculation module, a network training module and a feature extraction module; the data acquisition module, the data processing module, the charge setting module, the Coulomb force calculation module, the network training module and the feature extraction module are connected in series in sequence; the data acquisition module is used to acquire a known three-dimensional point cloud data set and upload the data information to the data processing module; the data processing module is used to pre-process the acquired three-dimensional point cloud data set according to the received data information, and upload the data information to the charge setting module; The charge setting module is used to set source charges at the vertices of the minimum bounding box of the three-dimensional point cloud data set based on the received data information and the concept of physical electrostatic field, and to construct the probe point charges of each point in the three-dimensional point cloud data set based on the set source charges, and upload the data information to the Coulomb force calculation module; The Coulomb force calculation module is used to calculate the point geometry Coulomb force of each point in the three-dimensional point cloud data set according to the received data information, based on the set source charge and probe point charge and physical Coulomb's law, and upload the data information to the network training module; The network training module is used to combine the point geometry Coulomb force of each point with the corresponding three-dimensional coordinates to obtain a training data set according to the received data information, train the deep learning network to obtain a feature extraction network, and upload the data information to the feature extraction module; The feature extraction module is used to complete the feature extraction of the target three-dimensional point cloud data based on the received data information and the obtained feature extraction network.