A 3D point cloud segmentation method integrating global and local geometric features
By integrating global geometric features and local geometric features, the problem of inaccurate block boundaries in three-dimensional point cloud segmentation is solved, and higher segmentation accuracy and boundary accuracy are achieved.
Patent Information
- Application Number
- CN202411141274.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2025-04-18
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The existing three-dimensional point cloud segmentation method is difficult to accurately locate the block boundary, which often leads to oversegment or undersegment, and the block boundary is inaccurate.
The region growth method that integrates global geometric features and local geometric features is used for segmentation. By calculating the local geometric features (such as normals and curvature) of each point and determining the segmentation threshold, the accurate positioning of the block boundary is achieved, and the blocks are merged according to the global geometric features.
The accuracy of point cloud data segmentation is improved, oversegment and undersegment are avoided, and the accuracy of block boundaries is ensured, providing reliable data for further computing of point clouds.
Smart Images
Figure CN119107326B_ABST
Abstract
Description
[0001] Technology Neighborhood
[0002] The present invention relates to the field of computer vision technology, and in particular to a three-dimensional point cloud segmentation method integrating global geometric features and local geometric features. Background Art
[0003] 3D point cloud data processing is a hot topic in the field of computer vision. Point cloud segmentation is one of the basic tasks in 3D point cloud data processing, and its purpose is to divide the point cloud into subsets with similar space and attributes. The accuracy of point cloud segmentation plays a decisive role in the efficiency and accuracy of further calculations (such as comparison, recognition, 3D reconstruction, geometric measurement, etc.). Note that point cloud segmentation is different from point cloud semantic segmentation. The former is a clustering problem, while the latter is a classification problem, which requires sufficient samples.
[0004] In the prior art, the basic algorithms for point cloud segmentation are mainly the extensions of image segmentation algorithms in three-dimensional space, such as region growing, clustering, min-cut, edge detection, etc. Since point clouds are irregular point sets, and images are regular grid point sets, there are often many difficulties in extending image algorithms on point clouds. For point cloud segmentation, the main problem with existing segmentation methods is that it is difficult to accurately locate the boundaries of the blocks. Usually, if the segmentation parameters are strict, over-segmentation is prone to occur (a large number of small blocks are obtained), and if the segmentation parameters are loose, under-segmentation may occur (there are points with different attributes in large blocks, such as perpendicular faces in blocks). If the parameters are moderate, over-segmentation and under-segmentation will occur in different local areas at the same time. Summary of the invention
[0005] The primary purpose of the present invention is to overcome the problems existing in the prior art and to provide a three-dimensional point cloud segmentation method that integrates global geometric features and local geometric features. The present invention can improve the accuracy of point cloud data segmentation.
[0006] In order to achieve the above object, the present invention provides a 3D point cloud segmentation method integrating global geometric features and local geometric features, comprising:
[0007] Obtain three-dimensional point cloud data of the target scene;
[0008] Calculate local geometric features of each point in the three-dimensional point cloud, wherein the local geometric features include a normal and a curvature;
[0009] Determine a segmentation threshold, wherein the segmentation threshold includes a surface spacing threshold, a normal angle threshold, and a curvature threshold;
[0010] Determine the number of blocks, segment the three-dimensional point cloud according to the number of blocks, the local geometric features and the segmentation threshold using a region growing method that fuses global geometric features and local geometric features, obtain multiple three-dimensional point cloud blocks, and obtain global geometric features of the multiple three-dimensional point cloud blocks, wherein the global geometric features are the center point and orientation of each block, wherein the region growing method that fuses global geometric features and local geometric features includes: (1) selecting points that do not belong to other blocks from the three-dimensional point cloud or selecting points that do not belong to other blocks and have the smallest curvature from the three-dimensional point cloud, and putting the selected points into an initial seed point set;
[0011] (2) Select a seed point from the seed point set and delete the seed point after selection to determine the neighborhood point set of the seed point; (3) Determine whether each neighborhood point in the neighborhood point set is a boundary point one by one. If it is a boundary point, do not process the neighborhood point; otherwise, add the neighborhood point to the seed point set to obtain a new seed point set, mark the neighborhood point as belonging to the current block, and calculate the center point and orientation of the current block; (4) Continue to perform the above operations on the new seed point set until the new seed point set is empty, and obtain the final center point and orientation of the current block;
[0012] Merging the multiple three-dimensional point cloud blocks according to the global geometric features to obtain multiple three-dimensional point cloud blocks;
[0013] The multiple three-dimensional point cloud blocks are renumbered, and it is determined whether the segmentation result meets the segmentation condition based on the multiple three-dimensional point cloud blocks. If so, the segmentation is stopped; otherwise, the region growing method that integrates global geometric features and local geometric features is used to continue segmenting the remaining points in the three-dimensional point cloud after removing the multiple three-dimensional point cloud blocks until the segmentation condition is met and the segmentation is completed.
[0014] Furthermore, the method of determining the neighborhood point set of the seed point is as follows:
[0015] |Xo-X|<
[0016] Among them, Xo is a point in the three-dimensional point cloud, X is a neighborhood point, The preset neighborhood range.
[0017] Furthermore, a neighborhood point is a boundary point if at least one of the following conditions is met:
[0018] The neighborhood points belong to other blocks;
[0019] The distance between the neighborhood point and the surface where the seed point is located is greater than the surface distance threshold;
[0020] The angle between the normal of the neighborhood point and the normal of the current block is greater than the normal angle threshold;
[0021] The curvature of the neighborhood point is greater than the curvature threshold.
[0022] Furthermore, the distance from the neighborhood point to the surface where the seed point is located is calculated as follows:
[0023]
[0024] in, is the distance from the neighborhood point to the surface where the seed point is located.
[0025] Furthermore, the angle between the normal line of the neighborhood point and the normal line of the current block is calculated as follows:
[0026]
[0027] in, is the normal angle, n(x) is the normal of the neighborhood, is the normal of block k.
[0028] Furthermore, the method of calculating the center point of the block is as follows:
[0029]
[0030] in, is the set of all points in the block, Yes The weight of the point cloud is If the point cloud is evenly distributed, then ;
[0031] The orientation of a block is calculated as follows:
[0032]
[0033] Perform singular value decomposition on ∑:
[0034]
[0035] in is a diagonal matrix consisting of the eigenvalues of ∑, and is an orthogonal matrix, and the minimum eigenvalue corresponds to The column vector of is the orientation of the block.
[0036] Furthermore, the multiple three-dimensional point cloud blocks are merged according to the global geometric features to obtain multiple three-dimensional point cloud blocks, as follows:
[0037] Initialize the number of the new block to which each block belongs;
[0038] The similarity between each two blocks in the plurality of three-dimensional point cloud blocks is determined according to the similarity condition. If
[0039] If two blocks are similar, the two blocks are merged, otherwise they are not merged, wherein the similarity condition is that the orientation angle and the spacing of the two blocks are both less than a preset threshold, wherein the calculation formula of the orientation angle is as follows:
[0040]
[0041] The spacing calculation formula is as follows:
[0042] ;
[0043] The above operation is repeated until every two blocks in the multiple three-dimensional point cloud blocks are compared.
[0044] Further, merging the two blocks specifically includes:
[0045] Set the new block number;
[0046] Mark the points in two similar blocks with the same mark to obtain a new block;
[0047] The center point and orientation of the new block are calculated.
[0048] Furthermore, the segmentation condition is that the number of merged blocks reaches a requirement and / or the ratio of the number of points of all merged blocks to the number of points of the three-dimensional point cloud is greater than a set value.
[0049] Compared with the prior art, the present invention has the following beneficial effects:
[0050] The present invention utilizes the differences in local geometric features such as surface spacing and curvature between each point in the point cloud and its adjacent points to accurately locate the block boundaries, and then merges the blocks according to the similarity of the global geometric features of the blocks to avoid the appearance of a large number of small blocks. This solves the problems of over-segmentation and under-segmentation and inaccurate block boundaries in point cloud segmentation in the prior art, improves the accuracy of point cloud segmentation, and provides reliable and effective data for further calculation of point clouds. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 is a flow chart of a three-dimensional point cloud segmentation method integrating global geometric features and local geometric features according to an embodiment of the present invention;
[0052] Figure 2 It is a block effect diagram of an embodiment of the present invention;
[0053] Figure 3This is a detailed diagram of the division of a cabinet placed on the ground according to an embodiment of the present invention;
[0054] Figure 4 is a detailed diagram of the division of the corner of the ceiling and the wall according to an embodiment of the present invention;
[0055] Figure 5 is a point cloud of a block numbered 5 in an embodiment of the present invention;
[0056] Figure 6 It is a geometric feature graph of the block numbered 5 in an embodiment of the present invention. DETAILED DESCRIPTION
[0057] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0058] In the description of the present invention, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.
[0059] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0060] Furthermore, in the description of the present invention, unless otherwise specified, “plurality” means two or more.
[0061] Example 1
[0062] like Figure 1-6 As shown, a three-dimensional point cloud segmentation method integrating global geometric features and local geometric features in a preferred embodiment of the present invention includes:
[0063] S1: Obtain 3D point cloud data of the target scene;
[0064] S2: Calculate the local geometric features of each point in the three-dimensional point cloud. The local geometric features include normal vectors and curvatures.
[0065] In one embodiment, the method for calculating geometric features point by point is surface model modeling (https: / / github.com / SCVision / PointSurfaces). The functions of PCL (https: / / pointclouds.org / ) or other algorithms can also be used.
[0066] S3: Determine the segmentation thresholds, which include surface distance threshold, normal vector angle threshold, and curvature threshold.
[0067] In one embodiment, the surface distance threshold , the normal vector angle threshold , and the curvature threshold .
[0068] S4: Determine the number of blocks. According to the number of blocks, geometric features, and segmentation thresholds, use the region growing method that fuses global geometric features and local geometric features to segment the three-dimensional point cloud, and obtain multiple three-dimensional point cloud blocks.
[0069] In one embodiment, select the number of blocks K and perform K times of point cloud segmentation. For the k-th (0 < k ≤ K) point cloud segmentation, use the region growing method for segmentation to obtain the block numbered k. In this embodiment, select K = 100.
[0070] It should be noted that the steps of the region growing method are as follows:
[0071] S4.1: Arbitrarily select a point from the three-dimensional point cloud that does not belong to other blocks or a point with the minimum curvature that does not belong to other blocks from the three-dimensional point cloud, and put the selected point into the initial seed point set; for example, select a point from the point cloud that does not belong to a certain block as the seed point , add it to the seed set O and mark it as belonging to block k. Initialize the center point and orientation
[0072] S4.2: Arbitrarily select a seed point from the seed point set and delete the seed point after selection, and determine the set of neighborhood points of the seed point; for example, extract a point from the seed set O (delete this point in O).
[0073] S4.3: Judging one by one whether each neighborhood point in the set of neighborhood points is a boundary point. If it is a boundary point, do not perform any processing on the neighborhood point. Otherwise, add the neighborhood point to the seed point set to obtain a new seed point set, mark the neighborhood point as belonging to the current block, and calculate the center point and orientation of the current block.
[0074] In one embodiment, a neighborhood point is a boundary point if at least one of the following conditions is met:
[0075] The neighborhood point x belongs to other blocks;
[0076] The distance from the neighborhood point x to the surface where the seed point is located Greater than the surface spacing threshold ,in,
[0077]
[0078] The angle between the normal of the neighborhood point x and the normal of the current block Greater than the normal angle threshold ,
[0079] in,
[0080]
[0081] The curvature of the neighborhood point x | (x)| is greater than the curvature threshold .
[0082] It should be noted that the method for calculating the center point of the block is as follows:
[0083]
[0084] in, is the set of all points in the block, Yes The weight of the point cloud is If the point cloud is evenly distributed, then ;
[0085] The orientation of a block is calculated as follows:
[0086]
[0087] Perform singular value decomposition on ∑:
[0088]
[0089] in is a diagonal matrix consisting of the eigenvalues of ∑, and is an orthogonal matrix, and the minimum eigenvalue corresponds to The column vector of is the orientation of the block.
[0090] S4.4: Continue to perform the above operations on the new seed point set until the new seed point set is empty, obtain the final center point and orientation of the current block, record the center and orientation of block k, go to S4.1, and end when k>K. At this time, the center and orientation of blocks 1 to K are obtained.
[0091] S5: calculating the global geometric features of multiple 3D point cloud blocks, and merging the multiple 3D point cloud blocks according to the global geometric features to obtain multiple 3D point cloud blocks;
[0092] In one embodiment, S5 specifically includes:
[0093] S5.1: Initialize the number of the new block to which each block belongs;
[0094] In one embodiment, the number S(k)=k of the new block to which each block k belongs is initialized.
[0095] The similarity between each two blocks in the plurality of three-dimensional point cloud blocks is determined according to the similarity condition. If
[0096] If two blocks are similar, the two blocks are merged, otherwise they are not merged, wherein the similarity condition is that the orientation angle and the distance between the two blocks are both less than a preset threshold;
[0097] In one embodiment, if the partitions p and q are similar, they are merged as follows:
[0098] Set the new block number S(q)=S(p);
[0099] Mark the points in two similar blocks with the same label to obtain a new block. For example, traverse the point cloud and mark the point cloud belonging to block q as belonging to S(q);
[0100] Calculate the center point and orientation of the new block and estimate the block S(q) (S(q)) center point and orientation (S(q)). The detailed estimation method is shown in S4.3.
[0101] The above operation is repeated until every two blocks in the multiple three-dimensional point cloud blocks are compared.
[0102] It should be noted that the orientation angle between the two blocks is calculated by the following formula:
[0103]
[0104] is the maximum angle between the orientations of similar blocks;
[0105] The distance between two blocks is calculated as follows:
[0106]
[0107] is the maximum distance between similar blocks, , These are preset thresholds.
[0108] S6: Renumber the multiple 3D point cloud blocks, and determine whether the segmentation result meets the segmentation condition based on the multiple 3D point cloud blocks. If so, stop segmenting. Otherwise, use the region growing method that integrates global geometric features and local geometric features to continue segmenting the remaining points in the 3D point cloud after removing the multiple 3D point cloud blocks until the segmentation condition is met and the segmentation is completed.
[0109] In one embodiment, the segmentation condition is that the number of merged blocks meets the requirement and / or the ratio of the number of points of all merged blocks to the number of points of the three-dimensional point cloud is greater than a set value.
[0110] In this embodiment, the number of blocks after the first segmentation and merging is M=74, and the number of blocks after the second segmentation and merging is M=79. The ratio of the number of points in all blocks to the number of points in the entire point cloud is 0.902, and the segmentation is ended. Figure 2 shown.
[0111] Figure 3 A detailed diagram of the cabinet placed on the ground in this embodiment is provided, and the boundaries between the two faces and between them and the ground are accurate; Figure 4 The red circle in the middle is the angle formed by the ceiling and the wall. The boundaries between the ceiling and the wall, as well as between the walls, are accurate.
[0112] Figure 5 is a point cloud of a block numbered 5 in an embodiment of the present invention; Figure 6 It is a geometric feature map of the block numbered 5 in the embodiment of the present invention. The block is the wall outside the room scanned from the door and window of the room, which is merged into a whole.
[0113] Example 2
[0114] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a three-dimensional point cloud segmentation method for fusing global geometric features and local geometric features as in Example 1 is implemented.
[0115] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are all well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.
[0116] In a specific embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the three-dimensional point cloud segmentation method for fusing global geometric features and local geometric features as described in Example 1 is implemented.
[0117] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0118] In summary, the embodiments of the present invention provide a three-dimensional point cloud segmentation method that integrates global geometric features and local geometric features. It uses the differences in local geometric features such as surface spacing and curvature between each point in the point cloud and its adjacent points to accurately locate the block boundaries, and then merges the blocks according to the similarity of the global geometric features of the blocks to avoid the appearance of a large number of small blocks. It solves the problems of over-segmentation and under-segmentation and inaccurate block boundaries in the prior art of point cloud segmentation, improves the accuracy of point cloud segmentation, and provides reliable and effective data for further calculation of the point cloud.
[0119] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention. These improvements and substitutions should also be regarded as the scope of protection of the present invention.
Claims
1. A 3D point cloud segmentation method integrating global geometric features and local geometric features, characterized in that: The method comprises the following steps: Obtain three-dimensional point cloud data of the target scene; Calculate local geometric features of each point in the three-dimensional point cloud, wherein the local geometric features include a normal and a curvature; Determine a segmentation threshold, wherein the segmentation threshold includes a surface spacing threshold, a normal angle threshold, and a curvature threshold; Determine the number of blocks, segment the three-dimensional point cloud according to the number of blocks, the local geometric features and the segmentation threshold using a region growing method that fuses global geometric features and local geometric features, obtain multiple three-dimensional point cloud blocks, and obtain global geometric features of the multiple three-dimensional point cloud blocks, wherein the global geometric features are the center point and orientation of each block, wherein the region growing method that fuses global geometric features and local geometric features includes: (1) selecting points that do not belong to other blocks from the three-dimensional point cloud or selecting points that do not belong to other blocks and have the smallest curvature from the three-dimensional point cloud, and putting the selected points into an initial seed point set; (2) Select a seed point from the seed point set and delete the seed point after selection to determine the neighborhood point set of the seed point; (3) Determine whether each neighborhood point in the neighborhood point set is a boundary point one by one. If it is a boundary point, do not process the neighborhood point; otherwise, add the neighborhood point to the seed point set to obtain a new seed point set, mark the neighborhood point as belonging to the current block, and calculate the center point and orientation of the current block; (4) Continue to perform the above operations on the new seed point set until the new seed point set is empty, and obtain the final center point and orientation of the current block; Merging the multiple three-dimensional point cloud blocks according to the global geometric features to obtain multiple three-dimensional point cloud blocks; The multiple three-dimensional point cloud blocks are renumbered, and it is determined whether the segmentation result meets the segmentation condition based on the multiple three-dimensional point cloud blocks. If so, the segmentation is stopped; otherwise, the region growing method that integrates global geometric features and local geometric features is used to continue segmenting the remaining points in the three-dimensional point cloud after removing the multiple three-dimensional point cloud blocks until the segmentation condition is met and the segmentation is completed.
2. The three-dimensional point cloud segmentation method integrating global geometric features and local geometric features according to claim 1, characterized in that: The method of determining the neighborhood point set of the seed point is as follows: |Xo-X|< Among them, Xo is a point in the three-dimensional point cloud, X is a neighborhood point, The preset neighborhood range.
3. The three-dimensional point cloud segmentation method integrating global geometric features and local geometric features according to claim 2, characterized in that: A neighborhood point is a boundary point if at least one of the following conditions is met: The neighborhood points belong to other blocks; The distance between the neighborhood point and the surface where the seed point is located is greater than the surface distance threshold; The angle between the normal of the neighborhood point and the normal of the current block is greater than the normal angle threshold; The curvature of the neighborhood point is greater than the curvature threshold.
4. The three-dimensional point cloud segmentation method integrating global geometric features and local geometric features according to claim 3, characterized in that: The distance from the neighborhood point to the surface where the seed point is located is calculated as follows: in, is the distance from the neighborhood point to the surface where the seed point is located.
5. The three-dimensional point cloud segmentation method integrating global geometric features and local geometric features according to claim 4, characterized in that: The calculation formula for the angle between the normal line of the neighborhood point and the normal line of the current block is as follows: in, is the normal angle, n(x) is the normal of the neighborhood, is the normal of block k.
6. The three-dimensional point cloud segmentation method integrating global geometric features and local geometric features according to claim 1, characterized in that: The way to calculate the center point of the block is as follows: in, is the set of all points in the block, Yes The weight of the point cloud is If the point cloud is evenly distributed, then ; The orientation of a block is calculated as follows: Perform singular value decomposition on ∑: in is a diagonal matrix consisting of the eigenvalues of ∑, and is an orthogonal matrix, and the minimum eigenvalue corresponds to The column vector of is the orientation of the block.
7. A three-dimensional point system integrating global geometric features and local geometric features according to claim 1 A cloud segmentation method, characterized in that: The multiple three-dimensional point cloud blocks are merged according to the global geometric features to obtain multiple three-dimensional point cloud blocks, as follows: Initialize the number of the new block to which each block belongs; The similarity between each two blocks in the plurality of three-dimensional point cloud blocks is determined according to the similarity condition. If If two blocks are similar, the two blocks are merged, otherwise they are not merged, wherein the similarity condition is that the orientation angle and the spacing of the two blocks are both less than a preset threshold, wherein the calculation formula of the orientation angle is as follows: The spacing calculation formula is as follows: ； The above operation is repeated until every two blocks in the multiple three-dimensional point cloud blocks are compared.
8. A three-dimensional point system integrating global geometric features and local geometric features according to claim 7 A cloud segmentation method, characterized in that: The two blocks are merged, specifically comprising: Set the new block number; Mark the points in two similar blocks with the same mark to obtain a new block; The center point and orientation of the new block are calculated.
9. The three-dimensional point cloud segmentation method integrating global geometric features and local geometric features according to claim 1, characterized in that: The segmentation condition is that the number of merged blocks reaches a requirement and / or the ratio of the number of points of all merged blocks to the number of points of the three-dimensional point cloud is greater than a set value.