Point cloud-based target object extraction method, device and electronic equipment

Abstract

The application provides a point cloud-based target object extraction method and device and electronic equipment. The original point cloud data is preprocessed, and the ground is removed from the preprocessed point cloud data. Then, the point cloud data after the ground is removed is divided into a plurality of first point clouds and a plurality of second point clouds according to a plurality of preset height ranges, each first point cloud is projected to an XOY plane and a local point cloud extraction of a point target object is performed, then each local point cloud and the point cloud within a preset range around the local point cloud are projected to the XOY plane and classified, then each local point cloud and corresponding target points in the plurality of second point clouds are clustered into corresponding overall point clouds, and the overall point clouds whose point number and / or height range meet a first preset condition are removed. The application can alleviate the problems of low extraction rate, long time consumption, poor universality and inaccurate extraction in the existing tree and street lamp extraction technical solutions.

Description

Technical Field

[0001] This invention relates to the field of point cloud data processing technology, and in particular to a method, apparatus and electronic device for extracting target objects based on point clouds. Background Technology

[0002] Existing tree and street light extraction technologies can be mainly divided into the following four categories:

[0003] (1) Spatial clustering-based methods: This method requires removing ground, lawns, and low-lying green belts from point cloud data, and then classifying and extracting them based on the spatial characteristics of different land features. This method can segment land features into different land features such as buildings, trees, poles, and vehicles. Commonly used ground removal algorithms include least squares and RANSAC plane fitting. These algorithms are not only computationally intensive and time-consuming, but also prone to local optima, failing to completely remove ground, lawns, and low-lying green belts, resulting in unsatisfactory subsequent clustering results. At the same time, this method only uses the spatial clustering characteristics of land features and cannot accurately distinguish similar land features such as streetlights and trees.

[0004] (2) Geometric Feature-Based Methods: Based on the pole-shaped features of trees and streetlights, least squares and RANSAC fitting are often used to construct cylindrical models for extracting trees and streetlights. In this method, the cylindrical features of trees and streetlights only target the features at the bottom of the trees and streetlights, but there are no good geometric features for the crown and lamp head for extraction and classification. At the same time, traditional column extraction algorithms, such as least squares and RANSAC column fitting algorithms, have the problems of being time-consuming and easily leading to local optima (i.e., obtaining incorrect extraction results), which can easily result in low extraction rates and excessively long extraction times for streetlights and trees.

[0005] (3) Super-voxel-based method: Voxels are constructed from point cloud data, and trees and streetlights are extracted and classified by the contextual relationships between voxels and the point cloud features inside the voxels. However, this method relies too much on the segmentation size of the super-voxels and the selection of point cloud features inside the voxels, resulting in large differences in extraction results for different data and lacking universal applicability.

[0006] (4) Template matching method: This method requires the establishment of a structural template library for different types of trees and streetlights. The object features in the point cloud are matched with the templates in the structural template library, and the trees and streetlights are obtained based on the matching results. However, this method requires the establishment of a feature template library in advance, and it is impossible to extract trees and streetlights that do not exist in the template library, so it is not universally applicable. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a method, apparatus and electronic device for extracting target objects based on point clouds, so as to alleviate the problems of low extraction rate, long time consumption, poor universality and insufficient extraction accuracy in existing tree and street light extraction technologies.

[0008] In a first aspect, embodiments of the present invention provide a target object extraction method based on point clouds. The method includes: preprocessing raw point cloud data and performing ground removal on the preprocessed point cloud data using a hierarchical filtering method; dividing the ground-removed point cloud data into multiple first point clouds and multiple second point clouds according to multiple preset height ranges; wherein the Z coordinate value of a point in each first point cloud is less than the median of at least one preset height range; projecting each first point cloud layer onto the XOY plane and extracting point cloud clusters, and then performing columnar extraction on each extracted point cloud cluster to obtain a local point cloud of the target object; wherein the target object includes trees and / or streetlights; projecting each local point cloud and the point cloud within a preset range around it onto the XOY plane and performing classification processing to obtain a corresponding first classification; clustering the corresponding target points in each local point cloud and the multiple second point clouds into a corresponding overall point cloud based on the point intensity information and spatial location information and the first classification of each local point cloud, and removing the overall point cloud whose number and / or height range of points meet a first preset condition.

[0009] Secondly, embodiments of the present invention also provide a target object extraction device based on point clouds. The device includes: a preprocessing module for preprocessing raw point cloud data and performing ground removal on the preprocessed point cloud data using a layered filtering method; a division module for dividing the ground-removed point cloud data into multiple first point clouds and multiple second point clouds according to multiple preset height ranges; wherein the Z-coordinate value of a point in each first point cloud is less than the median of at least one preset height range; and a first acquisition module for projecting each first point cloud layer onto the XOY plane and extracting point cloud clusters, subsequently... Each extracted point cloud cluster is subjected to columnar extraction to obtain a local point cloud of the target object; wherein the target object includes trees and / or streetlights; a classification module is used to project each local point cloud and the point cloud within a preset range around it onto the XOY plane and perform classification processing to obtain a corresponding first classification; a second acquisition module is used to cluster each local point cloud and the corresponding target points in the multiple second point clouds into a corresponding overall point cloud based on the intensity information and spatial location information of the points and the first classification of each local point cloud, and to remove the overall point cloud whose number of points and / or height range meet the first preset condition.

[0010] Thirdly, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-described point cloud-based target object extraction method.

[0011] This invention provides a method, apparatus, and electronic device for extracting target objects based on point clouds. The method involves preprocessing raw point cloud data and performing ground removal on the preprocessed point cloud data using a hierarchical filtering approach. The point cloud data after ground removal is divided into multiple first point clouds and multiple second point clouds according to multiple preset height ranges. Each first point cloud is projected onto the XOY plane and point cloud clusters are extracted. Then, each extracted point cloud cluster is subjected to columnar extraction to obtain a local point cloud of the target object. Each local point cloud and the point clouds within a preset range around it are projected onto the XOY plane and classified to obtain a corresponding first classification. Based on the point intensity information, spatial location information, and the first classification of each local point cloud, the corresponding target points in each local point cloud and the multiple second point clouds are clustered into a corresponding overall point cloud. Overall point clouds whose number of points and / or height range meet a first preset condition are removed. Using the above techniques, ground removal employs layered filtering, which is simple to operate and improves the extraction efficiency of trees and streetlights compared to traditional ground removal algorithms. Projecting the point cloud onto the XOY plane for relevant extraction operations further improves the extraction rate of trees and streetlights, avoiding the situation where trees and streetlights are missed due to being in flower beds, shrubs, or on slopes. Combining the intensity information and spatial location information of points can accurately extract entangled trees and streetlights, further improving the accuracy of tree and streetlight extraction.

[0012] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0013] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0014] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0015] Figure 1This is a flowchart illustrating a point cloud-based target object extraction method according to an embodiment of the present invention.

[0016] Figure 2 This is a flowchart illustrating a point cloud-based target object extraction method according to an embodiment of the present invention.

[0017] Figure 3 This is an example diagram of large-scene point cloud data in an embodiment of the present invention;

[0018] Figure 4 This is an example diagram illustrating the effect of extracting trees in an embodiment of the present invention;

[0019] Figure 5 This is an example diagram illustrating the effect of extracting streetlights in an embodiment of the present invention;

[0020] Figure 6 This is an example diagram illustrating the extraction of tree details in an embodiment of the present invention;

[0021] Figure 7 This is an example diagram illustrating the extraction of details from streetlights in an embodiment of the present invention;

[0022] Figure 8 This is a schematic diagram of the structure of a point cloud-based target object extraction device in an embodiment of the present invention;

[0023] Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Currently, existing technologies for extracting tree and street light data can be mainly divided into the following four categories:

[0026] (1) Spatial clustering-based methods: This method requires removing ground, lawns, and low-lying green belts from point cloud data, and then classifying and extracting them based on the spatial characteristics of different land features. This method can segment land features into different land features such as buildings, trees, poles, and vehicles. Commonly used ground removal algorithms include least squares and RANSAC plane fitting. These algorithms are not only computationally intensive and time-consuming, but also prone to local optima, failing to completely remove ground, lawns, and low-lying green belts, resulting in unsatisfactory subsequent clustering results. At the same time, this method only uses the spatial clustering characteristics of land features and cannot accurately distinguish similar land features such as streetlights and trees.

[0027] (2) Geometric Feature-Based Methods: Based on the pole-shaped features of trees and streetlights, least squares and RANSAC fitting are often used to construct cylindrical models for extracting trees and streetlights. In this method, the cylindrical features of trees and streetlights only target the features at the bottom of the trees and streetlights, but there are no good geometric features for the crown and lamp head for extraction and classification. At the same time, traditional column extraction algorithms, such as least squares and RANSAC column fitting algorithms, have the problems of being time-consuming and easily leading to local optima (i.e., obtaining incorrect extraction results), which can easily result in low extraction rates and excessively long extraction times for streetlights and trees.

[0028] (3) Super-voxel-based method: Voxels are constructed from point cloud data, and trees and streetlights are extracted and classified by the contextual relationships between voxels and the point cloud features inside the voxels. However, this method relies too much on the segmentation size of the super-voxels and the selection of point cloud features inside the voxels, resulting in large differences in extraction results for different data and lacking universal applicability.

[0029] (4) Template matching method: This method requires the establishment of a structural template library for different types of trees and streetlights. The object features in the point cloud are matched with the templates in the structural template library, and the trees and streetlights are obtained based on the matching results. However, this method requires the establishment of a feature template library in advance, and it is impossible to extract trees and streetlights that do not exist in the template library, so it is not universally applicable.

[0030] Based on this, the present invention provides a target object extraction method, device and electronic device based on point cloud, which can alleviate the problems of low extraction rate, long time consumption, poor universality and insufficient extraction accuracy in existing tree and street light extraction technologies.

[0031] To facilitate understanding of this embodiment, a method for extracting target objects based on point clouds, as disclosed in this embodiment of the invention, will first be described in detail. (See [link to relevant documentation]). Figure 1 As shown, the method may include the following steps:

[0032] Step S102: Preprocess the original point cloud data and use a hierarchical filtering method to remove ground from the preprocessed point cloud data.

[0033] The raw point cloud data can be generated by 3D LiDAR scanning, and each point cloud data has four types of information: X coordinate value, Y coordinate value, Z coordinate value, and laser reflection intensity.

[0034] Step S104: Divide the point cloud data after removing the ground into multiple first point clouds and multiple second point clouds according to multiple preset height ranges.

[0035] In this first point cloud, the Z-coordinate value of each point is less than the median of at least one preset height range. For a portion of the point cloud data where both X and Y coordinate values ​​are within a certain range, the Z-coordinate value range of this portion of the point cloud data can be used as a preset height range, and the median of this Z-coordinate value range can be calculated. Then, points in this portion of the point cloud data whose Z-coordinate values ​​are less than the median are grouped into a first point cloud, and points in this portion of the point cloud data whose Z-coordinate values ​​are greater than or equal to the median are grouped into a second point cloud. Similarly, point cloud data with X and Y coordinate values ​​in other ranges can be divided into corresponding first and second point clouds, thus obtaining multiple first point clouds and multiple second point clouds.

[0036] Step S106: Project each first point cloud layer onto the XOY plane and extract point cloud clusters. Then, extract cylinders from each extracted point cloud cluster to obtain the local point cloud of the target object.

[0037] The target objects may include trees and / or streetlights.

[0038] Step S108: Project each local point cloud and the point cloud within a preset range around it onto the XOY plane and perform classification processing to obtain the corresponding first classification.

[0039] Step S110: Based on the intensity information and spatial location information of the points and the first classification of each local point cloud, the corresponding target points in each local point cloud and multiple second point clouds are clustered into corresponding overall point clouds, and the overall point clouds whose number of points and / or height range meet the first preset conditions are removed.

[0040] The intensity information can refer to the laser reflection intensity, and the spatial location information can refer to the X, Y, and Z coordinate values. The first preset condition can include at least one of the following: the number of points is less than a first threshold, the number of points is greater than a second threshold, the height range is less than a third threshold, and the height range is greater than a fourth threshold. The first, second, third, and fourth thresholds can all be customized based on experience and are not limited thereto.

[0041] This invention provides a target object extraction method based on point clouds. The method involves preprocessing the original point cloud data and using a hierarchical filtering approach to remove ground features from the preprocessed point cloud data. The point cloud data after ground removal is divided into multiple first point clouds and multiple second point clouds according to multiple preset height ranges. Each first point cloud is projected onto the XOY plane and point cloud clusters are extracted. Then, each extracted point cloud cluster is subjected to columnar extraction to obtain a local point cloud of the target object. Each local point cloud and the point clouds within a preset range around it are projected onto the XOY plane and classified to obtain a corresponding first classification. Based on the point intensity information, spatial location information, and the first classification of each local point cloud, the corresponding target points in each local point cloud and the multiple second point clouds are clustered into a corresponding overall point cloud. Overall point clouds whose number of points and / or height range meet a first preset condition are removed. Using the above techniques, ground removal employs layered filtering, which is simple to operate and improves the extraction efficiency of trees and streetlights compared to traditional ground removal algorithms. Projecting the point cloud onto the XOY plane for relevant extraction operations further improves the extraction rate of trees and streetlights, avoiding the situation where trees and streetlights are missed due to being in flower beds, shrubs, or on slopes. Combining the intensity information and spatial location information of points can accurately extract entangled trees and streetlights, further improving the accuracy of tree and streetlight extraction.

[0042] As one possible implementation, the above-mentioned step of performing ground removal on the preprocessed point cloud data using a layered filtering method may include: (11) dividing the preprocessed point cloud data into multiple first data blocks according to the X and Y coordinates of the points; (12) dividing each first data block into multiple first sub-data blocks according to the X and Y coordinates of the points, and dividing each first sub-data block into multiple layers of first point cloud data according to the Z coordinate of the points; (13) removing the first data blocks of the corresponding layer of first point cloud data with a number of points less than a preset first quantity threshold or the first data blocks of the first point cloud data with a number of layers less than two.

[0043] For example, the XOY plane can be divided into multiple 25m*25m regions. Then, the portion of the preprocessed point cloud data where the X and Y coordinates fall within the same 25m*25m region is segmented as a first data block, thus obtaining multiple first data blocks. To facilitate parallel processing and improve processing efficiency, each first data block is divided into multiple 1m*1m grid data on the XOY plane, and each grid data is further divided into multiple layers of first point cloud data with a thickness of 0.5m along the Z-axis. Then, it is determined whether the number of points in each layer of the first point cloud data of the first data block is less than a preset first quantity threshold and whether the number of layers of the first point cloud data of the first data block is less than two. Then, the first point cloud data of the corresponding layer with a number of points less than the preset first quantity threshold or the first data block with a number of layers less than two is removed.

[0044] As one possible implementation, the above steps of dividing the point cloud data after removing the ground into multiple first point clouds and multiple second point clouds according to multiple preset height ranges may include: (21) dividing the point cloud data after removing the ground into multiple second data blocks according to the X and Y coordinates of the points; (22) obtaining the Z coordinate range of the points in each second data block as a preset height range, and calculating the median of each preset height range; (23) forming the corresponding first point cloud with the points in each second data block whose Z coordinate is less than the corresponding median, and forming the corresponding second point cloud with the points in each second data block whose Z coordinate is greater than or equal to the corresponding median.

[0045] For example, the XOY plane can be divided into multiple 25m*25m regions. Then, the portion of the point cloud data after removing the ground whose X and Y coordinates fall within the same 25m*25m region is segmented as a second data block, thus obtaining multiple second data blocks. For each second data block, the Z coordinate value of each point in the second data block is traversed, and the Z coordinate range and the median of the Z coordinate range of the second data block are calculated. Then, the points in the second data block whose Z coordinates are less than the median are formed into a first point cloud, and the points in the second data block whose Z coordinates are greater than or equal to the median are formed into a second point cloud.

[0046] As one possible implementation, the steps of projecting each first point cloud layer onto the XOY plane and extracting point cloud clusters may include: (31) dividing each first point cloud into multiple layers of second point cloud data according to the Z coordinate of the points; (32) projecting each layer of second point cloud data onto the XOY plane and extracting point cloud clusters using a spatial clustering algorithm to obtain the corresponding initial point cloud clusters; (33) removing the initial point cloud clusters whose horizontal range and / or height range of the points meet the second preset conditions.

[0047] For example, continuing from the previous example, for each first point cloud, the first point cloud can be divided into multiple layers of second point cloud data with thicknesses of 1.0m, 0.5m, and 0.5m respectively, according to the direction of Z coordinate values ​​from small to large. Then, for each layer of second point cloud data, the second point cloud data of the layer is projected onto the XOY plane (the Z coordinate value of the points after projection is zero), and a spatial clustering algorithm is used to extract point cloud clusters. Then, the Z coordinate values ​​of the points in each point cloud cluster are reassigned to their original Z coordinate values. Then, point cloud clusters that meet at least one of the following conditions are removed: the horizontal range is greater than the first range (i.e., the horizontal range is too large), the horizontal range is less than the second range (i.e., the horizontal range is too small), and the height range is less than the third range (i.e., the height is too short).

[0048] As one possible implementation, the above steps of extracting cylinders from each extracted point cloud cluster to obtain the local point cloud of the target object may include: (41) using the RANSAC cylinder fitting algorithm to extract cylinders from each point cloud cluster to obtain the initial local point cloud of the target object; (42) merging different initial local point clouds that have intersections in the horizontal range into the same local point cloud to obtain the local point cloud of the target object.

[0049] For example, continuing from the previous example, after extracting the point cloud clusters of the second point cloud data of each layer, for each point cloud cluster, the RANSAC column fitting algorithm is used to extract the columns of the point cloud cluster based on the columnar geometric features of the trees and streetlights, thereby calculating the position of the trees and / or streetlights and the initial local point cloud; the horizontal range (i.e. the range falling into the XOY plane) of each initial local point cloud is calculated respectively, and the intersection between the horizontal ranges of the initial local point clouds of two adjacent layers of second point cloud data is calculated respectively. Two initial local point clouds whose calculation results are not empty sets can be merged into the same local point cloud, thereby obtaining the final position of the trees and / or streetlights and the local point cloud.

[0050] As one possible implementation, the first classification can be trees or streetlights; based on this, the step of projecting each local point cloud and the point cloud within a preset range around it onto the XOY plane and performing classification processing to obtain the corresponding first classification can include: (51) projecting each local point cloud and the point cloud within a preset range around it onto the XOY plane to obtain the corresponding local point cloud projection image; (52) calculating the two-dimensional entropy of each local point cloud projection image, and classifying each local point cloud into the corresponding tree or streetlight according to the two-dimensional entropy of the local point cloud projection image.

[0051] For example, continuing from the previous example, for each local point cloud, the range of the local point cloud is calculated, and this range is expanded by a preset range (e.g., an expansion of 0.5m). Then, the point cloud within the expanded range is projected onto the XOY plane to obtain the corresponding local point cloud projection image. The two-dimensional entropy of each local point cloud projection image is then calculated using the following formula: Where i is the pixel value of a pixel in the local point cloud projection image, j is the mean value of the K-neighbor pixels of a pixel in the local point cloud projection image, (i,j) is a tuple consisting of the pixel value of a pixel and the mean value of its K-neighbor pixels, P (,) Let (i,j) be the probability of occurrence of the tuple (i,j). Then, based on the differences in the distribution of the local point cloud projection images of trees and streetlights, a two-dimensional entropy threshold is set. Local point clouds with entropy values ​​greater than this threshold are classified as trees, while those with entropy values ​​less than or equal to the threshold are classified as streetlights. The two-dimensional entropy threshold can be set empirically, for example, to 1.3 or 1.5; there is no limitation on its value.

[0052] As one possible implementation, the step of clustering each local point cloud and the corresponding target points in the plurality of second point clouds into a corresponding overall point cloud based on the point intensity information, spatial location information, and the first classification of each local point cloud may include: (61) selecting corresponding first candidate point clouds and second candidate point clouds from the plurality of second point clouds according to the point intensity information; wherein, the intensity value of the points in the first candidate point cloud is higher than the intensity value of the points in the second candidate point cloud; (62) for each local point cloud divided into street lamps, according to the point spatial location information, points in the first candidate point cloud whose first distance and second distance from the local point cloud are both less than a preset first distance threshold are taken as the first target points corresponding to the local point cloud, and then the local point cloud and its corresponding first target points are clustered into an overall point cloud; (63) for each local point cloud divided into trees, according to the point spatial location information, points in the second candidate point cloud whose third distance and fourth distance from the local point cloud are both less than a preset second distance threshold are taken as the second target points corresponding to the local point cloud, and then the local point cloud and its corresponding second target points are clustered into an overall point cloud.

[0053] For example, continuing from the previous example, after obtaining the final locations and local point clouds of trees and / or streetlights, to improve processing efficiency, a spatial clustering algorithm can be used to cluster adjacent local point clouds into corresponding point cloud blocks, thereby obtaining multiple point cloud blocks. Then, the following parallel processing can be performed on the multiple point cloud blocks: For each point cloud block, points with laser reflection intensity values ​​higher than a preset first intensity threshold are selected from multiple second point clouds to form a first candidate point cloud; points with laser reflection intensity values ​​lower than a preset second intensity threshold are selected from multiple second point clouds to form a second candidate point cloud; points in each local point cloud of the streetlights within the point cloud block are used as first-class seed points, and the first distance and first... The second distance between each point in the candidate point cloud is calculated, and points in the first candidate point cloud whose first and second distances are both less than a preset first distance threshold are taken as the first target points corresponding to the local point cloud. Then, the local point cloud and its corresponding first target points are clustered into a whole point cloud. Points in each local point cloud divided into trees in the point cloud block are taken as second-class seed points. The third distance between each second-class seed point and the points in the second candidate point cloud and the fourth distance between each point in the second candidate point cloud are calculated. Points in the second candidate point cloud whose third and fourth distances are both less than a preset second distance threshold are taken as the second target points corresponding to the local point cloud. Then, the local point cloud and its corresponding second target points are clustered into a whole point cloud.

[0054] In the above operation method, for streetlights (especially for complex situations where trees and streetlights are intertwined), the principle of prioritizing intensity and proximity is used to perform spatial clustering of the point cloud, thereby obtaining the local point cloud of the lamp head, and then obtaining the overall point cloud of the streetlight; for trees (especially for complex situations where trees are intertwined), the principle of proximity is used to perform spatial clustering of the point cloud, thereby obtaining the local point cloud of the tree crown, and then obtaining the overall point cloud of the tree.

[0055] As one possible implementation, the step of removing the overall point cloud that meets the first preset condition in terms of the number of points and / or height range may include: removing the overall point cloud that meets at least one of the following conditions: the number of points is greater than a preset second number threshold (i.e., the number of points is too large), the number of points is less than a preset third number threshold (i.e., the number of points is too small), the height range is less than a fourth range (i.e., the height is too short), and the height range is greater than a fifth range (i.e., the height is too high).

[0056] For example, in practical applications, point clouds with fewer than 600 points or more than 12,000 points and / or point clouds with a height range of less than 1m or more than 16m can be removed as abnormal point clouds.

[0057] As one possible implementation, the above-mentioned preprocessing steps for the raw point cloud data may include: dividing the raw point cloud data into multiple third data blocks, and denoising the multiple third data blocks using a parallel processing method; and thinning the denoised point cloud data. In this operation, the use of a block-based parallel processing method for point cloud denoising improves the efficiency of point cloud denoising; point cloud thinning can reduce the point cloud density, thereby achieving the goal of improving the extraction efficiency of trees and streetlights.

[0058] For ease of understanding, the above-described point cloud-based target object extraction method is illustrated below with a specific application as an example. See [link to documentation]. Figure 2 As shown, this method mainly follows the steps below:

[0059] Step 1: Input point cloud data.

[0060] Step 2, point cloud data preprocessing.

[0061] Point cloud data preprocessing mainly includes point cloud denoising and point cloud thinning, with point cloud denoising using block-based parallel processing.

[0062] Step 3, remove the ground.

[0063] Ground point clouds are removed using a block-based, layered filtering method.

[0064] Step 4: Calculate the positions of trees and streetlights and extract local point clouds.

[0065] Step 5: Local point cloud classification processing for trees and streetlights.

[0066] For each local point cloud extracted in step 4, the range of the local point cloud is calculated and expanded by 0.5m. Then, the point cloud within the expanded range is projected onto the XOY plane to obtain the corresponding local point cloud projection image. The two-dimensional entropy of each local point cloud projection image is then calculated using the following formula: Local point clouds corresponding to projected images with a 2D entropy value greater than 1.3 are classified as trees, while local point clouds corresponding to projected images with a 2D entropy value less than or equal to 1.3 are classified as streetlights.

[0067] Step 6: Extract local point cloud data of tree canopy and lamp head.

[0068] Step 7: Remove outlier point clouds.

[0069] For details on the operation methods of steps 1 to 7 above, please refer to the aforementioned relevant content, which will not be repeated here.

[0070] The above-mentioned point cloud-based target object extraction method can achieve fully automatic and rapid extraction of trees and streetlights in large scenes. Compared with traditional geometric feature-based and spatial clustering-based schemes, it improves the extraction rate of trees and streetlights. At the same time, the ground removal optimization algorithm and block parallel processing strategy improve the extraction efficiency.

[0071] by Figure 3 The large-scale point cloud data shown is used as the raw point cloud data. Using the point cloud-based target object extraction method described above, the corresponding trees and streetlights can be extracted from this part of the point cloud data. Figure 4 and Figure 5 The effects of extracting trees and streetlights are shown separately. Figure 6 and Figure 7 The details of extracting trees and streetlights are shown separately.

[0072] To further verify the beneficial effects of the above-mentioned point cloud-based target object extraction method, Table 1 shows the extraction rate statistics of the above-mentioned point cloud-based target object extraction method, Table 2 shows the comparison of the extraction rate of the above-mentioned point cloud-based target object extraction method with the traditional method, and Table 3 shows the improvement in extraction efficiency of the above-mentioned ground removal algorithm and parallel processing strategy.

[0073] Table 1

[0074] Data Number Successfully extracted the number of trees and streetlights Number of trees and streetlights not extracted Extraction rate 1 102 5 95.32％ 2 155 3 98.10％ 3 115 2 98.29％ 4 150 3 98.03％ 5 84 2 97.67％ 6 140 3 97.90％

[0075] Table 2

[0076] Data Number Extraction rate of traditional methods Extraction rate of the method of the present invention Efficiency Improvement 1 85.15％ 95.32％ 10.17％ 2 85.48％ 98.10％ 12.62％ 3 86.71％ 98.29％ 11.58％ 4 87.94％ 98.03％ 10.09％ 5 86.29％ 97.67％ 11.38％ 6 84.92％ 97.90％ 12.98％

[0077] Table 3

[0078] Data Number Time before optimization Optimized time Efficiency Improvement 1 28.53s 19.06s 33.19％ 2 27.31s 18.42s 32.55％ 3 26.37s 18.18s 31.05％ 4 34.63s 24.08s 30.46％ 5 46.83s 29.98s 35.98％ 6 51.28s 35.14s 30.93％

[0079] As shown in Tables 1 to 3 above, the point cloud-based target object extraction method can achieve fully automatic and rapid extraction of trees and streetlights in large scenes, with an extraction rate of up to 95%. Compared with traditional geometric feature-based and spatial clustering-based schemes, the extraction rate is improved by 10%. At the same time, the ground removal optimization algorithm and the block parallel processing strategy improve the extraction efficiency by up to 30%.

[0080] Based on the above-described point cloud-based target object extraction method, this invention also provides a point cloud-based target object extraction device. (See [link to device description]). Figure 8 As shown, the device may include the following modules:

[0081] The preprocessing module 802 is used to preprocess the raw point cloud data and to remove ground from the preprocessed point cloud data using a hierarchical filtering method.

[0082] The segmentation module 804 is used to divide the point cloud data after removing the ground into multiple first point clouds and multiple second point clouds according to multiple preset height ranges; wherein the Z coordinate value of each point in the first point cloud is less than the median value of at least one preset height range.

[0083] The first acquisition module 806 is used to project each first point cloud layer onto the XOY plane and extract point cloud clusters, and then extract cylinders from each extracted point cloud cluster to obtain the local point cloud of the target object; wherein, the target object includes trees and / or streetlights.

[0084] The classification module 808 is used to project each local point cloud and the point cloud within a preset range around it onto the XOY plane and perform classification processing to obtain the corresponding first classification.

[0085] The second acquisition module 810 is used to cluster the corresponding target points in each local point cloud and the plurality of second point clouds into a corresponding overall point cloud based on the intensity information and spatial location information of the points and the first classification of each local point cloud, and to remove the overall point cloud whose number of points and / or height range meet the first preset condition.

[0086] This invention provides a target object extraction device based on point clouds. The device preprocesses the original point cloud data and uses a hierarchical filtering method to remove ground data from the preprocessed point cloud data. It then divides the ground-removed point cloud data into multiple first point clouds and multiple second point clouds according to multiple preset height ranges. Each first point cloud is projected onto the XOY plane and point cloud clusters are extracted. Each extracted point cloud cluster is then subjected to columnar extraction to obtain a local point cloud of the target object. Each local point cloud and the point clouds within a preset range around it are projected onto the XOY plane and classified to obtain a corresponding first classification. Based on the point intensity information, spatial location information, and the first classification of each local point cloud, the corresponding target points in each local point cloud and the multiple second point clouds are clustered into a corresponding overall point cloud. Overall point clouds whose number of points and / or height range meet a first preset condition are then removed. Using the above techniques, ground removal employs layered filtering, which is simple to operate and improves the extraction efficiency of trees and streetlights compared to traditional ground removal algorithms. Projecting the point cloud onto the XOY plane for relevant extraction operations further improves the extraction rate of trees and streetlights, avoiding the situation where trees and streetlights are missed due to being in flower beds, shrubs, or on slopes. Combining the intensity information and spatial location information of points can accurately extract entangled trees and streetlights, further improving the accuracy of tree and streetlight extraction.

[0087] The preprocessing module 802 described above can also be used to: divide the preprocessed point cloud data into multiple first data blocks according to the X and Y coordinates of the points; divide each first data block into multiple first sub-data blocks according to the X and Y coordinates of the points, and divide each first sub-data block into multiple layers of first point cloud data according to the Z coordinate of the points; and remove the first data blocks of the corresponding layer of first point cloud data with a number of points less than a preset first quantity threshold or with a number of layers of first point cloud data less than two layers.

[0088] The aforementioned partitioning module 804 can also be used to: divide the point cloud data after removing the ground into multiple second data blocks according to the X and Y coordinates of the points; obtain the Z coordinate range of the points in each second data block as a preset height range, and calculate the median of each preset height range; form a corresponding first point cloud by combining the points in each second data block whose Z coordinates are less than the corresponding median, and form a corresponding second point cloud by combining the points in each second data block whose Z coordinates are greater than or equal to the corresponding median.

[0089] The first acquisition module 806 described above can also be used to: divide each first point cloud into multiple layers of second point cloud data according to the Z coordinate of the point; project each layer of second point cloud data onto the XOY plane, and use a spatial clustering algorithm to extract point cloud clusters to obtain the corresponding initial point cloud clusters; and remove the initial point cloud clusters whose horizontal range and / or height range of the points meet the second preset conditions.

[0090] The first acquisition module 806 described above can also be used to: extract columns from each point cloud cluster using the RANSAC column fitting algorithm to obtain the initial local point cloud of the target object; merge different initial local point clouds that have intersections in the horizontal range into the same local point cloud to obtain the local point cloud of the target object.

[0091] The first category mentioned above can be trees or streetlights; based on this, the classification module 808 can also be used to: project each local point cloud and the point cloud within a preset range around it onto the XOY plane to obtain the corresponding local point cloud projection image; calculate the two-dimensional entropy of each local point cloud projection image, and classify each local point cloud into the corresponding tree or streetlight according to the two-dimensional entropy of the local point cloud projection image.

[0092] The second acquisition module 810 described above can also be used to: filter out corresponding first candidate point clouds and second candidate point clouds from the plurality of second point clouds according to the point intensity information; wherein, the intensity value of the points in the first candidate point cloud is higher than the intensity value of the points in the second candidate point cloud; for each local point cloud divided into streetlights, according to the spatial location information of the points, points in the first candidate point cloud whose first distance and second distance to the local point cloud are both less than a preset first distance threshold are taken as first target points corresponding to the local point cloud, and then the local point cloud and its corresponding first target points are clustered into a whole point cloud; for each local point cloud divided into trees, according to the spatial location information of the points, points in the second candidate point cloud whose third distance and fourth distance to the local point cloud are both less than a preset second distance threshold are taken as second target points corresponding to the local point cloud, and then the local point cloud and its corresponding second target points are clustered into a whole point cloud.

[0093] The preprocessing module 802 described above can also be used to: divide the original point cloud data into multiple third-second data blocks, and denoise the multiple third-second data blocks using a parallel processing method; and thin the point cloud data after denoising.

[0094] The target object extraction device based on point cloud provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned target object extraction method embodiment based on point cloud. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0095] This invention also provides an electronic device, such as... Figure 9 The diagram shows the structure of the electronic device, which includes a processor 91 and a memory 90. The memory 90 stores computer-executable instructions that can be executed by the processor 91. The processor 91 executes the computer-executable instructions to implement the above-mentioned point cloud-based target object extraction method.

[0096] exist Figure 9 In the illustrated embodiment, the electronic device further includes a bus 92 and a communication interface 93, wherein the processor 91, the communication interface 93, and the memory 90 are connected via the bus 92.

[0097] The memory 90 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 93 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 92 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 92 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0098] The processor 91 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 91 or by software instructions. The processor 91 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the point cloud-based target object extraction method disclosed in the embodiments of this invention can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory. The processor 91 reads the information in the memory and, in conjunction with its hardware, completes the steps of the point cloud-based target object extraction method of the aforementioned embodiment.

[0099] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0100] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the point cloud-based target object extraction method described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0102] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for extracting target objects based on point clouds, characterized in that, The method includes: The raw point cloud data is preprocessed, and a layered filtering method is used to remove ground from the preprocessed point cloud data. The point cloud data after removing the ground is divided into multiple first point clouds and multiple second point clouds according to multiple preset height ranges; wherein, the Z coordinate value of each point in the first point cloud is less than the median value of at least one preset height range; Each first point cloud is projected onto the XOY plane and point cloud clusters are extracted. Then, each extracted point cloud cluster is subjected to cylinder extraction to obtain the local point cloud of the target object; where the target object includes trees and / or streetlights. Project each local point cloud and the point cloud within a preset range around it onto the XOY plane and classify them to obtain the corresponding first category; Based on the intensity information and spatial location information of the points, as well as the first classification of each local point cloud, the corresponding target points in each local point cloud and the multiple second point clouds are clustered into the corresponding overall point cloud, and the overall point cloud that meets the first preset condition in terms of the number of points and / or height range is removed. The step of clustering each local point cloud and the corresponding target points in the plurality of second point clouds into a corresponding overall point cloud based on the point intensity information, spatial location information, and the first classification of each local point cloud includes: selecting corresponding first candidate point clouds and second candidate point clouds from the plurality of second point clouds according to the point intensity information; wherein, the intensity value of the points in the first candidate point cloud is higher than the intensity value of the points in the second candidate point cloud; for each local point cloud divided into streetlights, points in the first candidate point cloud whose first distance from the local point cloud and the second distance between each point are both less than a preset first distance threshold are taken as first target points corresponding to the local point cloud according to the point spatial location information, and then the local point cloud and its corresponding first target points are clustered into an overall point cloud; for each local point cloud divided into trees, points in the second candidate point cloud whose third distance from the local point cloud and the fourth distance between each point are both less than a preset second distance threshold are taken as second target points corresponding to the local point cloud according to the point spatial location information, and then the local point cloud and its corresponding second target points are clustered into an overall point cloud.

2. The method according to claim 1, characterized in that, The steps for removing ground features from preprocessed point cloud data using a hierarchical filtering method include: The preprocessed point cloud data is divided into multiple first data blocks according to the X and Y coordinates of the points; Each first data block is divided into multiple first sub-data blocks according to the X and Y coordinates of the points, and each first sub-data block is further divided into multiple layers of first point cloud data according to the Z coordinate of the points. Remove the first point cloud data of the corresponding layer whose number of points is less than the preset first number threshold, or the first data block whose number of layers of the first point cloud data is less than two layers.

3. The method according to claim 1, characterized in that, The steps of dividing the point cloud data after removing the ground into multiple first point clouds and multiple second point clouds according to multiple preset height ranges include: The point cloud data after removing the ground is divided into multiple second data blocks according to the X and Y coordinates of the points; Obtain the Z coordinate range of points in each second data block as a preset height range, and calculate the median value of each preset height range; Points in each second data block whose Z-coordinate is less than the corresponding median are formed into a first point cloud, and points in each second data block whose Z-coordinate is greater than or equal to the corresponding median are formed into a second point cloud.

4. The method according to claim 1, characterized in that, The steps of projecting each first point cloud layer onto the XOY plane and extracting point cloud clusters include: Each first point cloud is divided into multiple layers of second point cloud data according to the Z coordinate of the point; The second point cloud data of each layer is projected onto the XOY plane, and a spatial clustering algorithm is used to extract point cloud clusters to obtain the corresponding initial point cloud clusters; Initial point cloud clusters that satisfy the second preset condition in terms of the horizontal and / or vertical range of points are removed.

5. The method according to claim 1, characterized in that, The steps for extracting cylinders from each extracted point cloud cluster to obtain the local point cloud of the target object include: The Ransac column fitting algorithm is used to extract columns for each point cloud cluster to obtain the initial local point cloud of the target object. Different initial local point clouds that have intersections in the horizontal range are merged into a single local point cloud to obtain the local point cloud of the target object.

6. The method according to claim 1, characterized in that, The first classification is trees or streetlights; the steps of projecting each local point cloud and the point cloud within a preset range around it onto the XOY plane and performing classification processing to obtain the corresponding first classification include: Project each local point cloud and the point cloud within a preset range around it onto the XOY plane to obtain the corresponding local point cloud projection image; Calculate the two-dimensional entropy of each local point cloud projection image, and classify each local point cloud into a corresponding tree or street lamp according to the two-dimensional entropy of the local point cloud projection image.

7. The method according to claim 1, characterized in that, The steps for preprocessing raw point cloud data include: The original point cloud data is divided into multiple third data blocks, and noise is denoised in multiple third data blocks using a parallel processing method. The point cloud data after noise reduction is thinned.

8. A target object extraction device based on point clouds, characterized in that, The device includes: The preprocessing module is used to preprocess the raw point cloud data and to remove ground data from the preprocessed point cloud data using a hierarchical filtering method. The segmentation module is used to divide the point cloud data after removing the ground into multiple first point clouds and multiple second point clouds according to multiple preset height ranges; wherein, the Z coordinate value of each point in the first point cloud is less than the median value of at least one preset height range; The first acquisition module is used to project each first point cloud layer onto the XOY plane and extract point cloud clusters, and then extract cylinders from each extracted point cloud cluster to obtain the local point cloud of the target object; wherein, the target object includes trees and / or street lamps; The classification module is used to project each local point cloud and the point cloud within a preset range around it onto the XOY plane and perform classification processing to obtain the corresponding first classification. The second acquisition module is used to cluster the corresponding target points in each local point cloud and the plurality of second point clouds into a corresponding overall point cloud based on the intensity information and spatial location information of the points and the first classification of each local point cloud, and to remove the overall point cloud whose number of points and / or height range meet the first preset condition. The second acquisition module is further configured to: filter corresponding first candidate point clouds and second candidate point clouds from the plurality of second point clouds according to the point intensity information; wherein, the intensity value of the points in the first candidate point cloud is higher than the intensity value of the points in the second candidate point cloud; for each local point cloud divided into streetlights, according to the spatial location information of the points, points in the first candidate point cloud whose first distance to the local point cloud and the second distance between each point are both less than a preset first distance threshold are taken as first target points corresponding to the local point cloud, and then the local point cloud and its corresponding first target points are clustered into a whole point cloud; for each local point cloud divided into trees, according to the spatial location information of the points, points in the second candidate point cloud whose third distance to the local point cloud and the fourth distance between each point are both less than a preset second distance threshold are taken as second target points corresponding to the local point cloud, and then the local point cloud and its corresponding second target points are clustered into a whole point cloud.

9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 7.

Images