Electric power corridor tree species identification and classification method based on three-dimensional laser point cloud

Abstract

The invention relates to the technical field of point cloud data processing, in particular to an electric power corridor tree species identification and classification method based on three-dimensional laser point cloud. According to the method, the SuperPoint Graphs framework model is used for carrying out segmentation processing on the three-dimensional point cloud in a large scene, ground features such as ground points, vegetation and buildings can be efficiently and rapidly classified, and therefore the obtained vegetation point cloud data are subjected to single-tree segmentation to obtain a single-tree segmentation result; according to the method, a deep learning network model is used for carrying out registration processing on crown feature information, trunk feature information and multispectral tree species classification information subjected to multispectral labeling, tree classification is obtained, the point cloud classification result can be improved, and the problem that three-dimensional point cloud data are difficult to segment in a large scene is solved.

Description

Technical Field

[0001] The invention relates to the technical field of point cloud data processing, and in particular to a method for identifying and classifying tree species in a power corridor based on three-dimensional laser point clouds. Background Art

[0002] Overhead lines are an important part of power grid infrastructure construction and the basis for ensuring the safe and stable operation of distribution networks. In the natural environment, power corridors are affected by many uncertain factors, especially the growth of vegetation, which will affect the normal operation of transmission lines. With the large-scale construction and development of distribution networks, more and more drones are used for transmission line inspections. In drone inspection technology, the sensors carried on the flight platform include laser radar. As an active remote sensing technology, laser radar (LiDAR) can obtain target spatial information by measuring the distance of the target object through the laser emitted by the sensor. However, due to the different terrain and environment of urban areas, the objects of LiDAR point cloud vegetation extraction and processing are also different. The most important key to using LiDAR to extract urban vegetation is to distinguish vegetation point clouds from buildings and ground point clouds. Accurately extracting vegetation groups and building point clouds is a key problem that needs to be solved in this field. Only the effective extraction of vegetation information can provide guarantee for the later tree species identification work.

[0003] In the process of studying point cloud processing methods based on deep learning, most methods pre-process 3D point cloud data, use point cloud data to generate a large number of 2D images, and further process them using deep convolutional neural networks. However, in the processing of 3D point cloud data by convolutional neural networks, both the amount of calculation and the amount of data storage have increased greatly, and the processing efficiency is not high.

[0004] In summary, the direct processing method of 3D point cloud data is still in its early stages, especially in the processing of 3D point cloud data in large-scale scenes. Summary of the invention

[0005] The purpose of the present invention is to provide a method for identifying and classifying tree species in power corridors based on three-dimensional laser point clouds, and at the same time to provide an electronic device and a storage medium.

[0006] The technical solution to achieve the purpose of the present invention is:

[0007] A method for identifying and classifying tree species in power corridors based on three-dimensional laser point clouds comprises the following steps:

[0008] S1, collect laser point cloud data and pre-process the point cloud data;

[0009] S2, train the preprocessed point cloud data using the SuperPointGraphs framework model, and output the model after a period of training;

[0010] S3, using the model output in step S2, segmenting the point cloud data to obtain plane-based point cloud geometric division, super-point graph and segmentation results;

[0011] S4, retrieving the point cloud data marked as high vegetation in the segmentation result in step S3, and performing single tree segmentation to obtain single tree point cloud data;

[0012] S5. Determine tree feature information through the obtained single tree point cloud data, wherein the tree features include crown feature information and trunk feature information;

[0013] S6. The crown feature information, trunk feature information and multispectral tree species classification information annotated with multispectral labels are registered through a preset deep learning network model to obtain the tree classification results.

[0014] Furthermore, in step S1, the point cloud data is preprocessed including using point coordinate information and RGB information, wherein the point coordinate data corresponds to the label of each point one by one, and the coordinate information and the label are converted into .hdf5 format respectively.

[0015] Furthermore, in step S2, the training of the SuperPointGraphs framework model includes the following steps:

[0016] S21, partitioning the entire point cloud data based on the geometric plane, dividing the point cloud by linear, horizontal or vertical planes, and processing each partition as a super point;

[0017] S22. Use the Pytorch framework to embed each superpoint and obtain an SPG. Each node of the SPG represents a part of the point cloud data.

[0018] S23 uses the super point graph and inputs PointNet to classify and segment the points.

[0019] Furthermore, the SuperPointGraphs framework model is trained using the Semantic3D data benchmark.

[0020] Furthermore, in step S4, a watershed segmentation algorithm and a point cloud distance-based segmentation algorithm are used to perform single tree segmentation.

[0021] Furthermore, before step S6, it also includes: obtaining tree species information to be identified; using the tree species information and a preset model, classifying and processing the collected multispectral data of the target area to obtain the multispectral tree species classification information, wherein the target area includes the area corresponding to the point cloud data originally collected in step S1.

[0022] Furthermore, the crown feature information includes the crown diameter and the crown area; the trunk feature information includes the trunk height.

[0023] Correspondingly, the present invention also provides an electronic device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the above-mentioned method for identifying and classifying tree species in power corridors based on three-dimensional laser point clouds are implemented.

[0024] Correspondingly, the present invention also provides a storage medium, which stores computer executable instructions. When the computer executable instructions are loaded and executed by a processor, the steps of the method for identifying and classifying tree species in power corridors based on three-dimensional laser point clouds as described in any one of claims 1 to 8 are implemented.

[0025] Compared with the prior art, the present invention has the following significant advantages:

[0026] 1. By using the SuperPointGraphs framework model to segment the 3D point cloud in a large scene, ground points, vegetation, buildings and other objects can be classified efficiently and quickly;

[0027] 2. The vegetation point cloud data is obtained by segmenting the three-dimensional point cloud in the large scene, segmenting the single tree, and then aligning the crown feature information, trunk feature information and multi-spectral tree species classification information marked by multi-spectral annotation through a preset deep learning network model to obtain tree classification, which can improve the results of point cloud classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the flow chart of the tree species identification and classification method of the present invention.

[0029] Figure 2 It is a schematic diagram of the results obtained by using the power corridor tree species identification and classification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0031] In order to overcome the problem of difficulty in segmenting three-dimensional point cloud data in large scenes, the present invention uses the SuperPointGraphs framework model to segment the three-dimensional point cloud in large scenes, which can efficiently and quickly classify ground points, vegetation, buildings and other objects, thereby obtaining the vegetation point cloud data, and then segmenting the single tree. The deep learning network model is used to align the crown feature information, trunk feature information and multi-spectral tree species classification information annotated by multi-spectral, and the tree classification is obtained, which can improve the results of point cloud classification.

[0032] A method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud. Figure 1 As shown, the following steps are included:

[0033] S1, collect laser point cloud data and pre-process the point cloud data;

[0034] S2, train the preprocessed point cloud data using the SuperPointGraphs framework model, and output the model after a period of training;

[0035] S3, using the model output in step S2, segmenting the point cloud data to obtain plane-based point cloud geometric division, super-point graph and segmentation results;

[0036] S4, retrieving the point cloud data marked as high vegetation in the segmentation result in step S3, and performing single tree segmentation to obtain single tree point cloud data;

[0037] S5. Determine tree feature information through the obtained single tree point cloud data, wherein the tree features include crown feature information and trunk feature information;

[0038] S6. The crown feature information, trunk feature information and multispectral tree species classification information annotated with multispectral labels are registered through a preset deep learning network model to obtain the tree classification results.

[0039] Specifically, in step S1, point coordinate information and RGB information are used, and the point coordinate data corresponds to the label of each point one by one, and the coordinate information and the label are converted into .hdf5 format input to realize point cloud data preprocessing. Preferably, the point cloud data can also be thinned because the point cloud data density is too high and the number is too large.

[0040] The deep learning framework based on the present invention is the Pytorch framework, which requires the installation of Pytorch and TorchNet, and the adaptive installation of Python packages such as cupy, h5py, scipy, numpy, sklearn, and future. In the present invention, the SuperPointGraphs framework model is trained using the Semantic3D data benchmark.

[0041] Specifically, in step S2, the training of the SuperPointGraphs framework model includes the following steps:

[0042] S21, partitioning the entire point cloud data based on the geometric plane, dividing the point cloud by linear, horizontal or vertical planes, and processing each partition as a super point;

[0043] S22. Use the Pytorch framework to embed each superpoint and obtain an SPG. Each node of the SPG represents a part of the point cloud data.

[0044] S23 uses the super point graph and inputs PointNet to classify and segment the points.

[0045] Specifically, in step S4, a watershed segmentation algorithm and a segmentation algorithm based on point cloud distance are used to perform single tree segmentation. More specifically, the watershed segmentation algorithm and the segmentation algorithm based on point cloud distance are implemented by the following steps:

[0046] S51, generating a digital surface model and a digital elevation model by interpolating the ground points and non-ground points in the point cloud data marked as high vegetation using an inverse distance weighted method;

[0047] S52, obtaining a canopy height model based on a digital surface model and a digital elevation model;

[0048] S53, measuring the tree height and the crown size, and filtering the crown height model through a variable window size to obtain a crown vertex model; wherein the crown size includes the crown diameter and the crown area, and the tree height includes the trunk height.

[0049] S54, using Gaussian filtering to smooth the tree crown vertex model, detecting the maximum value of the variable window size and marking the tree top;

[0050] S55, using the watershed algorithm to perform the first single tree segmentation;

[0051] S56. Use a point cloud distance-based segmentation method to classify the point cloud data marked as high vegetation from high to low in point order. Points with a spacing greater than a specified threshold are excluded from the target tree. Points with a spacing less than the threshold are classified according to the minimum spacing rule to complete the second single tree segmentation.

[0052] Before step S6, the method further includes: obtaining information of tree species to be identified; and classifying and processing the collected multispectral data of the target area according to the tree species information and the preset model to obtain the multispectral tree species classification information, wherein the target area includes the area corresponding to the point cloud data originally collected in step S1. The preset model is obtained by annotating and training the initial model with the training data.

[0053] Specifically, the crown feature information includes the crown diameter and the crown area; the trunk feature information includes the trunk height.

[0054] Based on the same technical solution, the present invention also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the method for identifying and classifying tree species in power corridors based on three-dimensional laser point clouds as described above are implemented.

[0055] Based on the same technical solution, a storage medium stores computer executable instructions. When the computer executable instructions are loaded and executed by a processor, the steps of the method for identifying and classifying tree species in power corridors based on three-dimensional laser point clouds as described above are implemented.

[0056] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0060] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0061] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this specification can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0062] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0063] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud, characterized by: The steps include: S1, collect laser point cloud data and pre-process the point cloud data; S2, train the preprocessed point cloud data using the SuperPointGraphs framework model, and output the model after a period of training; S3, using the model output in step S2, segmenting the point cloud data to obtain plane-based point cloud geometric division, super-point graph and segmentation results; S4, retrieving the point cloud data marked as high vegetation in the segmentation result in step S3, and performing single tree segmentation to obtain single tree point cloud data; S5. Determine tree feature information through the obtained single tree point cloud data, wherein the tree features include crown feature information and trunk feature information; S6. The crown feature information, trunk feature information and multispectral tree species classification information annotated with multispectral labels are registered through a preset deep learning network model to obtain the tree classification results.

2. The method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud according to claim 1 is characterized in that: In step S1, the point cloud data is preprocessed, including using point coordinate information and RGB information, wherein the point coordinate data corresponds to the label of each point one by one, and the coordinate information and the label are converted into .hdf5 format respectively.

3. The method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud according to claim 1 is characterized in that: In step S2, the training of the SuperPointGraphs framework model includes the following steps: S21, partitioning the entire point cloud data based on the geometric plane, dividing the point cloud by linear, horizontal or vertical planes, and processing each partition as a super point; S22. Use the Pytorch framework to embed each superpoint and obtain an SPG. Each node of the SPG represents a part of the point cloud data. S23 uses the super point graph and inputs PointNet to classify and segment the points.

4. The method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud according to claim 3 is characterized by: The SuperPointGraphs framework model is trained using the Semantic3D data benchmark.

5. The method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud according to claim 1 is characterized in that: In step S4, a watershed segmentation algorithm and a point cloud distance-based segmentation algorithm are used to perform single tree segmentation.

6. The method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud according to claim 5 is characterized by: S51, generating a digital surface model and a digital elevation model by interpolating the ground points and non-ground points in the point cloud data marked as high vegetation using an inverse distance weighted method; S52, obtaining a canopy height model based on a digital surface model and a digital elevation model; S53, measuring tree height and crown size, filtering the crown height model through a variable window size to obtain a crown vertex model; S54, using Gaussian filtering to smooth the tree crown vertex model, detecting the maximum value of the variable window size and marking the tree top; S55, using the watershed algorithm to perform the first single tree segmentation; S56. Use a point cloud distance-based segmentation method to classify the point cloud data marked as high vegetation from high to low in point order. Points with a spacing greater than a specified threshold are excluded from the target tree. Points with a spacing less than the threshold are classified according to the minimum spacing rule to complete the second single tree segmentation.

7. The method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud according to claim 1, characterized in that: Before step S6, it also includes: obtaining tree species information to be identified; using the tree species information and a preset model, classifying and processing the collected multispectral data of the target area to obtain the multispectral tree species classification information, wherein the target area includes the area corresponding to the point cloud data originally collected in step S1.

8. The method for identifying and classifying tree species in power corridors based on three-dimensional laser point cloud according to claim 6 is characterized by: The tree crown characteristic information includes the tree crown diameter and the tree crown area; the tree trunk characteristic information includes the tree trunk height.

9. An electronic device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it implements the steps of the power corridor tree species identification and classification method based on three-dimensional laser point cloud as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores computer executable instructions, which, when loaded and executed by the processor, implement the steps of the method for identifying and classifying tree species in power corridors based on three-dimensional laser point clouds as described in any one of claims 1 to 8.

Images