Method for determining crown breadth of tree in power transmission line corridor and electronic equipment

Abstract

The invention provides a method for determining the crown breadth of a tree in a power transmission line corridor and electronic equipment. The method comprises the following steps: acquiring multi-source remote sensing data of the power transmission line corridor; extracting wave band image features and vegetation index features from the multispectral remote sensing image and superposing the wave band image features and the vegetation index features to obtain a spectral feature combined image with the same resolution as the multispectral remote sensing image; extracting a canopy height feature, an intensity feature and a leaf area index feature from the laser radar point cloud data and superposing the features to obtain a point cloud feature combined image; and respectively inputting the spectral feature combined image and the point cloud feature combined image into a dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combined image and the point cloud feature combined image to obtain a crown breadth extraction image. According to the method and the device, the problem that the tree crown breadth is difficult to comprehensively and accurately extract in coping with a complex and changeable natural environment due to the fact that a deep learning method in the prior art focuses on extraction of the tree crown breadth by using a single data source is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for determining tree crown widths in a transmission line corridor, a computer-readable storage medium, a computer program product, and an electronic device. Background Art

[0002] The extraction of tree crowns around transmission line corridors is crucial to ensuring power safety and reliability. By timely identifying and pinpointing potential threats, power failures can be prevented, operation and maintenance efficiency can be improved, the cost of manual inspections can be reduced, and tree obstacles can be accurately located and removed. Deep learning technology uses multi-layer neural networks to extract and represent complex features, has strong nonlinear representation capabilities, can process large-scale data, and performs well in tasks such as image recognition, natural language processing, and speech recognition. Deep learning does not require manual feature design, which reduces human intervention and bias. With the support of big data and high-performance computing hardware, the accuracy and generalization ability of deep learning models are constantly improving. In the task of extracting tree crowns around transmission line corridors, combining multi-source remote sensing data and deep learning technology for semantic segmentation is a cutting-edge research in the current field of remote sensing science and computer vision. Remote sensing data includes optical images, LiDAR (Light Detection and Ranging) point clouds, etc., which can provide rich spatial and spectral information. Deep learning technology, especially semantic segmentation methods, can accurately classify pixels in remote sensing images and realize automatic extraction of tree crown widths. This method can effectively distinguish different types of targets in complex scenes by training deep convolutional neural networks, significantly improving the accuracy and efficiency of tree crown extraction. This technology significantly improves the efficiency and accuracy of tree crown extraction around transmission line corridors, providing a strong guarantee for the safe operation of the power system. It has promoted the development of power system operation and maintenance towards intelligence and efficiency.

[0003] Traditional tree crown extraction methods usually require a lot of manual intervention and parameter adjustment, are complex to operate and difficult to adapt to different forest types and environmental conditions. Today's deep learning methods also focus on using a single data source to extract tree crowns. This method cannot fully utilize the data advantages of various types of data. When dealing with complex and changeable natural environments, it is often difficult to comprehensively and accurately extract tree crowns, which not only affects the accuracy and comprehensiveness of the data, but also limits the scope of application of the method. Summary of the invention

[0004] The main purpose of the present application is to provide a method for determining the tree crown width in a transmission line corridor, a computer-readable storage medium, a computer program product and an electronic device, so as to at least solve the problem that the deep learning method in the prior art focuses on using a single data source to extract the tree crown width, resulting in difficulty in comprehensively and accurately extracting the tree crown width when dealing with complex and changeable natural environments.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for determining the tree crown width in a transmission line corridor is provided, comprising: obtaining multi-source remote sensing data of the transmission line corridor, the multi-source remote sensing data comprising multispectral remote sensing images and lidar point cloud data; extracting band image features and vegetation index features from the multispectral remote sensing images, the band image features representing the radiation energy data of each band of the multispectral remote sensing image, and the vegetation index features representing the vegetation index of the tree crown width in the transmission line corridor; superimposing the band image features and the vegetation index features to obtain a resolution with the same resolution as the multispectral remote sensing image. Spectral feature combination image; extracting canopy height feature, intensity feature and leaf area index feature from the laser radar point cloud data, the canopy height feature characterizes the height information of the tree crown from the ground, the intensity feature characterizes the reflection intensity information of each point in the laser radar point cloud data, and the leaf area index feature characterizes the leaf area index of the tree crown; superimposing the canopy height feature, the intensity feature and the leaf area index feature to obtain a point cloud feature combination image; respectively inputting the spectral feature combination image and the point cloud feature combination image into a dual-channel target VGG (Visual Geometry Group)-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown extraction image, the target VGG-19 network is trained by neural network learning using multiple sets of data, and each set of data in the multiple sets of data includes: a sample spectral feature combination image, a sample point cloud feature combination image and a sample crown extraction image.

[0006] Optionally, extracting band image features and vegetation index features from the multispectral remote sensing image includes: performing radiation correction and geometric correction on the multispectral remote sensing image; and extracting the band image features and the vegetation index features from the corrected multispectral remote sensing image.

[0007] Optionally, the band image features and the vegetation index features are extracted from the corrected multispectral remote sensing image, including: extracting red light band data, green light band data, blue light band data and near-infrared light band data from the corrected multispectral remote sensing image to obtain the band image features; and determining the normalized vegetation index and the photochemical vegetation index based on the band image features to obtain the vegetation index features.

[0008] Optionally, the target VGG-19 network is obtained by improving the VGG-19 network, and the target VGG-19 network has two channels, namely a first channel and a second channel, the first channel is used to input the spectral feature combination image, and the second channel is used to input the point cloud feature combination image, the activation function in the target VGG-19 network is a PReLU function, each of the channels includes two Dropout layers and at least seven convolution blocks arranged in sequence, the input end of one of the Dropout layers is connected to the output end of the fourth convolution block, and the input end of the other Dropout layer is connected to the output end of the fifth convolution block, and the spectral feature combination image and the point cloud feature combination image are respectively input into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown extraction image, including: inputting the spectral feature combination image into the target VGG-19 network through the first channel 9 network, so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and performs upsampling and feature selection on the multi-level features in the spectral feature combination image layer by layer and step by step to obtain a first crown-width extraction image; the point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and performs upsampling and feature selection on the multi-level features in the point cloud feature combination image layer by layer and step by step to obtain a second crown-width extraction image, and the first crown-width extraction image and the second crown-width extraction image are fused with high-dimensional semantic features to obtain the crown-width extraction image, wherein the high-dimensional semantic features are features containing semantic information extracted by the target VGG-19 network in the process of performing upsampling and feature selection on the multi-level features in the spectral feature combination image and the multi-level features in the point cloud feature combination image layer by layer and step by step.

[0009] Optionally, the spectral feature combination image is input into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and performs layer-by-layer and level-by-level upsampling and feature selection on the multi-level features in the spectral feature combination image to obtain a first crown extraction image, including: inputting the spectral feature combination image into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image to obtain seven first feature maps whose sizes are successively reduced by 1 / 2, the seven first feature maps contain feature information from low level to high level in the spectral feature combination image, and performing two The invention relates to an embodiment of the present invention which comprises the following steps of: performing convolution operation on the first five first feature maps and the last first feature map to generate six second feature maps with a feature dimension of 32, and processing the six second feature maps by upsampling and adding feature elements to obtain five third feature maps, and the second stage comprises: performing convolution operation on the five third feature maps to obtain five fourth feature maps with a feature dimension of 2, and processing the five fourth feature maps by upsampling and adding feature elements to obtain four fifth feature maps, and extracting the four fifth feature maps by using the first Softmax layer in the target VGG-19 network to obtain the first crown extraction image.

[0010] Optionally, the point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and performs upsampling and feature selection on the multi-level features in the point cloud feature combination image layer by layer and level by level to obtain a second crown width extraction image, and the first crown width extraction image and the second crown width extraction image are fused with high-dimensional semantic features to obtain the crown width extraction image, wherein the high-dimensional semantic features are the target VGG-19 network in the spectral feature combination image. The multi-level features and the multi-level features in the point cloud feature combination image are up-sampled layer by layer and selected, and the features containing semantic information are extracted, including: inputting the point cloud feature combination image into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image to obtain seven sixth feature maps whose sizes are successively reduced by 1 / 2, the seven sixth feature maps contain low-level to high-level feature information in the point cloud feature combination image, and performing two-stage upsampling on the seven sixth feature maps. And feature selection, wherein the two stages are respectively the third stage and the fourth stage, the third stage includes: performing a convolution operation on the first five sixth feature maps and the last sixth feature map to generate six seventh feature maps with a feature dimension of 32, and processing the six seventh feature maps by upsampling and adding feature elements to obtain five eighth feature maps, the fourth stage includes: performing a convolution operation on the five eighth feature maps to obtain five ninth feature maps with a feature dimension of 2, and processing the five ninth feature maps by upsampling and adding feature elements to obtain four tenth feature maps, The second Softmax layer in the target VGG-19 network is used to extract the four tenth feature maps to obtain the second crown extraction image, and the second-to-last fifth feature map is upsampled to obtain the first high-dimensional semantic feature, and the second-to-last tenth feature map is upsampled to obtain the second high-dimensional semantic feature, and the first high-dimensional semantic feature and the second high-dimensional semantic feature are feature fused to obtain the high-dimensional semantic feature, and the first crown extraction image and the second crown extraction image are fused with the high-dimensional semantic features to obtain the crown extraction image.

[0011] Optionally, determining a normalized vegetation index and a photochemical vegetation index according to the band image characteristics includes: determining the normalized vegetation index according to the red light band data and the near-infrared light band data; and determining the photochemical vegetation index according to the green light band data.

[0012] According to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for determining the tree crown width in the transmission line corridor.

[0013] According to another aspect of the present application, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement any of the above methods for determining tree crown widths in a transmission line corridor.

[0014] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for determining the tree crown width in the transmission line corridor as described above.

[0015] By applying the technical solution of the present application, multi-source remote sensing data including multispectral remote sensing images and lidar point cloud data of the transmission line corridor are first obtained, and then band image features and vegetation index features are extracted from the multispectral remote sensing image, and the band image features are superimposed with the vegetation index features to obtain a spectral feature combination image with the same resolution as the multispectral remote sensing image, and then canopy height features, intensity features and leaf area index features are extracted from the lidar point cloud data, and the canopy height features, intensity features and leaf area index features are superimposed to obtain a point cloud feature combination image, and the spectral feature combination image and the point cloud feature combination image are respectively input into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown width extraction image. Compared with the deep learning methods in the prior art that focus on using a single data source to extract tree crowns, resulting in difficulty in comprehensively and accurately extracting tree crowns when dealing with complex and changeable natural environments, the present application obtains multispectral remote sensing images and lidar point cloud data, utilizes the data advantages of multispectral data and point cloud data, and obtains spectral feature combination images and point cloud feature combination images through certain extraction and overlay processing. Finally, the spectral feature combination images and point cloud feature combination images are respectively input into the dual-channel target VGG-19 network to obtain crown extraction images, making full use of the data advantages of various types of data, combining spectral information and three-dimensional structure information, and ensuring that the extraction of tree crowns is relatively accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for determining tree crown width in a transmission line corridor provided by an embodiment of the present invention;

[0018] Figure 2 It is a flowchart of a method for determining the tree crown width in a transmission line corridor provided in an embodiment of the present application;

[0019] Figure 3 It is a flowchart of a method for determining the tree crown width in a transmission line corridor according to an embodiment of the present application;

[0020] Figure 4 2 is a schematic diagram of the architecture of a dual-channel target VGG-19 network provided according to an embodiment of the present application.

[0021] The above drawings include the following reference numerals:

[0022] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] As introduced in the background technology, the deep learning methods in the prior art focus on using a single data source to realize the extraction of tree crown width, which makes it difficult to comprehensively and accurately extract the tree crown width when dealing with complex and changeable natural environments. To solve the above problems, the embodiments of the present application provide a method for determining the tree crown width in a transmission line corridor, a computer-readable storage medium, a computer program product and an electronic device.

[0027] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal for determining a tree crown width in a transmission line corridor according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the tree crown width in the transmission line corridor in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a method for determining the tree crown width in a transmission line corridor running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] Figure 2 FIG. 1 is a flow chart of a method for determining the tree crown width in a transmission line corridor according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0032] Step S201, obtaining multi-source remote sensing data of the transmission line corridor, the multi-source remote sensing data including multi-spectral remote sensing images and lidar point cloud data.

[0033] Step S202, extracting band image features and vegetation index features from the multispectral remote sensing image, the band image features represent the radiation energy data of each band of the multispectral remote sensing image, and the vegetation index features represent the vegetation index of the tree crown in the transmission line corridor.

[0034] Step S203, superimposing the band image features and the vegetation index features to obtain a spectral feature combined image with the same resolution as the multispectral remote sensing image.

[0035] Specifically, resolution refers to spatial resolution.

[0036] Step S204, extracting canopy height features, intensity features and leaf area index features from the lidar point cloud data, wherein the canopy height feature represents the height information of the tree crown from the ground, the intensity feature represents the reflection intensity information of each point in the lidar point cloud data, and the leaf area index feature represents the leaf area index of the tree crown.

[0037] Step S205 , superimposing the canopy height feature, the intensity feature and the leaf area index feature to obtain a point cloud feature combined image.

[0038] Step S206, respectively inputting the spectral feature combination image and the point cloud feature combination image into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown width extraction image, wherein the target VGG-19 network is trained by neural network learning using multiple sets of data, and each set of data in the multiple sets of data includes: a sample spectral feature combination image, a sample point cloud feature combination image, and a sample crown width extraction image.

[0039] Through the above embodiment, multi-source remote sensing data including multispectral remote sensing images and lidar point cloud data of the transmission line corridor is first obtained, and then the band image features and vegetation index features are extracted from the multispectral remote sensing image, and the band image features are superimposed with the vegetation index features to obtain a spectral feature combination image with the same resolution as the multispectral remote sensing image, and then the canopy height features, intensity features and leaf area index features are extracted from the lidar point cloud data, and the canopy height features, intensity features and leaf area index features are superimposed to obtain a point cloud feature combination image, and the spectral feature combination image and the point cloud feature combination image are respectively input into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown width extraction image. Compared with the deep learning methods in the prior art that focus on using a single data source to extract tree crowns, resulting in difficulty in comprehensively and accurately extracting tree crowns when dealing with complex and changeable natural environments, the present application obtains multispectral remote sensing images and lidar point cloud data, utilizes the data advantages of multispectral data and point cloud data, and obtains spectral feature combination images and point cloud feature combination images through certain extraction and overlay processing. Finally, the spectral feature combination images and point cloud feature combination images are respectively input into the dual-channel target VGG-19 network to obtain crown extraction images, making full use of the data advantages of various types of data, combining spectral information and three-dimensional structure information, and ensuring that the extraction of tree crowns is relatively accurate.

[0040] Specifically, in data processing, "overlay" is a common data integration technology, which mainly refers to aligning and combining two or more sets of data from different sources or different types according to a certain spatial reference system to form a new data set. This new data set will contain the information of all the original data sets.

[0041] Specifically, multispectral data covers multiple bands such as visible light, near infrared and short-wave infrared, and can capture the reflectivity differences of different tree species in each band. These differences provide a unique spectral feature basis for the extraction of tree crowns; point cloud data is a set of vectors in a three-dimensional coordinate system obtained by advanced technologies such as laser radar (LiDAR), which provides high-precision three-dimensional structural information of trees. Point cloud data can accurately measure geometric features such as tree height, canopy shape and trunk diameter, which vary significantly among different tree species. Specifically, by superimposing canopy height features, intensity features and leaf area index features, a combined image containing rich information on tree crown height, intensity and leaf area index can be obtained, which can more comprehensively characterize the canopy characteristics of trees. Specifically, by combining spectral features and point cloud data, tree crowns can be more accurately identified and distinguished. Using multispectral remote sensing images and laser radar point cloud data, the characteristics of tree crowns can be understood from different angles and dimensions. Through the dual-channel target VGG-19 network, spectral and spatial information can be analyzed more finely.

[0042] Specifically, the above-mentioned multispectral remote sensing images are high-resolution remote sensing images. Multi-source remote sensing data in the transmission line corridor can be collected based on the drone multi-sensor platform: airborne sensors are devices installed on aircraft or drones for ground remote sensing detection, including common high-resolution optical sensors and lidar sensors, through which a variety of remote sensing data can be obtained, such as multispectral remote sensing images and lidar point cloud data. Multispectral remote sensing images can provide radiation energy data in different bands, and LiDAR point cloud data can provide high-precision spatial data. In the embodiment of the present application, a DJI M300 RTK drone equipped with an H20T camera and a Zenmuse L2 lidar is used to obtain multi-source remote sensing data, wherein the H20T camera can obtain high-resolution RGB-NIR remote sensing images, and the Zenmuse L2 airborne LiDAR device can obtain point cloud data with a plane accuracy of 5 cm and an elevation accuracy of 4 cm.

[0043] According to some exemplary embodiments of the present application, band image features and vegetation index features are extracted from multispectral remote sensing images, including: performing radiation correction and geometric correction on the multispectral remote sensing images; and extracting band image features and vegetation index features from the corrected multispectral remote sensing images.

[0044] In this embodiment, the multispectral high-resolution remote sensing image is subjected to radiation correction to eliminate atmospheric interference and sensor noise, and the image is subjected to geometric correction to correct the geographic location deviation so that the image is aligned with the actual geographic coordinates. The extracted band image features and vegetation index features can more accurately reflect the real features of the ground objects, which can significantly improve the quality and availability of the remote sensing image and facilitate subsequent data analysis.

[0045] According to some exemplary embodiments of the present application, band image features and vegetation index features are extracted from the corrected multispectral remote sensing images, including: extracting red light band data, green light band data, blue light band data and near-infrared light band data from the corrected multispectral remote sensing images to obtain band image features; determining the normalized vegetation index and the photochemical vegetation index based on the band image features to obtain vegetation index features.

[0046] In this embodiment, different bands are extracted from the corrected multispectral remote sensing image to capture the response of different bands to vegetation characteristics. Band image characteristics can be obtained through band data, and vegetation index characteristics can be further determined, which can more accurately reflect the health status and biophysical characteristics of trees and enhance the ability to identify tree crown width in subsequent analysis.

[0047] It should be noted that the spatial resolution of the obtained Normalized Difference Vegetation Index (NDVI) and Photochemical Reflectance Index (PRI) is consistent with the spatial resolution of multispectral remote sensing images.

[0048] Figure 3 is a flowchart of a method for determining the tree crown width in a transmission line corridor according to an embodiment of the present application, such as Figure 3As shown in FIG. 1 , to generate a spectral feature combination image through multispectral remote sensing images, it is necessary to first extract four-band data of red (Red, R), green (Green, G), blue (Blue, B), and near infrared (NIR) from the spectral data (i.e., the multispectral remote sensing image), obtain the normalized vegetation index (NDVI) and the photochemical vegetation index (PRI) through these four-band data, and superimpose the six-band data of R, G, B, NIR, NDVI, and PRI to generate a spectral feature combination (RGB-NIR-NDVI-PRI) (i.e., a spectral feature combination image) that is consistent with the spatial resolution of the multispectral remote sensing image; to generate a point cloud feature combination image through lidar point cloud data, it is necessary to first extract intensity from the point cloud data (i.e., the lidar point cloud data), and obtain the canopy height model (Canopy Height Model, CHM) and leaf area index (Leaf area The intensity (Intensity), canopy height model (CHM) and leaf area index (LAI) are superimposed and combined to obtain a point cloud feature combination (Intensity-CHM-LAI) (i.e., a point cloud feature combination image); the spectral feature combination (RGB-NIR-NDVI-PRI) is used as the input of one channel of the dual-channel target VGG-19 network, and the point cloud feature combination (Intensity-CHM-LAI) is used as the input of the other channel of the dual-channel target VGG-19 network to obtain a crown width extraction image (i.e., a detailed tree crown width extraction result).

[0049] Specifically, extracting the canopy height model (CHM) (i.e., canopy height features) from the lidar point cloud data includes the following steps: 1) removing discrete outliers in the lidar point cloud data; 2) distinguishing ground points and non-ground points in the lidar point cloud data; 3) rasterizing the ground points and non-ground points into a digital surface model (DSM) and a digital elevation model (DEM) respectively by the natural neighbor interpolation method; 4) calculating the difference between the digital surface model (DSM) and the digital elevation model (DEM), which is the canopy height model (CHM).

[0050] Specifically, extracting leaf area index (LAI) (i.e., leaf area index feature) from lidar point cloud data includes the following steps: 1) removing discrete outliers in lidar point cloud data; 2) distinguishing ground points from non-ground points in lidar point cloud data; 3) for each statistical unit, first constructing a three-dimensional grid with 1.5 times the average point spacing of the point cloud according to the average point spacing of the point cloud; 4) based on the constructed three-dimensional network, counting the total number of three-dimensional grids in each layer and the frequency of laser points Where, N(s) is the laser point frequency of the Sth layer, n I (s) is the total number of grids containing laser points in the Sth layer, n T (s) is the total number of three-dimensional grids in the Sth layer; 5) Calculate the leaf area index of the Sth layer: l(s) = α(θ)N(s), where α(θ) is the leaf inclination correction factor, usually set to 1.1. Finally, the leaf area index of each layer is accumulated to obtain the leaf area index of the entire statistical unit. Specifically, the resolution of each statistical unit is consistent with the resolution of the multispectral remote sensing image.

[0051] According to some exemplary embodiments of the present application, the target VGG-19 network is obtained by improving the VGG-19 network, and the target VGG-19 network has two channels, namely a first channel and a second channel, the first channel is used to input a spectral feature combination image, and the second channel is used to input a point cloud feature combination image, and the activation function in the target VGG-19 network is a PReLU function, and each channel includes two Dropout layers and at least seven convolution blocks arranged in sequence, wherein the input end of one Dropout layer is connected to the output end of the fourth convolution block, and the input end of the other Dropout layer is connected to the output end of the fifth convolution block, and the spectral feature combination image and the point cloud feature combination image are respectively input into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown extraction image, including: inputting the spectral feature combination image into the target VGG-19 network through the first channel The target VGG-19 network is inputted into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and upsamples and selects the multi-level features in the spectral feature combination image layer by layer and step by step to obtain a first crown width extraction image; the point cloud feature combination image is inputted into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and upsamples and selects the multi-level features in the point cloud feature combination image layer by layer and step by step to obtain a second crown width extraction image, and the first crown width extraction image and the second crown width extraction image are fused with the high-dimensional semantic features to obtain the crown width extraction image, wherein the high-dimensional semantic features are features containing semantic information extracted by the target VGG-19 network in the process of upsampling and feature selecting the multi-level features in the spectral feature combination image and the multi-level features in the point cloud feature combination image layer by layer and step by step.

[0052] In this embodiment, by respectively inputting the spectral feature combination image and the point cloud feature combination image into the dual-channel target VGG-19 network, the accurate extraction of the tree crown is achieved. The target VGG-19 network can capture rich feature information from low levels to high levels. Through layer-by-layer upsampling and feature selection, the network can gradually restore the spatial resolution of the image while maintaining the feature space dimension.

[0053] Figure 4 is a schematic diagram of the architecture of a dual-channel target VGG-19 network provided according to an embodiment of the present application, such as Figure 4As shown in the figure, in the dual-channel target VGG-19 network, the input of one channel (i.e., the first channel) is a multispectral remote sensing image, and the input of the other channel (i.e., the second channel) is a point cloud feature combination image. The ReLU function in the original VGG-19 network is replaced by a PReLU activation function. At the same time, a Dropout layer is added after the fourth and fifth convolution blocks, and the dual-channel target VGG-19 network is used to learn multi-level features from the two input feature combination images.

[0054] According to some exemplary embodiments of the present application, the spectral feature combination image is input into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and performs upsampling and feature selection on the multi-level features in the spectral feature combination image layer by layer and level by level to obtain a first crown extraction image, including: inputting the spectral feature combination image into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image to obtain seven first feature maps whose sizes are successively reduced by 1 / 2, the seven first feature maps contain feature information from low level to high level in the spectral feature combination image, and performing two-level upsampling on the seven first feature maps. The upsampling and feature selection of the segment are performed, wherein the two stages are respectively the first stage and the second stage, the first stage includes: performing a convolution operation on the first five first feature maps and the last first feature map to generate six second feature maps with a feature dimension of 32, and processing the six second feature maps by upsampling and adding feature elements to obtain five third feature maps, the second stage includes: performing a convolution operation on the five third feature maps to obtain five fourth feature maps with a feature dimension of 2, and processing the five fourth feature maps by upsampling and adding feature elements to obtain four fifth feature maps, and extracting the four fifth feature maps by using the first Softmax layer in the target VGG-19 network to obtain a first crown extraction image.

[0055] In this embodiment, the spectral feature combined image is input into the target VGG-19 network through the first channel, so as to extract and learn the multi-level features in the image through a deep learning model to extract and identify the crown. The network can learn visual features from low-level to high-level, and can accurately identify and segment the crown area. By upsampling and fusing feature maps of different sizes, it can better process crowns of different scales. Due to the feature fusion process, more image details can be retained to a certain extent, thereby ensuring the recognition accuracy of the crown edge.

[0056] According to some exemplary embodiments of the present application, the point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and performs upsampling and feature selection on the multi-level features in the point cloud feature combination image layer by layer and level by level to obtain a second crown width extraction image, and the first crown width extraction image and the second crown width extraction image are fused with the high-dimensional semantic features to obtain the crown width extraction image, wherein the high-dimensional semantic features are the target VGG-19 network in the spectral feature combination image The method comprises: inputting the point cloud feature combination image into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image to obtain seven sixth feature maps whose sizes are successively reduced by 1 / 2, the seven sixth feature maps contain low-level to high-level feature information in the point cloud feature combination image, and performing two-stage upsampling on the seven sixth feature maps to obtain the seventh feature map. and feature selection, wherein the two stages are respectively the third stage and the fourth stage, the third stage comprising: performing a convolution operation on the first five sixth feature maps and the last sixth feature map to generate six seventh feature maps with a feature dimension of 32, and processing the six seventh feature maps by upsampling and adding feature elements to obtain five eighth feature maps, the fourth stage comprising: performing a convolution operation on the five eighth feature maps to obtain five ninth feature maps with a feature dimension of 2, and processing the five ninth feature maps by upsampling and adding feature elements to obtain four tenth feature maps, and extracting the four tenth feature maps by using the second Softmax layer in the target VGG-19 network to obtain a second crown width extracted image, and upsampling the second-to-last fifth feature map to obtain a first high-dimensional semantic feature, and upsampling the second-to-last tenth feature map to obtain a second high-dimensional semantic feature, and feature fusion of the first high-dimensional semantic feature and the second high-dimensional semantic feature to obtain a high-dimensional semantic feature, and fusion of the first crown width extracted image and the second crown width extracted image with the high-dimensional semantic feature to obtain a crown width extracted image.

[0057] In this embodiment, the point cloud feature combination image is input into the target VGG-19 network through the second channel to generate a rough tree crown extraction result (i.e., the second crown extraction image). By learning the multi-level features of the point cloud feature combination image and using two stages of upsampling and feature selection, the target VGG-19 network can extract a crown feature map with rich details. By upsampling the second to last feature map, high-dimensional semantic features are obtained, which enriches the expressive ability of the features. After being processed by two Softmax layers, the first crown extraction image and the second crown extraction image obtained can more accurately represent the range and shape of the tree crown. The high and low level features are fused to generate the final crown extraction image, which improves the accuracy and robustness of the crown extraction.

[0058] like Figure 4 As shown in the figure, the spectral feature combination image and the point cloud feature combination image are input into the first channel and the second channel respectively, and seven groups of feature maps with sizes reduced by 1 / 2 are learned from the two input images through multiple convolution and pooling operations from the convolutional neural network. The gradient vanishing problem caused by the convolution process is reduced by replacing the ReLU function in the original VGG-19 network with the PreLU function, and a Dropout layer is added after the fourth and fifth convolution blocks to prevent the model from overfitting. Figure 4 As shown in the figure, these feature maps generated by the dual-channel target VGG-19 network extract various feature information of the input image from low level to high level, and perform two stages of upsampling and feature selection operations (i.e., feature selection) on the seven groups of feature maps of the dual channels: in the first stage, the seven groups of feature maps (excluding the sixth group) are convolved, and after the convolution process, six groups of feature maps with a feature dimension of 32 are obtained, and five groups of feature maps are output by upsampling and adding feature elements; in the second stage, the five groups of feature maps are convolved again to generate five groups of feature maps with a feature dimension of 2, and four groups of feature maps are output by upsampling and feature superposition. Finally, the Softmax extraction layer (i.e., Softmax layer) is used to extract the feature maps obtained by processing the dual channels respectively, and a rough tree crown extraction result (i.e., the first crown extraction image and the second crown extraction image) is obtained for each channel. Specifically, as Figure 4 As shown in the figure, each upsampled feature map is processed by the feature fusion module at different levels and then fused through the splicing operation to generate a rough tree crown extraction result.

[0059] like Figure 4As shown in the figure, the high-dimensional semantic features are taken from the upsampling process in the second stage and the output of the penultimate layer of feature selection. First, these high-dimensional semantic features are doubled upsampled to keep them consistent with the resolution of the dual-channel extraction results. Then the two are superimposed and fused, and then passed through a convolution layer to further enhance the expression ability of the feature map, highlight important features, suppress noise and irrelevant information, and finally pass through the Softmax extraction layer to extract each pixel and output the probability distribution of each pixel belonging to each category, finally generating a detailed tree crown extraction result (i.e., crown extraction image).

[0060] Specifically, Figure 4 As shown, upsampling (2) represents double upsampling, upsampling (0) represents zero upsampling, each cuboid represents a feature map, A represents pixel-level addition, and C represents concatenation.

[0061] According to some exemplary embodiments of the present application, the normalized vegetation index and the photochemical vegetation index are determined according to the band image characteristics, including: determining the normalized vegetation index according to the red light band data and the near-infrared light band data; determining the photochemical vegetation index according to the green light band data.

[0062] In this embodiment, by using the normalized difference vegetation index and the photochemical vegetation index, vegetation can be more accurately identified and distinguished, can adapt to different environmental conditions and vegetation types, and provide more comprehensive vegetation status information.

[0063] Specifically, the data of the red light band, near infrared band, 531nm band and 570nm band are extracted from the multispectral image respectively; the normalized vegetation index value of each pixel is calculated using the NDVI formula Among them, NIR is the reflectance value of the near-infrared band, and RED is the reflectance value of the red light band; the photochemical vegetation index of each pixel is calculated using the PRI formula Among them, R 531 is the reflectivity value in the 531nm band, R 570 It is the reflectivity value in the 570nm band, and 531nm-570nm belongs to the green light band value.

[0064] Through the technical solution provided in the above embodiments of the present application, multi-source remote sensing data is adopted, and the data advantages of multispectral data and point cloud data are utilized. Through a certain feature extraction method, multispectral feature combination data and point cloud feature combination data are obtained, and they are used as the input of the deep learning network to more accurately extract the tree crown width. The dual-channel target VGG-19 network has the following two characteristics: 1) Dual-channel input can simultaneously learn the feature combination of multispectral data and the feature combination of point cloud data; 2) Compared with the original VGG-19 network, the ReLU function in the original VGG-19 network is replaced by the PReLU activation function. At the same time, a Dropout layer is added after the fourth and fifth convolution blocks.

[0065] As can be seen from the above, in the embodiment of the present application, the data advantages of multispectral data and point cloud data are fully utilized, and four-band data of red (R), green (G), blue (B), and near infrared (NIR) are extracted from the multispectral remote sensing image. The normalized vegetation index (NDVI) and the photochemical vegetation index (PRI) are obtained through the four-band data, and the six-band data of R, G, B, NIR, NDVI, and PRI are superimposed to generate a spectral feature combination image consistent with the resolution of the multispectral remote sensing image, and the intensity feature (Intensity), canopy height model (CHM) and leaf area index (LAI) features are extracted from the lidar point cloud data. The three-band data of Intensity, CHM and LAI are superimposed to obtain a point cloud feature combination image, and the accurate extraction of the tree crown width in the transmission line corridor is realized through the dual-channel target VGG-19 network.

[0066] Specifically, the method for determining the tree crown width in the transmission line corridor provided in the embodiment of the present application comprehensively considers various factors, and extracts corresponding feature data from the multispectral remote sensing images and lidar point cloud data collected by the drone, respectively, and constructs two sets of feature combination images through the overlay method as the input of the dual-channel target VGG-19 network. The target VGG-19 network is used to learn multi-level features from the two input feature combination images, that is, multi-level features are learned from the spectral feature combination image and the point cloud feature combination image, respectively, and the multi-level features are upsampled and feature selected layer by layer to generate a dual-channel rough tree crown width extraction result, which is fused with the high-dimensional semantic features to finally generate a fine tree crown width extraction result.

[0067] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for determining the tree crown width in the transmission line corridor of the present application will be described in detail below in combination with specific embodiments.

[0068] This embodiment relates to a specific method for determining the tree crown width in a transmission line corridor, comprising the following steps:

[0069] Step S1: Collect multi-source remote sensing data in the transmission corridor based on the UAV multi-sensor platform, including multi-spectral remote sensing images and lidar point cloud data;

[0070] Step S2: extracting band image features and vegetation index features from the multispectral remote sensing image, and superimposing the band image features and the vegetation index features to obtain a spectral feature combined image with the same resolution as the multispectral remote sensing image;

[0071] Step S3: extracting canopy height features, intensity features, and leaf area index features from the laser radar point cloud data, and superimposing the canopy height features, intensity features, and leaf area index features to obtain a point cloud feature combined image;

[0072] Step S4: Using the dual-channel target VGG-19 network, learn multi-level features from the spectral feature combination image and the point cloud feature combination image respectively, and upsample and select the multi-level features layer by layer to generate a dual-channel rough tree crown width extraction result;

[0073] Step S5: Fuse the rough crown width extraction results of the dual channels with the high-dimensional semantic features to finally generate a detailed tree crown width extraction result.

[0074] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0075] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is run, the device where the computer-readable storage medium is located is controlled to execute a method for determining tree crown widths in a transmission line corridor.

[0076] Specifically, the method for determining the tree crown width in the transmission line corridor includes:

[0077] Step S201, obtaining multi-source remote sensing data of the transmission line corridor, the multi-source remote sensing data including multi-spectral remote sensing images and laser radar point cloud data;

[0078] Step S202, extracting band image features and vegetation index features from the multispectral remote sensing image, the band image features represent the radiation energy data of each band of the multispectral remote sensing image, and the vegetation index features represent the vegetation index of the tree crown width in the transmission line corridor;

[0079] Step S203, superimposing the band image features and the vegetation index features to obtain a spectral feature combined image with the same resolution as the multispectral remote sensing image;

[0080] Step S204, extracting a canopy height feature, an intensity feature, and a leaf area index feature from the laser radar point cloud data, wherein the canopy height feature represents the height information of the tree canopy from the ground, the intensity feature represents the reflection intensity information of each point in the laser radar point cloud data, and the leaf area index feature represents the leaf area index of the tree canopy;

[0081] Step S205, superimposing the canopy height feature, the intensity feature and the leaf area index feature to obtain a point cloud feature combined image;

[0082] Step S206, respectively inputting the spectral feature combination image and the point cloud feature combination image into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown width extraction image, wherein the target VGG-19 network is trained by neural network learning using multiple sets of data, and each set of data in the multiple sets of data includes: a sample spectral feature combination image, a sample point cloud feature combination image, and a sample crown width extraction image.

[0083] Optionally, band image features and vegetation index features are extracted from the multispectral remote sensing image, including: performing radiation correction and geometric correction on the multispectral remote sensing image; and extracting band image features and vegetation index features from the corrected multispectral remote sensing image.

[0084] Optionally, band image features and vegetation index features are extracted from the corrected multispectral remote sensing image, including: extracting red light band data, green light band data, blue light band data and near-infrared light band data from the corrected multispectral remote sensing image to obtain band image features; and determining the normalized vegetation index and the photochemical vegetation index based on the band image features to obtain vegetation index features.

[0085] Optionally, the target VGG-19 network is obtained by improving the VGG-19 network, the target VGG-19 network has two channels, namely a first channel and a second channel, the first channel is used to input the spectral feature combination image, and the second channel is used to input the point cloud feature combination image, the activation function in the target VGG-19 network is a PReLU function, each channel includes two Dropout layers and at least seven convolution blocks arranged in sequence, the input end of one Dropout layer is connected to the output end of the fourth convolution block, and the input end of the other Dropout layer is connected to the output end of the fifth convolution block, the spectral feature combination image and the point cloud feature combination image are respectively input into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain the crown extraction image, including: inputting the spectral feature combination image into the target VGG- 19 network, so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and upsamples and selects the multi-level features in the spectral feature combination image layer by layer to obtain a first crown-width extracted image; the point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and upsamples and selects the multi-level features in the point cloud feature combination image layer by layer to obtain a second crown-width extracted image, and the first crown-width extracted image and the second crown-width extracted image are fused with high-dimensional semantic features to obtain a crown-width extracted image, wherein the high-dimensional semantic features are features containing semantic information extracted by the target VGG-19 network in the process of upsampling and selecting the multi-level features in the spectral feature combination image and the multi-level features in the point cloud feature combination image layer by layer.

[0086] Optionally, the spectral feature combination image is input into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and performs upsampling and feature selection on the multi-level features in the spectral feature combination image layer by layer and level by level to obtain a first crown extraction image, including: inputting the spectral feature combination image into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image to obtain seven first feature maps whose sizes are successively reduced by 1 / 2, the seven first feature maps contain feature information from low level to high level in the spectral feature combination image, and performing upsampling on the seven first feature maps in two stages And feature selection, wherein the two stages are respectively the first stage and the second stage, the first stage includes: performing convolution operations on the first five first feature maps and the last first feature map to generate six second feature maps with a feature dimension of 32, and processing the six second feature maps by upsampling and adding feature elements to obtain five third feature maps, the second stage includes: performing convolution operations on the five third feature maps to obtain five fourth feature maps with a feature dimension of 2, and processing the five fourth feature maps by upsampling and adding feature elements to obtain four fifth feature maps, and extracting the four fifth feature maps by using the first Softmax layer in the target VGG-19 network to obtain a first crown extraction image.

[0087] Optionally, the point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and performs upsampling and feature selection on the multi-level features in the point cloud feature combination image layer by layer and level by level to obtain a second crown width extraction image, and the first crown width extraction image and the second crown width extraction image are fused with the high-dimensional semantic features to obtain a crown width extraction image, wherein the high-dimensional semantic features are the multi-level features of the target VGG-19 network in the spectral feature combination image and The features containing semantic information are extracted in the process of upsampling and feature selection of multi-level features in the point cloud feature combination image layer by layer, including: inputting the point cloud feature combination image into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image to obtain seven sixth feature maps whose sizes are successively reduced by 1 / 2, the seven sixth feature maps contain low-level to high-level feature information in the point cloud feature combination image, and performing two-stage upsampling and feature selection on the seven sixth feature maps, Among them, the two stages are the third stage and the fourth stage, and the third stage includes: performing convolution operations on the first five sixth feature maps and the last sixth feature map to generate six seventh feature maps with a feature dimension of 32, and processing the six seventh feature maps by upsampling and adding feature elements to obtain five eighth feature maps. The fourth stage includes: performing convolution operations on the five eighth feature maps to obtain five ninth feature maps with a feature dimension of 2, and processing the five ninth feature maps by upsampling and adding feature elements to obtain four tenth feature maps, and extracting the four tenth feature maps using the second Softmax layer in the target VGG-19 network to obtain a second crown width extracted image, and upsampling the second-to-last fifth feature map to obtain a first high-dimensional semantic feature, and upsampling the second-to-last tenth feature map to obtain a second high-dimensional semantic feature, and feature fusion of the first high-dimensional semantic feature and the second high-dimensional semantic feature to obtain a high-dimensional semantic feature, and fusion of the first crown width extracted image and the second crown width extracted image with the high-dimensional semantic feature to obtain a crown width extracted image.

[0088] Optionally, the normalized vegetation index and the photochemical vegetation index are determined according to the band image characteristics, including: determining the normalized vegetation index according to the red light band data and the near-infrared light band data; and determining the photochemical vegetation index according to the green light band data.

[0089] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement at least the following method steps: step S201, obtaining multi-source remote sensing data of a transmission line corridor, the multi-source remote sensing data including multi-spectral remote sensing images and lidar point cloud data; step S202, extracting band image features and vegetation index features from the multi-spectral remote sensing images, the band image features representing the radiation energy data of each band of the multi-spectral remote sensing images, and the vegetation index features representing the vegetation index of the tree crown width in the transmission line corridor; step S203, superimposing the band image features and the vegetation index features to obtain a spectral feature combined image with the same resolution as the multi-spectral remote sensing image; step S204, extracting canopy height features, intensity features, and leaf area index features from the lidar point cloud data, and the canopy height features, intensity features, and leaf area index features from the lidar point cloud data. The layer height feature characterizes the height information of the tree crown from the ground, the intensity feature characterizes the reflection intensity information of each point in the lidar point cloud data, and the leaf area index feature characterizes the leaf area index of the tree crown; step S205, superimpose the crown height feature, intensity feature and leaf area index feature to obtain a point cloud feature combination image; step S206, respectively input the spectral feature combination image and the point cloud feature combination image into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown extraction image, and the target VGG-19 network is trained by neural network learning using multiple sets of data, and each set of data in the multiple sets of data includes: a sample spectral feature combination image, a sample point cloud feature combination image and a sample crown extraction image.

[0090] Optionally, band image features and vegetation index features are extracted from the multispectral remote sensing image, including: performing radiation correction and geometric correction on the multispectral remote sensing image; and extracting band image features and vegetation index features from the corrected multispectral remote sensing image.

[0091] Optionally, band image features and vegetation index features are extracted from the corrected multispectral remote sensing image, including: extracting red light band data, green light band data, blue light band data and near-infrared light band data from the corrected multispectral remote sensing image to obtain band image features; and determining the normalized vegetation index and the photochemical vegetation index based on the band image features to obtain vegetation index features.

[0092] Optionally, the target VGG-19 network is obtained by improving the VGG-19 network, the target VGG-19 network has two channels, namely a first channel and a second channel, the first channel is used to input the spectral feature combination image, and the second channel is used to input the point cloud feature combination image, the activation function in the target VGG-19 network is a PReLU function, each channel includes two Dropout layers and at least seven convolution blocks arranged in sequence, the input end of one Dropout layer is connected to the output end of the fourth convolution block, and the input end of the other Dropout layer is connected to the output end of the fifth convolution block, the spectral feature combination image and the point cloud feature combination image are respectively input into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain the crown extraction image, including: inputting the spectral feature combination image into the target VGG- 19 network, so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and upsamples and selects the multi-level features in the spectral feature combination image layer by layer to obtain a first crown-width extracted image; the point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and upsamples and selects the multi-level features in the point cloud feature combination image layer by layer to obtain a second crown-width extracted image, and the first crown-width extracted image and the second crown-width extracted image are fused with high-dimensional semantic features to obtain a crown-width extracted image, wherein the high-dimensional semantic features are features containing semantic information extracted by the target VGG-19 network in the process of upsampling and selecting the multi-level features in the spectral feature combination image and the multi-level features in the point cloud feature combination image layer by layer.

[0093] Optionally, the spectral feature combination image is input into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and performs upsampling and feature selection on the multi-level features in the spectral feature combination image layer by layer and level by level to obtain a first crown extraction image, including: inputting the spectral feature combination image into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image to obtain seven first feature maps whose sizes are successively reduced by 1 / 2, the seven first feature maps contain feature information from low level to high level in the spectral feature combination image, and performing upsampling on the seven first feature maps in two stages And feature selection, wherein the two stages are respectively the first stage and the second stage, the first stage includes: performing convolution operations on the first five first feature maps and the last first feature map to generate six second feature maps with a feature dimension of 32, and processing the six second feature maps by upsampling and adding feature elements to obtain five third feature maps, the second stage includes: performing convolution operations on the five third feature maps to obtain five fourth feature maps with a feature dimension of 2, and processing the five fourth feature maps by upsampling and adding feature elements to obtain four fifth feature maps, and extracting the four fifth feature maps by using the first Softmax layer in the target VGG-19 network to obtain a first crown extraction image.

[0094] Optionally, the point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and performs upsampling and feature selection on the multi-level features in the point cloud feature combination image layer by layer and level by level to obtain a second crown width extraction image, and the first crown width extraction image and the second crown width extraction image are fused with the high-dimensional semantic features to obtain a crown width extraction image, wherein the high-dimensional semantic features are the multi-level features of the target VGG-19 network in the spectral feature combination image and The features containing semantic information are extracted in the process of upsampling and feature selection of multi-level features in the point cloud feature combination image layer by layer, including: inputting the point cloud feature combination image into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image to obtain seven sixth feature maps whose sizes are successively reduced by 1 / 2, the seven sixth feature maps contain low-level to high-level feature information in the point cloud feature combination image, and performing two-stage upsampling and feature selection on the seven sixth feature maps, Among them, the two stages are the third stage and the fourth stage, and the third stage includes: performing convolution operations on the first five sixth feature maps and the last sixth feature map to generate six seventh feature maps with a feature dimension of 32, and processing the six seventh feature maps by upsampling and adding feature elements to obtain five eighth feature maps. The fourth stage includes: performing convolution operations on the five eighth feature maps to obtain five ninth feature maps with a feature dimension of 2, and processing the five ninth feature maps by upsampling and adding feature elements to obtain four tenth feature maps, and extracting the four tenth feature maps using the second Softmax layer in the target VGG-19 network to obtain a second crown width extracted image, and upsampling the second-to-last fifth feature map to obtain a first high-dimensional semantic feature, and upsampling the second-to-last tenth feature map to obtain a second high-dimensional semantic feature, and feature fusion of the first high-dimensional semantic feature and the second high-dimensional semantic feature to obtain a high-dimensional semantic feature, and fusion of the first crown width extracted image and the second crown width extracted image with the high-dimensional semantic feature to obtain a crown width extracted image.

[0095] Optionally, the normalized vegetation index and the photochemical vegetation index are determined according to the band image characteristics, including: determining the normalized vegetation index according to the red light band data and the near-infrared light band data; and determining the photochemical vegetation index according to the green light band data.

[0096] An embodiment of the present application also provides an electronic device, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the methods for determining the tree crown width in a transmission corridor.

[0097] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0098] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0103] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0104] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0105] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0106] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0107] In the method for determining the tree crown width in the transmission line corridor of the present application, multi-source remote sensing data of the transmission line corridor including multispectral remote sensing images and lidar point cloud data are first obtained, and then band image features and vegetation index features are extracted from the multispectral remote sensing images, and the band image features are superimposed with the vegetation index features to obtain a spectral feature combination image with the same resolution as the multispectral remote sensing image, and then canopy height features, intensity features and leaf area index features are extracted from the lidar point cloud data, and the canopy height features, intensity features and leaf area index features are superimposed to obtain a point cloud feature combination image, and the spectral feature combination image and the point cloud feature combination image are respectively input into a dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown width extraction image. Compared with the deep learning methods in the prior art that focus on using a single data source to extract tree crowns, resulting in difficulty in comprehensively and accurately extracting tree crowns when dealing with complex and changeable natural environments, the present application obtains multispectral remote sensing images and lidar point cloud data, utilizes the data advantages of multispectral data and point cloud data, and obtains spectral feature combination images and point cloud feature combination images through certain extraction and overlay processing. Finally, the spectral feature combination images and point cloud feature combination images are respectively input into the dual-channel target VGG-19 network to obtain crown extraction images, making full use of the data advantages of various types of data, combining spectral information and three-dimensional structure information, and ensuring that the extraction of tree crowns is relatively accurate.

[0108] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining the tree crown width in a transmission line corridor, characterized in that: include: Acquire multi-source remote sensing data of the transmission line corridor, wherein the multi-source remote sensing data includes multi-spectral remote sensing images and lidar point cloud data; Extracting band image features and vegetation index features from the multispectral remote sensing image, wherein the band image features represent the radiation energy data of each band of the multispectral remote sensing image, and the vegetation index features represent the vegetation index of the tree crown width in the transmission line corridor; Overlaying the band image features with the vegetation index features to obtain a spectral feature combined image having the same resolution as the multispectral remote sensing image; Extracting a canopy height feature, an intensity feature, and a leaf area index feature from the laser radar point cloud data, wherein the canopy height feature represents the height information of the tree canopy from the ground, the intensity feature represents the reflection intensity information of each point in the laser radar point cloud data, and the leaf area index feature represents the leaf area index of the tree canopy; Superimposing the canopy height feature, the intensity feature and the leaf area index feature to obtain a point cloud feature combined image; The spectral feature combination image and the point cloud feature combination image are respectively input into a dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown width extraction image. The target VGG-19 network is trained by neural network learning using multiple sets of data, and each set of data in the multiple sets of data includes: a sample spectral feature combination image, a sample point cloud feature combination image, and a sample crown width extraction image.

2. The method for determining the tree crown width in the transmission line corridor according to claim 1, characterized in that: Extracting band image features and vegetation index features from the multispectral remote sensing image includes: Performing radiation correction and geometric correction on the multispectral remote sensing image; The band image features and the vegetation index features are extracted from the corrected multispectral remote sensing image.

3. The method for determining the tree crown width in the transmission line corridor according to claim 2, characterized in that: Extracting the band image features and the vegetation index features from the corrected multispectral remote sensing image includes: Extracting red light band data, green light band data, blue light band data and near infrared light band data from the corrected multispectral remote sensing image to obtain the band image features; According to the band image characteristics, the normalized vegetation index and the photochemical vegetation index are determined to obtain the vegetation index characteristics.

4. The method for determining the tree crown width in the transmission line corridor according to claim 1, characterized in that: The target VGG-19 network is obtained by improving the VGG-19 network. The target VGG-19 network has two channels, namely a first channel and a second channel. The first channel is used to input the spectral feature combination image, and the second channel is used to input the point cloud feature combination image. The activation function in the target VGG-19 network is a PReLU function. Each of the channels includes two Dropout layers and at least seven convolution blocks arranged in sequence, wherein the input end of one of the Dropout layers is connected to the output end of the fourth convolution block, and the input end of the other Dropout layer is connected to the output end of the fifth convolution block. The spectral feature combination image and the point cloud feature combination image are respectively input into the dual-channel target VGG-19 network, so that the target VGG-19 network analyzes the spectral feature combination image and the point cloud feature combination image to obtain a crown width extraction image, including: Inputting the spectral feature combination image into the target VGG-19 network through the first channel, so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and performs layer-by-layer and level-by-level upsampling and feature selection on the multi-level features in the spectral feature combination image to obtain a first crown extraction image; The point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and upsamples and selects the multi-level features in the point cloud feature combination image layer by layer and level by level to obtain a second crown extraction image, and the first crown extraction image and the second crown extraction image are fused with high-dimensional semantic features to obtain the crown extraction image, wherein the high-dimensional semantic features are features containing semantic information extracted by the target VGG-19 network in the process of upsampling and feature selecting the multi-level features in the spectral feature combination image and the multi-level features in the point cloud feature combination image layer by layer and level by level.

5. The method for determining the tree crown width in the transmission line corridor according to claim 4, characterized in that: Inputting the spectral feature combination image into the target VGG-19 network through the first channel so that the target VGG-19 network learns the multi-level features in the spectral feature combination image, and performs layer-by-layer and level-by-level upsampling and feature selection on the multi-level features in the spectral feature combination image to obtain a first crown-width extracted image, including: The spectral feature combination image is input into the target VGG-19 network through the first channel, so that the target VGG-19 network learns the multi-level features in the spectral feature combination image to obtain seven first feature maps with sizes reduced by 1 / 2 in sequence, the seven first feature maps contain feature information from low level to high level in the spectral feature combination image, and two stages of upsampling and feature selection are performed on the seven first feature maps, wherein the two stages are respectively the first stage and the second stage, and the first stage includes: upsampling the first five first feature maps and the last first feature map The first stage comprises: performing a convolution operation on the five third feature maps to obtain five fourth feature maps with a feature dimension of 2, and processing the five fourth feature maps by upsampling and adding feature elements to obtain four fifth feature maps, and extracting the four fifth feature maps by using the first Softmax layer in the target VGG-19 network to obtain the first crown extraction image.

6. The method for determining the tree crown width in the transmission line corridor according to claim 5, characterized in that: The point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image, and performs layer-by-layer and level-by-level upsampling and feature selection on the multi-level features in the point cloud feature combination image to obtain a second crown width extraction image, and the first crown width extraction image and the second crown width extraction image are fused with high-dimensional semantic features to obtain the crown width extraction image, wherein the high-dimensional semantic features are features containing semantic information extracted by the target VGG-19 network in the process of performing layer-by-layer and level-by-level upsampling and feature selection on the multi-level features in the spectral feature combination image and the multi-level features in the point cloud feature combination image, including: The point cloud feature combination image is input into the target VGG-19 network through the second channel, so that the target VGG-19 network learns the multi-level features in the point cloud feature combination image to obtain seven sixth feature maps whose sizes are reduced by 1 / 2 in sequence, and the seven sixth feature maps contain feature information from low level to high level in the point cloud feature combination image, and the seven sixth feature maps are subjected to two stages of upsampling and feature selection, wherein the two stages are respectively the third stage and the fourth stage, and the third stage includes: performing convolution operation on the first five sixth feature maps and the last sixth feature map to generate six seventh feature maps with a feature dimension of 32, and processing the six seventh feature maps by upsampling and adding feature elements to obtain five eighth feature maps, and the fourth stage includes: performing convolution operation on the five A convolution operation is performed on the eighth feature maps to obtain five ninth feature maps with a feature dimension of 2, and the five ninth feature maps are processed by upsampling and adding feature elements to obtain four tenth feature maps, and the four tenth feature maps are extracted by using the second Softmax layer in the target VGG-19 network to obtain the second crown-width extracted image, and the second-to-last fifth feature map is upsampled to obtain a first high-dimensional semantic feature, and the second-to-last tenth feature map is upsampled to obtain a second high-dimensional semantic feature, and the first high-dimensional semantic feature and the second high-dimensional semantic feature are feature fused to obtain the high-dimensional semantic feature, and the first crown-width extracted image and the second crown-width extracted image are fused with the high-dimensional semantic feature to obtain the crown-width extracted image.

7. The method for determining the tree crown width in the transmission line corridor according to claim 3, characterized in that: According to the band image characteristics, the normalized vegetation index and the photochemical vegetation index are determined, including: Determining the normalized vegetation index according to the red light band data and the near infrared light band data; The photochemical vegetation index is determined according to the green light band data.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for determining the tree crown width in the transmission line corridor according to any one of claims 1 to 7.

9. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by the processor, the method for determining the tree crown width in the transmission line corridor as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for determining the tree crown width in the transmission line corridor as described in any one of claims 1 to 7.

Images