A forest vegetation detection method and system based on a UAV

Abstract

The present application relates to the technical field of forest detection, and specifically discloses a forest vegetation detection method and system based on a UAV, which collects visible light images, spectral images and radar data of a forest area through a UAV carrying a visible light camera, a spectral camera and a radar, constructs a three-dimensional model of a tree based on the radar data, determines tree crown data and tree species of the tree based on the visible light images and the spectral images, and determines the volume, biomass and carbon storage of the tree based on the three-dimensional model, the tree species and the tree crown data, so as to calculate the total volume, total biomass and total carbon storage of all trees in the target forest area, without manually inspecting and sampling tree data, thereby shortening the time consumption of forest detection, improving the detection efficiency, and calculating the volume, biomass and carbon storage of each tree through single-tree modeling combined with visible light images and spectral images, and then summing up the total volume, total biomass and total carbon storage of the entire target forest area, so that the detection result is more accurate.

Description

Technical Field

[0001] This invention belongs to the field of forest tree detection technology, and in particular relates to a method and system for detecting forest vegetation based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Forests are the foundation for the sustained, rapid, and healthy development of the national economy, and have three major benefits: ecological, economic, and social. They are the link and key between ecology and economy. It is crucial to test forest vegetation (especially trees) to determine the timber volume (timber volume), biomass (mass of living trees), and carbon storage of forests.

[0003] Traditional forest monitoring relies heavily on ground sampling and manual inspections, such as manually measuring the diameter and height of trees in the forest. This data sampling process is lengthy and inefficient, and it is difficult to manually measure the canopy data of trees, resulting in an inability to accurately measure the timber volume, biomass, and carbon storage of the forest. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for forest vegetation detection based on unmanned aerial vehicles (UAVs), which aims to solve the problems of long detection time, low efficiency and inability to accurately detect relevant forest data indicators in traditional forest detection methods.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A method for detecting forest vegetation based on unmanned aerial vehicles (UAVs) specifically includes the following steps:

[0007] Control the drone equipped with a visible light camera, a spectral camera, and radar to collect visible light images, spectral images of the forest area, and radar data of the target forest area;

[0008] A three-dimensional model of the trees in the target forest area is constructed based on the radar data;

[0009] Based on the visible light image and the forest area spectral image, determine the canopy data and tree species of the trees in the target forest area;

[0010] The volume, biomass, and carbon storage of the trees are determined based on the three-dimensional model, tree species, and canopy data of the trees.

[0011] The sum of the volume, biomass, and carbon storage of all trees in the target forest area is calculated to obtain the total volume, total biomass, and total carbon storage of the target forest area.

[0012] As a further technical solution of the present invention, controlling a drone equipped with a visible light camera, a spectral camera, and radar to collect visible light images, spectral images of the forest area, and radar data of the target forest area specifically includes the following steps:

[0013] The drone is controlled to fly along a preset flight path, which has multiple sampling points. The flight path is set according to the terrain of the target forest area and a preset image overlap rate.

[0014] When the drone flies to the sampling point, control the visible light camera on the drone to collect visible light images of the target forest area, and control the spectral camera on the drone to collect spectral images of the target forest area;

[0015] While the UAV is flying along a preset flight path, the radar is controlled to collect radar data from the target forest area.

[0016] As a further technical solution of the present invention, a three-dimensional model of trees in the target forest area is constructed based on the radar data, specifically including the following steps:

[0017] The radar point cloud data of each tree is determined from the radar data;

[0018] A 3D model of each tree is constructed based on the radar point cloud data of each tree, and the first coordinates of each tree are determined and associated with the 3D model.

[0019] As a further technical solution of the present invention, determining the canopy data and tree species of trees in the target forest area based on the visible light image and the forest area spectral image specifically includes the following steps:

[0020] The visible light image is input into the tree canopy detection model to obtain tree canopy data, which includes a second coordinate and the canopy perimeter, canopy width, canopy coverage and a first tree species associated with the second coordinate;

[0021] The third coordinate of each tree and the second tree species associated with the third coordinate are determined based on the spectral image of the forest area.

[0022] The first and second tree species of the same tree are determined by the second and third coordinates;

[0023] Determine whether the first tree species and the second tree species of the same tree are the same tree species;

[0024] If so, the first tree species or the second tree species shall be used as the tree species of the tree;

[0025] If not, obtain ground-measured data of the trees and determine the tree species based on the ground-measured data;

[0026] The second coordinate is associated with the canopy data and the tree species.

[0027] As a further technical solution of the present invention, the canopy detection model includes a canopy segmentation sub-model and a canopy recognition sub-model. The step of inputting the visible light image into the canopy detection model to obtain the canopy data of the tree specifically includes the following steps:

[0028] The visible light image is input into the canopy segmentation sub-model to obtain the canopy region of each tree in the visible light image;

[0029] The second coordinates of the tree are determined by the image position of the canopy region in the visible light image;

[0030] The canopy region is input into the canopy recognition sub-model to obtain the tree's canopy perimeter, canopy width, canopy coverage, and first tree species;

[0031] The second coordinate is associated with the crown perimeter, crown width, crown coverage, and tree species.

[0032] As a further technical solution of the present invention, determining the third coordinate of each tree and the second tree species associated with the third coordinate based on the spectral image of the forest area specifically includes the following steps:

[0033] The tree spectral image of each tree is segmented from the spectral image of the forest area, and the third coordinate of each tree is determined based on the position of the tree spectral image in the spectral image of the forest area.

[0034] The tree spectral image of each tree is matched with spectral image samples of different tree species in a pre-configured spectral image tree species library to obtain the tree species that match the tree spectral image as the second tree species, and the third coordinate is associated with the second tree species.

[0035] As a further technical solution of the present invention, the volume, biomass, and carbon storage of the tree are determined based on the three-dimensional model, tree species, and canopy data of the tree, specifically including the following steps:

[0036] Calculate the distance between the first coordinate and the second coordinate, and determine the target first coordinate and the target second coordinate where the distance is less than a threshold. The target first coordinate and the target second coordinate are the coordinates of the same tree.

[0037] The volume of the tree is calculated based on the three-dimensional model associated with the first coordinate of the target.

[0038] The biomass of the tree is inverted based on the canopy data and tree species associated with the second coordinate of the target and the three-dimensional model associated with the first coordinate of the target.

[0039] The carbon storage of the tree is calculated based on the biomass and the tree species associated with the target second coordinate.

[0040] As a further technical solution of the present invention, the biomass of the tree is inverted based on the canopy data and tree species associated with the second coordinate of the target and the three-dimensional model associated with the first coordinate of the target, specifically including the following steps:

[0041] The biomass W of each tree is calculated using the following equation:

[0042] ;

[0043] Wherein, a, b, and c are parameters determined in a pre-configured parameter table based on tree species and crown data. The parameter table is a mapping table of tree species, crown, and parameters generated based on ground measurement data. D is the diameter at breast height of the tree determined based on the three-dimensional model, and H is the tree height determined based on the three-dimensional model.

[0044] As a further technical solution of the present invention, the carbon storage of the tree is calculated based on the biomass and the tree species, specifically including the following steps:

[0045] Obtain the carbon content factor corresponding to the tree species, wherein the carbon content factor is the carbon content factor of the tree species corresponding to the tree determined based on ground measurement data;

[0046] The carbon storage of the tree is calculated using the biomass and the carbon content factor.

[0047] A forest vegetation detection system based on unmanned aerial vehicles (UAVs) specifically includes the following units:

[0048] The data acquisition unit is used to control the UAV equipped with a visible light camera, a spectral camera, and radar to acquire visible light images, spectral images of the forest area, and radar data of the target forest area.

[0049] A three-dimensional model building unit is used to build a three-dimensional model of the trees in the target forest area based on the radar data;

[0050] The canopy data and tree species determination unit is used to determine the canopy data and tree species of trees in the target forest area based on the visible light image and the forest area spectral image;

[0051] A single tree data calculation unit is used to determine the volume, biomass, and carbon storage of the tree based on its three-dimensional model, tree species, and canopy data.

[0052] The total forest area data calculation unit is used to calculate the sum of the volume, biomass, and carbon storage of all trees in the target forest area, and obtain the total volume, total biomass, and total carbon storage of the target forest area.

[0053] Compared with the prior art, the beneficial effects of the present invention are:

[0054] This invention utilizes a drone equipped with a visible light camera, a spectral camera, and a lidar to collect visible light images, spectral images, and radar data from a target forest area. Based on the radar data, a 3D model of the trees within the target forest area is constructed. Furthermore, based on the visible light images and the forest area's spectral images, the canopy data and tree species of the trees in the target forest area are determined. Further, based on the 3D model of the trees, the tree species, and the canopy data, the volume, biomass, and carbon storage of the trees are determined. Finally, the total volume, total biomass, and total carbon storage of all trees in the target forest area are summed to calculate the total volume, total biomass, and total carbon storage. This eliminates the need for manual inspection and sampling of tree diameter, height, and other data. By using a drone equipped with a visible light camera, a spectral camera, and a lidar to collect data, the time required for forest detection is shortened, efficiency is improved, and the ability to model individual trees and combine visible light and spectral images to calculate the volume, biomass, and carbon storage of each tree is achieved. Finally, the total volume, total biomass, and total carbon storage of all trees are calculated to obtain the total volume, total biomass, and total carbon storage of the entire target forest area, resulting in more accurate detection results. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0056] Figure 1 A flowchart of a UAV-based forest vegetation detection method according to an embodiment of the present invention is shown.

[0057] Figure 2 This is a schematic diagram of a tree canopy in a visible light image;

[0058] Figure 3 The diagram illustrates the application architecture of a drone-based forest vegetation detection system according to an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] Figure 1 A flowchart of a UAV-based forest vegetation detection method according to an embodiment of the present invention is shown. Specifically, the UAV-based forest vegetation detection method according to an embodiment of the present invention includes the following steps:

[0061] Step S101: Control the UAV equipped with a visible light camera, a spectral camera and radar to collect visible light images, spectral images of the forest area and radar data of the target forest area.

[0062] In this embodiment, the drone can be an agricultural drone or a surveying drone. The drone is equipped with a gimbal that can carry a camera and radar. A visible light camera, a spectral camera, and radar can be installed on the gimbal. The visible light camera is a camera used to collect visible light images, and the collected visible light images are the images seen by the human eye. The spectral camera is used to collect spectral images. The radar can be a millimeter-wave radar that transmits millimeter waves to penetrate the surface of vegetation canopy and reach the ground to collect three-dimensional point clouds of trees as radar data.

[0063] In this embodiment, the flight trajectory can be planned first according to the terrain of the forest area and the overlap rate requirements of the acquired images. For example, the flight altitude can be set to 100 meters (this is just an example and can be determined according to the terrain and tree height). The overlap rate of the forward acquisition images is 85%, and the lateral overlap rate is 85%. After planning the flight trajectory including multiple sampling points, the UAV can be controlled to fly according to the preset flight trajectory. When the UAV flies to the sampling point, the visible light camera on the UAV is controlled to acquire visible light images of the target forest area, and the spectral camera on the UAV is controlled to acquire spectral images of the target forest area. Thus, visible light images and spectral images with a forward overlap rate of 85% and a lateral overlap rate of 85% are obtained. During the flight, the radar is controlled to transmit radar signals below the target forest area, and radar data is generated after receiving the radar signals reflected by the target forest area.

[0064] The visible light images collected at each sampling point are stitched together, arranged, and the overlapping areas are merged according to the order of the sampling points to obtain a panoramic visible light image and a panoramic spectral image of the entire target forest area.

[0065] Step S102: Construct a three-dimensional model of the trees in the target forest area based on radar data.

[0066] Radar data can be point cloud data. Radar point cloud data for each tree can be determined from the radar data. A 3D model of each tree can be constructed based on its radar point cloud data, and the first coordinates of each tree can be determined. For example, a point cloud image containing the point clouds of multiple trees can be used, and after labeling the point cloud of each tree, a tree point cloud segmentation model can be trained using a supervised method. This allows the tree point cloud segmentation model to learn the ability to identify and segment the point cloud of a single tree from the point cloud. After collecting the point cloud of the entire forest area, the point cloud of the entire forest area can be input into the tree point cloud segmentation model to obtain the point cloud of each individual tree. The world coordinates of each tree are calculated based on its position within the entire point cloud and its position when the UAV collected the point cloud data, thus obtaining the first coordinates. A 3D model is then generated using the point cloud of each tree, and the first coordinates are associated with the 3D model to locate the 3D model of each tree using the first coordinates. The method of generating a 3D model from the point cloud can refer to existing technologies and will not be detailed here.

[0067] Step S103: Determine the canopy data and tree species of trees in the target forest area based on the visible light image and the forest area spectral image.

[0068] In this embodiment, the canopy data can be data describing the geometry of the canopy of each tree, and the tree species can be the type of tree. In an optional embodiment, step S103 may include the following sub-steps:

[0069] Sub-step S1031: Input the visible light image into the tree canopy detection model to obtain the tree canopy data. The tree canopy data includes the second coordinate and the canopy perimeter, canopy width, canopy coverage and the first tree species associated with the second coordinate.

[0070] This embodiment can pre-train a canopy detection model, which is used to identify canopy regions in visible light images, and to identify tree species and estimate canopy perimeter, canopy width, and canopy coverage based on the canopy regions.

[0071] For example, the canopy detection model may include a canopy segmentation sub-model and a canopy recognition sub-model. It can acquire canopy sample images including various types of tree species, and label the canopy region, tree species, canopy perimeter, canopy width, and canopy coverage in the canopy sample images. Then, the canopy sample images are input into the canopy segmentation sub-model to segment and estimate the canopy region. The segmented and estimated canopy region is input into the canopy recognition sub-model to estimate the tree species, canopy perimeter, canopy width, and canopy coverage. Then, the total loss value is calculated using the labeled and estimated canopy regions, tree species, canopy perimeter, canopy width, and canopy coverage. The model parameters are adjusted using the total loss value and iteratively trained until the total loss value is less than a preset threshold.

[0072] For example, the canopy segmentation loss value (such as mean squared error loss) is calculated using labeled and estimated canopy regions, and the canopy attribute loss value is calculated using labeled and estimated tree species, canopy perimeter, canopy width, and canopy coverage. For instance, first calculate the loss values ​​(such as mean squared error) for tree species, canopy perimeter, canopy width, and canopy coverage, then calculate the weighted average of the losses as the canopy attribute loss value, and calculate the sum of the canopy segmentation loss value and the canopy attribute loss value as the total loss value. When adjusting the model parameters, the parameters of the canopy segmentation sub-model are subjected to gradient descent using the canopy segmentation loss value, and the parameters of the canopy recognition sub-model are subjected to gradient descent using the canopy attribute loss value.

[0073] After the canopy detection model is trained, the visible light images captured by the visible light camera on the drone can be input into the canopy segmentation sub-model to obtain the canopy region of each tree in the visible light image. The second coordinate of the tree is determined by the image position of the canopy region in the visible light image. The canopy region is input into the canopy recognition sub-model to obtain the canopy perimeter, canopy width, canopy coverage and the first tree species of the tree. The second coordinate is associated with the canopy perimeter, canopy width, canopy coverage and the first tree species, where the first tree species represents the tree species predicted by the visible light image.

[0074] Figure 2 A schematic diagram of canopy data, such as Figure 2 As shown, each tree canopy is associated with a second coordinate (x, y), the tree species, and data such as the canopy perimeter, canopy width, and canopy coverage associated with the coordinates. The second coordinate can be the geometric center of the canopy's top view outline, such as latitude and longitude coordinates. The first tree species can be the type of tree to which the canopy belongs. For each tree canopy, the canopy perimeter, canopy width, and canopy coverage can be associated with the second coordinate of the canopy and stored in the database.

[0075] Sub-step S1032: Determine the third coordinate of each tree and the second tree species associated with the third coordinate based on the spectral image of the forest area.

[0076] Specifically, the tree spectral image of each tree can be segmented from the forest spectral image, and the third coordinate of each tree can be determined based on the position of the tree spectral image in the forest spectral image. The tree spectral image of each tree can be matched with the spectral image samples of different tree species in a pre-configured spectral image tree species library to obtain the tree species that match the tree spectral image as the second tree species, and the third coordinate is associated with the second tree species.

[0077] A spectral image is an image formed by light reflected from the surface of a tree. The spectral image mainly depends on the various pigments in the tree, especially chlorophyll in the leaves. Different types of trees contain different amounts of various pigments. Tree species can be identified through spectral images. For example, a spectral image segmentation model can be trained first. This spectral image segmentation model can segment the spectral image of each tree from the spectral image of the forest area. The spectral image segmentation model can be obtained through supervised training of spectral image samples with spectral image regions of each tree labeled. This will not be described in detail here.

[0078] This embodiment can pre-construct a tree species library, which includes spectral images of various types of trees. The spectral image of each segmented tree can be matched with the spectral images in the tree species library. For example, spectral indices such as NDVI (Normalized Difference Vegetation Index) and RVI (Relative Vegetation Index) can be calculated from the tree spectral images. The similarity between the spectral indices of the tree spectral images and the spectral indices of the spectral images of each tree species in the tree species library is calculated. The tree species corresponding to the spectral image with the highest similarity is determined as the second tree species. The third coordinate of the tree is determined according to the position of the tree spectral image in the forest area spectral image (e.g., converting the image coordinates to camera coordinates based on imaging principles, and then converting the camera coordinates to world coordinates). The third coordinate is then associated with the second tree species.

[0079] Sub-step S1033: Determine the first and second tree species of the same tree using the second and third coordinates.

[0080] In this embodiment, the first tree species of each tree is determined by the visible light image, and the second tree species of each tree is determined by the spectral image. For the same tree, there may be cases where the first tree species and the second tree species are the same or different. The same tree can be determined first by the second coordinate and the third coordinate. For example, the distance between the second coordinate and the third coordinate can be calculated. If the distance is less than a threshold, the trees at the second coordinate and the third coordinate are determined to be the same tree. Furthermore, by obtaining the first tree species associated with the second coordinate and the second tree species associated with the third coordinate of the same tree, the tree species of the same tree identified by the visible light image and the spectral image are obtained respectively.

[0081] Sub-step S1034: Determine whether the first and second tree species of the same tree are the same tree species.

[0082] Specifically, after obtaining the first and second tree species of the same tree through visible light images and spectral images, it can be determined whether the first tree species determined by the visible light image and the second tree species determined by the spectral image are the same. If yes, sub-step S1035 is executed; if no, sub-step S1036 is executed.

[0083] Sub-step S1035: Select the first tree species or the second tree species as the tree species.

[0084] If the first tree species identified by the visible light image and the second tree species identified by the spectral image are the same, it can be determined that the tree species identified by the visible light image and the spectral image is correct.

[0085] Sub-step S1036: Obtain ground measurement data of the trees and determine the tree species based on the ground measurement data.

[0086] If the first tree species determined by the visible light image and the second tree species determined by the spectral image are different, it can be determined that the tree species determined by the visible light image and the spectral image may be incorrect. Ground measurement data of the tree can be obtained. This ground measurement data can be data that is measured and determined manually on the trees in the target forest area. This ground measurement data can include the tree species confirmed by humans. The tree species in the ground measurement data is taken as the final tree species, and the tree species is associated with the second coordinate of the tree.

[0087] This invention uses both visible light and spectral images to identify tree species, and further uses ground-based measurement data to determine the final tree species when the identified species differs. This improves the accuracy of tree species identification and enables accurate calculation of tree biomass and carbon storage through accurate tree species inversion.

[0088] Step S104: Determine the tree's volume, biomass, and carbon storage based on the tree's three-dimensional model, tree species, and canopy data.

[0089] In this embodiment, the volume can be the volume of the tree's branches and trunk, i.e., the volume of the wood; the biomass can refer to the mass of the tree; and the carbon storage can be the carbon content of the entire tree. In an optional embodiment, step S104 may include the following sub-steps:

[0090] Sub-step S1041: Calculate the distance between the first coordinate and the second coordinate, and determine the target first coordinate and the target second coordinate where the distance is less than the threshold. The target first coordinate and the target second coordinate are the coordinates of the same tree.

[0091] In this embodiment, the first coordinate is associated with the 3D model of each tree, and the second coordinate is associated with the crown data and tree species of each tree. In order to obtain the 3D model, crown data and tree species of the same tree, the distance between the first coordinate and the second coordinate can be calculated. For example, the distance between each first coordinate and each second coordinate is calculated, and the target first coordinate and target second coordinate with a distance less than a threshold are determined. The 3D model associated with the target first coordinate and the crown data and tree species associated with the target second coordinate are the 3D model, crown data and tree species of the same tree.

[0092] Sub-step S1042: Calculate the tree volume based on the three-dimensional model associated with the first coordinate of the target.

[0093] Since the volume of timber is the volume of usable wood in a tree, 3D software can be used to calculate the volume of a 3D model. For example, the volume of the part of the 3D model excluding the leaves can be calculated as the volume of the tree, or the volume of only the trunk of the tree can be calculated as the volume of the tree based on the 3D model.

[0094] Sub-step S1043: Based on the tree canopy data associated with the second coordinate of the target and the three-dimensional model associated with the tree species and the first coordinate of the target, the biomass of the tree is inverted.

[0095] This embodiment can pre-configure the biomass inversion equation for trees. This biomass inversion equation can be determined based on statistical analysis of measured biomass and canopy data of different tree species. In an optional embodiment, the biomass W of each tree can be calculated using the following equation:

[0096] ;

[0097] Where a, b, and c are parameters determined in a pre-configured parameter table based on tree species and crown data. The parameter table is a mapping table of tree species, crown data, and parameters generated based on ground measurement data. D is the diameter at breast height of the tree determined based on the three-dimensional model, and H is the tree height determined based on the three-dimensional model.

[0098] For example, the diameter at breast height (DBH) and height of the tree can be determined in the 3D model first. Then, the corresponding parameter table can be obtained according to the tree species. Further, parameters a, b, and c can be matched according to the crown data in the parameter table of the tree species. Here, a can be the reference density of the wood corresponding to the tree species (the average density of various trees after ground measurement), and b and c can be the influencing factors when the tree of this species grows to a DBH of D, a height of H, and various crowns (crown perimeter, crown width, and canopy).

[0099] Sub-step S1044: Calculate the carbon storage of trees based on biomass and the tree species associated with the target second coordinate.

[0100] Specifically, the carbon content factor corresponding to the tree species can be obtained. This carbon content factor is determined based on ground-measured data of the tree species and the corresponding tree. The carbon storage of the tree is calculated using biomass and the carbon content factor. For example, in one example, the product of biomass and carbon content factor can be calculated as the carbon storage. In another example, carbon storage inversion equations for different tree species can be fitted using ground-measured data of various tree species. These carbon storage inversion equations use biomass as the independent variable and carbon storage as the dependent variable.

[0101] Step S105: Calculate the sum of the volume, biomass and carbon storage of all trees in the target forest area to obtain the total volume, total biomass and total carbon storage of the target forest area.

[0102] After calculating the volume, biomass, and carbon storage of each tree, the total volume, biomass, and carbon storage of the target forest area are obtained by calculating the volume, biomass, and carbon storage of all trees detected in the target forest area.

[0103] Furthermore, a forest vegetation monitoring report can be generated. This report can display the total stock volume, total biomass, and total carbon storage of the target forest area in a list. It can also display the total stock volume, total biomass, and total carbon storage of various tree species, as well as the coordinate distribution of different tree species, or the total stock volume, total biomass, and total carbon storage of various areas in the target forest area.

[0104] This invention utilizes a drone equipped with a visible light camera, a spectral camera, and a lidar to collect visible light images, spectral images, and radar data from a target forest area. Based on the radar data, a 3D model of the trees within the target forest area is constructed. Furthermore, based on the visible light images and the forest area's spectral images, the canopy data and tree species of the trees in the target forest area are determined. Further, based on the 3D model of the trees, the tree species, and the canopy data, the volume, biomass, and carbon storage of the trees are determined. Finally, the total volume, total biomass, and total carbon storage of all trees in the target forest area are summed to calculate the total volume, total biomass, and total carbon storage. This eliminates the need for manual inspection and sampling of tree diameter, height, and other data. By using a drone equipped with a visible light camera, a spectral camera, and a lidar to collect data, the time required for forest detection is shortened, efficiency is improved, and the ability to model individual trees and combine visible light and spectral images to calculate the volume, biomass, and carbon storage of each tree is achieved. Finally, the total volume, total biomass, and total carbon storage of all trees are calculated to obtain the total volume, total biomass, and total carbon storage of the entire target forest area, resulting in more accurate detection results.

[0105] Figure 3 The diagram illustrates the application architecture of a drone-based forest vegetation detection system according to an embodiment of the present invention. This drone-based forest vegetation detection system specifically includes the following units:

[0106] The data acquisition unit 301 is used to control the UAV equipped with a visible light camera, a spectral camera and radar to acquire visible light images, spectral images of the forest area and radar data of the target forest area.

[0107] The three-dimensional model building unit 302 is used to build a three-dimensional model of the trees in the target forest area based on the radar data;

[0108] The canopy data and tree species determination unit 303 is used to determine the canopy data and tree species of trees in the target forest area based on the visible light image and the forest area spectral image;

[0109] The single tree data calculation unit 304 is used to determine the volume, biomass, and carbon storage of the tree based on the tree's three-dimensional model, tree species, and canopy data.

[0110] The total forest area data calculation unit 305 is used to calculate the sum of the volume, biomass and carbon storage of all trees in the target forest area, and obtain the total volume, total biomass and total carbon storage of the target forest area.

[0111] As a further technical solution of this embodiment of the invention, the data acquisition unit 301 specifically includes the following modules:

[0112] The flight control module is used to control the UAV to fly according to a preset flight trajectory, which is set with multiple sampling points. The flight trajectory is set according to the terrain of the target forest area and the preset image overlap rate.

[0113] The image acquisition module is used to control the visible light camera on the UAV to acquire visible light images of the target forest area when the UAV flies to the sampling point, and to control the spectral camera on the UAV to acquire spectral images of the target forest area.

[0114] The radar data acquisition module is used to control the radar to acquire radar data of the target forest area while the UAV is flying along a preset flight trajectory.

[0115] As a further technical solution of this embodiment of the invention, the three-dimensional model construction unit 302 specifically includes the following modules:

[0116] A point cloud data segmentation module is used to determine the radar point cloud data of each tree from the radar data;

[0117] The 3D modeling module is used to construct a 3D model of each tree based on radar point cloud data of each tree, determine the first coordinates of each tree, and associate the first coordinates with the 3D model.

[0118] As a further technical solution of this invention, the tree canopy data and tree species determination unit 303 specifically includes the following modules:

[0119] The tree crown data and first tree species identification module is used to input the visible light image into the tree crown detection model to obtain the tree crown data, wherein the tree crown data includes a second coordinate and the crown perimeter, crown width, crown coverage and first tree species associated with the second coordinate;

[0120] The second tree species identification module is used to determine the third coordinate of each tree and the second tree species associated with the third coordinate based on the spectral image of the forest area;

[0121] A tree identification module is used to determine the first and second tree species of the same tree using the second and third coordinates;

[0122] The tree species identification module is used to determine whether the first tree species and the second tree species of the same tree are the same tree species; if yes, the tree species determination module is executed; if no, the ground-measured tree species determination module is executed.

[0123] A tree species determination module is used to identify the first tree species or the second tree species as the tree species of the tree;

[0124] The ground-measured tree species determination module is used to acquire ground-measured data of the trees and determine the tree species based on the ground-measured data.

[0125] The data association module is used to associate the second coordinates with the canopy data and the tree species.

[0126] As a further technical solution of this invention, the canopy detection model includes a canopy segmentation sub-model and a canopy recognition sub-model, and the canopy data and the first tree species recognition module specifically include the following sub-modules:

[0127] The canopy region segmentation submodule is used to input the visible light image into the canopy segmentation submodel to obtain the canopy region of each tree in the visible light image;

[0128] A coordinate determination submodule is used to determine the second coordinates of the tree based on the image position of the canopy region in the visible light image;

[0129] The canopy and tree species identification submodule is used to input the canopy area into the canopy identification submodel to obtain the tree's canopy perimeter, canopy width, canopy coverage, and first tree species;

[0130] The tree canopy and tree species association submodule is used to associate the second coordinate with the canopy perimeter, canopy width, canopy coverage and tree species.

[0131] As a further technical solution of this invention, the second tree species identification module specifically includes the following sub-modules:

[0132] The spectral image segmentation submodule is used to segment the tree spectral image of each tree from the forest area spectral image, and determine the third coordinate of each tree based on the position of the tree spectral image in the forest area spectral image;

[0133] The spectral image matching submodule is used to match the spectral image of each tree with spectral image samples of different tree species in a pre-configured spectral image tree species library, obtain the tree species that match the spectral image of the tree as the second tree species of the tree, and associate the third coordinate with the second tree species.

[0134] As a further technical solution of this invention, the single tree data calculation unit 304 specifically includes the following modules:

[0135] The coordinate distance calculation module is used to calculate the distance between the first coordinate and the second coordinate, and to determine the target first coordinate and the target second coordinate, which are less than a threshold distance. The target first coordinate and the target second coordinate are the coordinates of the same tree.

[0136] The volume calculation module is used to calculate the volume of the tree based on the three-dimensional model associated with the first coordinate of the target;

[0137] The biomass inversion module is used to invert the biomass of the tree based on the canopy data associated with the second coordinate of the target and the three-dimensional model associated with the tree species and the first coordinate of the target.

[0138] A carbon storage calculation module is used to calculate the carbon storage of the tree based on the biomass and the tree species associated with the target second coordinate.

[0139] As a further technical solution of this invention, the biomass inversion module specifically includes the following sub-modules:

[0140] The biomass calculation submodule is used to calculate the biomass W of each tree using the following equation:

[0141] ;

[0142] Wherein, a, b, and c are parameters determined in a pre-configured parameter table based on tree species and crown data. The parameter table is a mapping table of tree species, crown, and parameters generated based on ground measurement data. D is the diameter at breast height of the tree determined based on the three-dimensional model, and H is the tree height determined based on the three-dimensional model.

[0143] As a further technical solution of this invention, the carbon storage calculation module specifically includes the following sub-modules:

[0144] The carbon content factor determination submodule is used to obtain the carbon content factor corresponding to the tree species, wherein the carbon content factor is the carbon content factor of the tree species corresponding to the tree determined based on ground measurement data.

[0145] A carbon storage calculation submodule is used to calculate the carbon storage of the tree using the biomass and the carbon content factor.

[0146] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0147] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0148] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

[0150] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1.A forest vegetation detection method based on a UAV, characterized in that, Specifically comprising the following steps: Controlling a UAV carrying a visible light camera, a spectral camera and a radar to collect visible light images, forest spectral images and radar data of a target forest area; Based on the radar data, constructing a three-dimensional model of trees in the target forest area; Based on the visible light images and the forest spectral images, determining the crown data and species of trees in the target forest area; Based on the three-dimensional model of the trees, the species and the crown data of the trees, determining the volume, biomass and carbon storage of the trees; Respectively calculating the sum of the volume, biomass and carbon storage of all trees in the target forest area, to obtain the total volume, total biomass and total carbon storage of the target forest area; Based on the visible light images and the forest spectral images, determining the crown data and species of trees in the target forest area, specifically comprising the following steps: Inputting the visible light images into a crown detection model to obtain the crown data of the trees, the crown data comprising a second coordinate and a crown perimeter, crown width, crown coverage and a first species associated with the second coordinate; Based on the forest spectral images, determining a third coordinate of each tree and a second species associated with the third coordinate; Determining the first species and the second species of the same tree through the second coordinate and the third coordinate; Judging whether the first species and the second species of the same tree are the same species; If yes, taking the first species or the second species as the species of the tree; If no, obtaining ground measured data of the tree, and determining the species of the tree based on the ground measured data; Associating the second coordinate with the crown data and the species of the tree. 2.The UAV-based forest vegetation detection method of claim 1, wherein, Controlling a UAV carrying a visible light camera, a spectral camera and a radar to collect visible light images, forest spectral images and radar data of a target forest area, specifically comprising the following steps: Controlling the UAV to fly according to a preset flight trajectory, the flight trajectory being provided with a plurality of sampling points, wherein the flight trajectory is set according to the topography of the target forest area and a preset image overlap rate; Controlling the visible light camera on the UAV to collect visible light images of the target forest area and controlling the spectral camera on the UAV to collect forest spectral images of the target forest area when the UAV flies to the sampling points; Controlling the radar to collect radar data of the target forest area during the UAV flying according to the preset flight trajectory. 3.The UAV-based forest vegetation detection method of claim 1, wherein, Based on the radar data, constructing a three-dimensional model of trees in the target forest area, specifically comprising the following steps: Determining radar point cloud data of each tree from the radar data; Based on the radar point cloud data of each tree, constructing a three-dimensional model of each tree, determining a first coordinate of each tree, and associating the first coordinate with the three-dimensional model. 4.The UAV-based forest vegetation detection method of claim 1, wherein, The crown detection model comprises a crown segmentation sub-model and a crown recognition sub-model, and the inputting of the visible light images into the crown detection model to obtain the crown data of the trees specifically comprises the following steps: Inputting the visible light images into the crown segmentation sub-model to obtain the crown area of each tree in the visible light images; Determining the second coordinate of the tree through the image position of the crown area in the visible light images; inputting the crown region into a crown recognition sub-model to obtain a crown perimeter, a crown width, a crown coverage, and a first tree species of the tree; associating the second coordinates with the crown perimeter, the crown width, the crown coverage, and the tree species. 5.The UAV-based forest vegetation detection method of claim 4, wherein, determining a third coordinate of each tree and a second tree species associated with the third coordinate based on the forest spectral image, specifically including the following steps: segmenting a tree spectral image of each tree from the forest spectral image, and determining a third coordinate of each tree based on a position of the tree spectral image in the forest spectral image; matching the tree spectral image of each tree with spectral image samples of different tree species in a pre-configured spectral image tree species library to obtain a tree species matched with the tree spectral image as the second tree species of the tree, and associating the third coordinate with the second tree species. 6.The UAV-based forest vegetation detection method of claim 5, wherein, determining a volume, a biomass, and a carbon storage of the tree based on the three-dimensional model, the tree species, and the crown data of the tree, specifically including the following steps: calculating distances between the first coordinates and the second coordinates, and determining target first coordinates and target second coordinates with distances less than a threshold, the target first coordinates and the target second coordinates being coordinates of the same tree; calculating the volume of the tree based on the three-dimensional model associated with the target first coordinates; inverting the biomass of the tree based on the three-dimensional model associated with the target first coordinates, the crown data associated with the target second coordinates, and the tree species; calculating the carbon storage of the tree based on the biomass and the tree species associated with the target second coordinates. 7.The UAV-based forest vegetation detection method of claim 6, wherein, inverting the biomass of the tree based on the three-dimensional model associated with the target first coordinates, the crown data associated with the target second coordinates, and the tree species, specifically including the following steps: calculating the biomass W of each tree by the following equation: ； wherein a, b, and c are parameters determined according to the tree species and the crown data in a pre-configured parameter table, the parameter table being a mapping table of tree species, crown, and parameters generated according to ground measured data, D is a diameter at breast height of the tree determined according to the three-dimensional model, and H is a tree height of the tree determined according to the three-dimensional model. 8.The UAV-based forest vegetation detection method of claim 6, wherein, calculating the carbon storage of the tree based on the biomass and the tree species, specifically including the following steps: obtaining a carbon content factor corresponding to the tree species, the carbon content factor being a carbon content factor of a tree corresponding to the tree species determined according to ground measured data; calculating the carbon storage of the tree using the biomass and the carbon content factor. 9.A forest vegetation detection system based on a UAV, characterized in that, specifically including the following units: a data acquisition unit configured to control a drone carrying a visible light camera, a spectral camera, and a radar to collect a visible light image, a forest spectral image, and radar data of a target forest area; a three-dimensional model construction unit configured to construct a three-dimensional model of trees in the target forest area based on the radar data; a crown data and tree species determination unit configured to determine crown data and tree species of the trees in the target forest area based on the visible light image and the forest spectral image; a single tree data calculation unit configured to determine a volume, a biomass, and a carbon storage of the tree based on the three-dimensional model, the tree species, and the crown data of the tree; The forest area total data calculation unit is configured to calculate the sum of the volume, the biomass and the carbon storage of all trees in the target forest area, and obtain the total volume, the total biomass and the total carbon storage of the target forest area. The crown data and tree species determination unit specifically comprises the following modules: The crown data and first tree species identification module is configured to input the visible light image into a crown detection model to obtain crown data of the trees, the crown data comprising second coordinates and a crown perimeter, a crown width, a crown coverage and a first tree species associated with the second coordinates; The second tree species identification module is configured to determine third coordinates and a second tree species associated with the third coordinates of each tree based on the forest area spectral image; The tree determination module is configured to determine the first tree species and the second tree species of the same tree through the second coordinates and the third coordinates; The tree species judgment module is configured to determine whether the first tree species and the second tree species of the same tree are the same tree species; If yes, the tree species determination module is executed, and if no, the ground measured tree species determination module is executed; The tree species determination module is configured to determine the first tree species or the second tree species as the tree species of the tree; The ground measured tree species determination module is configured to obtain ground measured data of the tree, and determine the tree species of the tree based on the ground measured data; The data association module is configured to associate the second coordinates with the crown data and the tree species of the tree.

Images