A method and device for fine inversion of tree biomass based on hyperspectral lidar

Abstract

The embodiment of the present application discloses a method and device for fine inversion of tree biomass based on hyperspectral laser radar, wherein the method includes: using hyperspectral laser radar to obtain tree point cloud data and its measured spectral data of tree objects in forestry detection scenes; performing multiple mapping and spectral inversion according to the tree point cloud data and the measured spectral data to obtain a fine environmental model; segmenting the tree point cloud data to obtain the position information of a single tree and the leaf point cloud data of each leaf in a single tree; calculating the solar light utilization coefficient of each leaf according to the leaf point cloud data of each leaf, and calculating the biomass weighting coefficient of each leaf based on the real spectral data; using the position information of a single tree and the biomass weighting coefficient of each leaf, calculating the biomass index of a single tree. The present application realizes fine environmental modeling and can accurately and quickly invert the biomass index of trees.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of laser radar technology, and specifically to a method and device for fine inversion of tree biomass based on hyperspectral laser radar. Background Art

[0002] In recent years, with the protection of ecological environment and economic development, there is an urgent need for refined management and utilization of forestry resources. At present, forestry resources are rich in diversity, including vast coniferous forests, broad-leaved forests, bamboo forests, and landscape greening woodlands in large urban areas. How to quickly and accurately grasp the growth and health status of forests not only has great economic value, but also has social value in terms of ecological environment protection. To achieve refined management and utilization of forestry resources, it is necessary to use efficient and accurate digital and intelligent information acquisition equipment and information analysis technology.

[0003] Traditional methods of obtaining forestry resources usually include the following: using airborne laser radar or cameras to obtain data and information about tree crowns, but this method cannot effectively detect information inside the forest; using ground-based laser radar to obtain three-dimensional structural information inside the forest, but this method is inefficient and cannot be applied on a large scale; relying on manual survey methods, but this method is not only inefficient, but also has errors that cannot be controlled by humans; using point cloud or image data alone cannot effectively analyze and monitor information such as the type, growth, and pest and disease status of trees. How to effectively obtain forestry resource information has become an urgent problem that needs to be solved. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a method and device for fine inversion of tree biomass based on hyperspectral lidar that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of an embodiment of the present application, a method for fine inversion of tree biomass based on hyperspectral laser radar is provided, comprising:

[0006] Use hyperspectral lidar to obtain tree point cloud data and measured spectral data of tree objects in forestry detection scenes;

[0007] According to the tree point cloud data and the measured spectral data, multiple mapping and spectral inversion are performed to obtain a fine environmental model; wherein the fine environmental model includes the tree point cloud data and the real spectral data of the tree object;

[0008] According to the refined environment model, the tree point cloud data is segmented to obtain the location information of a single tree and the leaf point cloud data of each leaf in a single tree;

[0009] Calculate the solar light utilization coefficient of each leaf based on the leaf point cloud data of each leaf, and calculate the biomass weighted coefficient of each leaf based on the real spectrum data;

[0010] The biomass index of a single tree is calculated using the location information of the single tree and the biomass weighting coefficient of each leaf.

[0011] Furthermore, multiple mapping and spectral inversion are performed based on the tree point cloud data and the measured spectral data to obtain a fine environmental model, which further includes:

[0012] The point cloud geometric residual is determined by using the point cloud data of each scanning frame in the tree point cloud data. The first target value of the correlation factor corresponding to the hyperspectral laser radar is determined by iteratively optimizing the point cloud geometric residual. The point cloud data of each scanning frame is transformed into the global coordinate to realize the initial mapping and form an initial environment model.

[0013] According to the pre-built output response model of the hyperspectral lidar and the smoothness data of the tree object, the measured spectral data of the tree object is inverted to obtain the real spectral data of the tree object; wherein the smoothness data of the tree object is calculated using the initial environment model;

[0014] The spectral residual is determined using the real spectral data of the tree object, and the second target value of the correlation factor is determined by iteratively optimizing the point cloud geometric residual and the spectral residual. The point cloud data of each scanning frame and the real spectral data of the tree object are transformed into global coordinates to achieve secondary mapping and form a detailed environmental model.

[0015] Furthermore, based on the refined environment model, the tree point cloud data is segmented to obtain the location information of a single tree and the leaf point cloud data of each leaf in the single tree, further including:

[0016] Find the single tree seed point from the tree point cloud data, use the region growing algorithm to segment the point cloud data of a single tree from the tree point cloud data, and calculate the location information of a single tree;

[0017] A deep neural network is used to separate the branches and leaves of the point cloud data of a single tree, and the leaf point cloud data of each leaf in a single tree and the branch point cloud data are obtained.

[0018] Furthermore, using a deep neural network to separate the point cloud data of a single tree into branches and leaves, segmenting the leaf point cloud data of each leaf in a single tree and the branch point cloud data further includes:

[0019] According to the three-dimensional coordinates of any point in the point cloud data of a single tree, the covariance matrix of the neighborhood of the point is constructed, and the geometric eigenvalue of the point is calculated according to the covariance matrix;

[0020] The three-dimensional coordinates, geometric feature values ​​and real spectral data of each point are input into the deep neural network, and the deep neural network is used to classify each point;

[0021] According to the category label of each point, the leaf point cloud data of each leaf and the branch point cloud data are segmented from the point cloud data of a single tree.

[0022] Furthermore, according to the leaf point cloud data of each leaf, calculating the sunlight utilization coefficient of each leaf further includes:

[0023] For each leaf, a preset number of points on the leaf are randomly selected from the leaf point cloud data of the leaf using the Monte Carlo method, and a normal vector of each point is determined;

[0024] The ray tracing method is used to calculate the number of normal vectors of a preset number of points that pass through the upper leaves and branches, and the angle between the normal vector of the leaf and the solar zenith vector is calculated;

[0025] The solar light utilization coefficient of each leaf is calculated based on the preset number, the number of vectors covered by the upper leaves and branches, and the angle.

[0026] Furthermore, based on the real spectral data, calculating the biomass weighting coefficient of each leaf further includes:

[0027] For each leaf, the normalized vegetation index of the leaf is calculated based on the real spectral data of the leaf;

[0028] The biomass weighted coefficient of the leaf is calculated according to the solar light utilization coefficient of the leaf, the normalized vegetation index of the leaf and the area of ​​the leaf; wherein the area of ​​the leaf is calculated according to the leaf point cloud data of the leaf.

[0029] According to another aspect of an embodiment of the present application, a tree biomass fine inversion device based on a hyperspectral laser radar is provided, comprising:

[0030] An acquisition module, adapted to acquire tree point cloud data and measured spectral data of tree objects in a forestry detection scene by using a hyperspectral laser radar;

[0031] A model building module is suitable for performing multiple mapping and spectral inversion according to tree point cloud data and measured spectral data to obtain a fine environmental model; wherein the fine environmental model includes tree point cloud data and real spectral data of tree objects;

[0032] A segmentation module is suitable for segmenting the tree point cloud data according to the refined environment model to obtain the location information of a single tree and the leaf point cloud data of each leaf in the single tree;

[0033] The calculation module is suitable for calculating the solar light utilization coefficient of each leaf according to the leaf point cloud data of each leaf, and calculating the biomass weighted coefficient of each leaf based on the real spectrum data; and calculating the biomass index of a single tree using the location information of a single tree and the biomass weighted coefficient of each leaf.

[0034] According to another aspect of the embodiment of the present application, there is provided a computing device, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0035] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned tree biomass fine inversion method based on hyperspectral laser radar.

[0036] According to another aspect of the embodiment of the present application, a computer storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned tree biomass fine inversion method based on hyperspectral lidar.

[0037] According to another aspect of the embodiments of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned method for fine inversion of tree biomass based on hyperspectral lidar.

[0038] According to the tree biomass fine inversion method and device based on hyperspectral lidar provided in the embodiment of the present application, the forestry detection scene is detected by using hyperspectral lidar, and the tree point cloud data of the tree objects in the forestry detection scene and the measured spectral data of the tree objects in the forestry detection scene can be obtained simultaneously, which effectively overcomes the disadvantage that the optical hyperspectral camera is easily affected by the change of natural light, and the hyperspectral lidar can work normally even in the environment of no light or weak light. According to the tree point cloud data and the measured spectral data, the real spectral data of the tree objects can be obtained through multiple mapping and spectral fine inversion, which improves the accuracy of the spectral data, effectively improves the accuracy and reliability of the registration, and realizes fine environmental modeling by using geometric-spectral joint matching, and obtains a fine environmental model that contains both the tree point cloud data and the real spectral data of the tree objects. A deep neural network is used to perform fine segmentation on tree point cloud data and real spectral data of tree objects. By statistically calculating factors such as the three-dimensional size, orientation, sunlight utilization coefficient, and biomass weighting coefficient of the leaves, the tree biomass index can be accurately and quickly inverted, realizing the precise calculation of the biomass index of a single tree and optimizing the way of acquiring forest resources. In addition, using real spectral data, it is also possible to quantitatively invert forest diseases and pests and growth conditions, effectively monitor the growth of trees and the occurrence and development of diseases and pests. The flexible and convenient mobile mapping technology of this solution has great advantages in data acquisition efficiency and cost compared to traditional airborne scanning, ground-based fixed scanning, manual surveys, and the use of point cloud or image data alone. It can be applied to a variety of application needs in fine forestry, and will greatly promote the deep integration of informatization and intelligence with fine forestry.

[0039] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation method of the embodiment of the present application is listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only used for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the embodiments of the present application. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0041] Figure 1 A schematic flow chart of a method for fine inversion of tree biomass based on hyperspectral laser radar according to Embodiment 1 of the present invention is shown;

[0042] Figure 2The structure block diagram of a tree biomass fine inversion device based on a hyperspectral laser radar according to the second embodiment of the present invention is shown;

[0043] Figure 3 A schematic structural diagram of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0045] Figure 1 FIG. 4 is a flow chart of a method for fine inversion of tree biomass based on hyperspectral laser radar according to Embodiment 1 of the present invention. Figure 1 As shown, the method comprises the following steps:

[0046] Step S101: using a hyperspectral laser radar to obtain tree point cloud data and measured spectral data of tree objects in a forestry detection scene.

[0047] In an embodiment of the present application, a forestry detection scene is detected using a hyperspectral laser radar, and tree point cloud data of tree objects in the forestry detection scene and measured spectral data of tree objects in the forestry detection scene can be obtained simultaneously. Among them, the spectral data of tree objects directly obtained by using the hyperspectral laser radar is referred to as the measured spectral data of tree objects. Hyperspectral laser radar technology based on the principle of laser active detection has gradually attracted attention and matured. Hyperspectral laser radar relies on actively emitting lasers to simultaneously obtain ranging information and spectral information, overcoming the shortcomings of optical hyperspectral cameras that are easily affected by changes in natural light, and can work normally even in lightless or weak light environments. Due to the strong directivity of the laser, the energy and direction of the emitted laser of the hyperspectral laser radar are easier to measure and calculate. Compared with traditional optical hyperspectral cameras, the surface morphology of objects based on ranging information can also greatly improve the inversion accuracy of spectral information, and an accurate object reflectivity spectrum can be inverted without ground synchronous measurement.

[0048] The fine management and utilization of forestry resources has put forward an urgent need for the rapid acquisition and intelligent analysis of multi-dimensional information such as geometric structure and spectrum. The growth and development of trees, as well as their health status (such as whether they are infected with pests and diseases, etc.) are directly reflected in the spectral properties of leaves, branches and trunks. Therefore, hyperspectral remote sensing technology can effectively monitor the growth of trees and the occurrence and development of pests and diseases, which has great advantages. Hyperspectral LiDAR overcomes the shortcomings of optical hyperspectral cameras that are easily affected by changes in natural light, and can more effectively carry out fine forestry monitoring. Hyperspectral LiDAR reflects the spectral information characteristics of tree objects with the voltage change of the echo signal. The output value is related to the laser emission energy, the distance of the tree object, the surface shape of the tree object and the reflectivity of the tree object. The measured spectral data of tree objects obtained by hyperspectral LiDAR cannot be directly used for matching and mapping.

[0049] Step S102, performing multiple mapping and spectral inversion according to the tree point cloud data and the measured spectral data to obtain a detailed environment model.

[0050] Taking into account that the measured spectral data of tree objects obtained by using hyperspectral lidar cannot be directly used for matching and mapping, in an embodiment of the present application, the tree point cloud data can be first used for simultaneous positioning and mapping, that is, initial mapping, to achieve coarse mapping, establish the spatial structure information of the environment, and form an initial environmental model. Based on this, fine spectral inversion is performed to obtain accurate reflectance spectral data of tree objects, wherein the accurate reflectance spectral data obtained by inverting the measured spectral data of tree objects is called the true spectral data of the tree objects; then, geometric-spectral joint matching is used for secondary mapping to achieve fine mapping and obtain a fine environmental model that contains both tree point cloud data and the true spectral data of tree objects.

[0051] Specifically, the point cloud geometric residual is determined by using the point cloud data of each scanning frame in the tree point cloud data, and the first target value of the correlation factor corresponding to the hyperspectral laser radar is determined by iteratively optimizing the point cloud geometric residual. The point cloud data of each scanning frame is transformed to the global coordinate to realize the initial mapping and form an initial environmental model; then, according to the pre-constructed output response model of the hyperspectral laser radar and the smoothness data of the tree object, the measured spectral data of the tree object is inverted to obtain the real spectral data of the tree object; wherein the smoothness data of the tree object is calculated using the initial environmental model; then, the real spectral data of the tree object is used to determine the spectral residual, and the second target value of the correlation factor is determined by iteratively optimizing the point cloud geometric residual and the spectral residual, and the point cloud data of each scanning frame and the real spectral data of the tree object are transformed to the global coordinate to realize secondary mapping and form a fine environmental model. The embodiment of this application introduces the hyperspectral laser radar information acquisition and analysis method into the field of forestry resource detection, and can integrate and fuse inertial measurement units, hyperspectral laser radars and other multi-source sensing sensors to realize integrated spectrum-laser mobile mapping, and can simultaneously and efficiently obtain the geometric structure of forestry detection scenes and the integrated multi-dimensional environmental information of the spectrum, which can effectively improve the autonomous positioning and orientation, mobile mapping and intelligent analysis capabilities of forestry environmental elements. This application can be applied to various application needs of fine forestry, and will greatly promote the deep integration of informatization and intelligence with fine forestry.

[0052] Hyperspectral lidar can obtain ranging information and three-dimensional point cloud data of forestry detection scenes at a frequency lower than that of inertial measurement units. By matching point cloud frames and point clouds with maps, the relative position and posture between two frames of point clouds are added to the position and posture update. For unstructured scenes, the clutter of point cloud data is calculated and evaluated, and the weight of point cloud data in the registration is adaptively adjusted to improve the reliability and accuracy of point cloud registration.

[0053] The tree point cloud data of the tree object includes multiple scan frame point cloud data. For each scan frame point cloud data, the point cloud geometric residual of the scan frame point cloud data is determined. The point cloud geometric residual refers to the distance from the current laser point to the nearest plane in the point cloud map in the global coordinate system. In practical applications, a first optimization target can be set based on the point cloud geometric residual. The first optimization target can be specifically

[0054]

[0055] Among them, r g (x,p i ) represents the geometric residual of the point cloud; x represents the correlation factor corresponding to the hyperspectral laser radar, which may include the position information, attitude information, inertial measurement unit noise, etc. of the hyperspectral laser radar; p i Represents different laser points in the same scanning frame; ∑g Represents the covariance of the geometric residual of the point cloud. Use the LM algorithm to iteratively optimize to achieve the first optimization goal, and use the value of the correlation factor when the first optimization goal is achieved as the first target value of the correlation factor, so as to obtain the position information, attitude information, etc. of the hyperspectral laser radar, and transform the measurement value of the hyperspectral laser radar (for example, including each scanning frame point cloud data) to the global coordinate to realize the initial mapping and form the initial environment model.

[0056] The output response model of the hyperspectral laser radar records the correspondence between the laser detector's pupil energy and the measured spectral data of the tree object output by the hyperspectral laser radar, and also records the correspondence between the laser emission energy, the detected tree object properties, etc. and the laser detector's pupil energy. With the help of the three-dimensional representation of the environment established by the spectrum-laser mobile mapping technology, accurate tree object properties are established; the relationship between the laser emission energy of different wavelengths, the laser receiving energy, the detection distance, the laser incident / exit angle, and the smoothness data of the tree object is established, and the numerical value directly output by the hyperspectral laser radar is corresponded to the reflectivity of the tree object, thereby obtaining the real spectral data of the tree object.

[0057] The real spectral data of the tree object obtained through fine inversion is not affected by light and reflects the real attributes of the tree object. It can not only serve as an additional constraint for point cloud-point cloud registration to improve the accuracy and reliability of registration, but also realize loop detection of simultaneous positioning and mapping technology based on multi-dimensional information such as point cloud and spectrum, making it suitable for scenes with a single texture as well as scenes with a single spatial geometric structure. After obtaining the real spectral data of the tree object, the real spectral data can be combined for secondary mapping. In the secondary mapping process, the real spectral data of the tree object is used to determine the spectral residual. The spectral residual refers to the Euclidean distance from the spectrum of the current laser point in the global coordinate system to the average spectrum of the nearest point set in the point cloud map. In practical applications, a second optimization target can be set based on the point cloud geometric residual and the spectral residual. The second optimization target can be to add relevant requirements for the spectral residual to the first optimization target. The second optimization target can be specifically

[0058]

[0059] Among them, r s (x,s i ) represents the spectral residual; s i Represents different laser points p i The real spectrum data of sRepresents the covariance of the spectral residual. The LM algorithm is used for iterative optimization to achieve the second optimization goal. The value of the correlation factor when the second optimization goal is achieved is used as the second target value of the correlation factor, so as to obtain the position information, posture information, etc. of the hyperspectral laser radar, and transform the measurement value of the hyperspectral laser radar (for example, including the point cloud data of each scanning frame and the real spectral data of the tree object, etc.) to the global coordinate to realize secondary mapping and form a fine environment model. The fine environment model contains not only the three-dimensional point cloud but also the real spectral data of each point.

[0060] Step S103 , segmenting the tree point cloud data according to the refined environment model to obtain the location information of a single tree and the leaf point cloud data of each leaf in the single tree.

[0061] By integrating the features of point cloud and spectrum in different aspects such as geometric structure and spectrum, the deep neural network is used to perform fine segmentation on the acquired tree point cloud data and real spectrum data, and the leaves, branches, trunks and other parts are segmented. For example, the single tree seed point is found from the tree point cloud data, and the point cloud data of a single tree is segmented from the tree point cloud data using the regional growing algorithm, and the position information of a single tree is calculated; the point cloud data of a single tree is separated from the branches using the deep neural network, and the leaf point cloud data and branch point cloud data of each leaf in a single tree are obtained by segmentation. Among them, in the process of separating branches and leaves, the covariance matrix of the neighborhood of any point in the point cloud data of a single tree can be constructed according to the three-dimensional coordinates of the point cloud data of the single tree, and the geometric eigenvalue of the point can be calculated according to the covariance matrix; the three-dimensional coordinates, geometric eigenvalues ​​and real spectrum data of each point are input into the deep neural network, and the deep neural network is used to classify each point; according to the category label of each point, the leaf point cloud data and branch point cloud data of each leaf are segmented from the point cloud data of a single tree.

[0062] Specifically, the real spectral data of the tree object can be denoised first, the ground points can be separated and a digital elevation model (DEM) can be generated. The single tree seed point is searched in the point cloud data remaining after the ground points are separated. Since the trunk can reflect a large number of points, the point density of the trunk or tree peak projected into the two-dimensional plane grid is usually greater than its neighborhood, so the tree peak search problem is converted into a local extreme value search problem. Then the regional growing algorithm is used to complete the single tree segmentation of the point cloud data, thereby obtaining the point cloud data of a single tree, and calculating the location information, tree height and other information of the single tree. Among them, the location information of a single tree may include longitude lon and latitude lat.

[0063] After obtaining the point cloud data of a single tree, quantifying the differences in structure between different tree species and achieving accurate separation of leaf and branch structures is very important for understanding the relationship between tree structure and tree physiological function. i (x i ,y i , z i ) constructs the covariance matrix of its neighborhood, and calculates the three geometric eigenvalues ​​(λ1, λ2, λ3) of the point based on this covariance matrix.

[0064] The deep neural network used in the embodiments of the present application may include PointNet++, etc. Among them, PointNet++ is a deep learning architecture for point cloud data processing, which can effectively learn the feature representation of point cloud data. PointNet++ adopts a hierarchical feature learning strategy to improve the feature learning effect of point cloud data. In the process of point cloud data processing, PointNet++ first subsamples the initial point cloud data to generate a series of point sets of different resolutions. For each point set of resolution, PointNet++ will build a multi-layer perceptron (MLP) network to learn the feature representation at that resolution. These MLP networks constitute the backbone network of PointNet++. Then PointNet++ uses an operation called set pooling to merge feature representations of different resolutions. The basic idea of ​​the set pooling operation is to convert the feature representation at each resolution into a fixed-size feature vector for subsequent classification or segmentation tasks. Finally, PointNet++ uses a series of fully connected layers to process the feature vectors after set pooling to achieve classification, segmentation or other tasks of point cloud data.

[0065] Taking PointNet++ as an example, the PointNet++ deep neural network architecture is used to perform fine branch and leaf separation on the point cloud data of a single tree in an end-to-end manner. The point cloud data containing three-dimensional coordinates (x, y, z), geometric features (λ1, λ2, λ3) and real spectral data s iMultiple hyperspectral laser points are directly input into the PointNet++ network structure. The output categories of PointNet++ include leaves, branches, and unidentified categories. The training and test data are manually labeled as corresponding categories using CloudCompare software. The classification output results of PointNet++ may contain a lot of noise and need to be further processed. Regional growth is performed on the branch point cloud data, and the points that are mistakenly classified as leaves are re-labeled as branches. The leaf point cloud data is denoised, and then the leaf point cloud data is clustered using the DBSCAN density clustering algorithm to obtain the leaf point cloud data of each leaf.

[0066] Step S104, calculating the solar light utilization coefficient of each leaf according to the leaf point cloud data of each leaf, and calculating the biomass weighted coefficient of each leaf based on the real spectrum data.

[0067] In this embodiment, the three-dimensional size, orientation and other elements of each leaf can be statistically calculated based on the leaf point cloud data of each leaf. The sunlight utilization coefficient of each leaf is statistically calculated using the ray tracing method based on the geographical location information and the leaf orientation. The biomass weighted coefficient of each leaf is statistically calculated based on the real spectrum data.

[0068] Specifically, for each leaf, a preset number of points on the leaf are randomly selected from the leaf point cloud data of the leaf using the Monte Carlo method, and the normal vector of each point is determined; then, the number of vectors covered by the upper leaves and branches in the normal vectors of the preset number of points is calculated using the ray tracing method, and the angle between the normal vector of the leaf and the solar zenith vector is calculated; then, the solar light utilization coefficient of each leaf is calculated based on the preset number, the number of vectors covered by the upper leaves and branches, and the angle. For each leaf, the normalized vegetation index of the leaf is calculated based on the real spectral data of the leaf; the biomass weighted coefficient of the leaf is calculated based on the solar light utilization coefficient of the leaf, the normalized vegetation index of the leaf, and the area of ​​the leaf; wherein the area of ​​the leaf is calculated based on the leaf point cloud data of the leaf.

[0069] Phenotypic characteristics are extracted and biomass weighted coefficients are calculated for each leaf. For each leaf, the area A of the leaf can be calculated. Then, the Monte Carlo method is used to randomly select N points on the leaf and calculate the normal vector of each point. In this way, N normal vectors are extracted from a single leaf. Then, the ray tracing method is used to calculate the number of vectors N′ that can pass through the upper leaves and branches, and the angle θ between the normal vector of the leaf and the solar zenith vector is calculated. Then the solar light utilization coefficient of the leaf is

[0070] The Normalized Difference Vegetation Index (NDVI) is calculated using the real spectral data of the leaf: NDVI = (NIR-Red) / (NIR+Red). NIR represents the reflectivity of the infrared band; Red represents the reflectivity of the visible light band. By calculating the difference and ratio of the reflectivity of these two bands, the value of NDVI can be obtained. The value range of NDVI is usually between -1 and 1. The larger the value, the higher the vegetation coverage and the better the growth condition. In the embodiment of the present application, the biomass weighting coefficient of a single leaf is Among them, B1 represents the biomass weighting coefficient of leaves; ρ is a constant.

[0071] Step S105, calculating the biomass index of the single tree using the location information of the single tree and the biomass weighting coefficient of each leaf.

[0072] For a single tree, the biomass index of a single tree can be accurately calculated by combining the three-dimensional dimensions of all leaves of the single tree, the solar light utilization coefficient, and the biomass weighting coefficient. Specifically, the biomass index of a single tree can be accurately calculated by combining the location information of all leaves and the single tree. The biomass index of a single tree is Among them, B represents the biomass index of a single tree; ρ is a constant; lat represents location information, such as latitude; A i represents the area of ​​the i-th leaf in the single tree; N i Represents the number of normal vectors extracted on the i-th leaf; N′ i N i The number of vectors that can penetrate the upper leaves and branches; θ i Represents the angle between the normal vector of the i-th leaf and the solar zenith vector; NDVI i Represents the normalized vegetation index of the i-th leaf.

[0073] According to the tree biomass fine inversion method based on hyperspectral lidar provided in the embodiment of the present application, the forestry detection scene is detected by using hyperspectral lidar, and the tree point cloud data of the tree objects in the forestry detection scene and the measured spectral data of the tree objects in the forestry detection scene can be obtained simultaneously, which effectively overcomes the disadvantage that the optical hyperspectral camera is easily affected by the change of natural light, and the hyperspectral lidar can work normally even in the environment of no light or weak light. According to the tree point cloud data and the measured spectral data, the real spectral data of the tree objects can be obtained through multiple mapping and spectral fine inversion, which improves the accuracy of the spectral data, effectively improves the accuracy and reliability of the registration, and realizes fine environmental modeling by using geometric-spectral joint matching, and obtains a fine environmental model that contains both the tree point cloud data and the real spectral data of the tree objects. The deep neural network performs fine segmentation on tree point cloud data and real spectral data of tree objects. By statistically calculating factors such as the three-dimensional size, orientation, sunlight utilization coefficient, and biomass weighting coefficient of the leaves, it can accurately and quickly invert the tree biomass index, realize the precise calculation of the biomass index of a single tree, and optimize the way of acquiring forestry resources. In addition, using real spectral data, it is also possible to quantitatively invert forest diseases and pests and growth conditions, effectively monitor the growth of trees and the occurrence and development of diseases and pests, etc. The flexible and convenient mobile mapping technology of this solution has great advantages in data acquisition efficiency and cost compared with traditional airborne scanning, ground-based fixed scanning, manual surveys, and the use of point cloud or image data alone. It can be applied to a variety of application needs of fine forestry, and will greatly promote the deep integration of informatization and intelligence with fine forestry.

[0074] Figure 2 FIG. 4 shows a structural block diagram of a tree biomass fine inversion device based on a hyperspectral laser radar according to Embodiment 2 of the present invention. Figure 2 As shown, the device includes: an acquisition module 210, a model building module 220, a segmentation module 230 and a calculation module 240.

[0075] The acquisition module 210 is adapted to: use a hyperspectral laser radar to acquire tree point cloud data and measured spectral data of tree objects in a forestry detection scene.

[0076] The model building module 220 is adapted to: perform multiple mapping and spectral inversion according to the tree point cloud data and the measured spectral data to obtain a fine environmental model, wherein the fine environmental model includes the tree point cloud data and the real spectral data of the tree object;

[0077] The segmentation module 230 is adapted to segment the tree point cloud data according to the refined environment model to obtain the location information of a single tree and the leaf point cloud data of each leaf in the single tree.

[0078] The calculation module 240 is suitable for: calculating the solar light utilization coefficient of each leaf according to the leaf point cloud data of each leaf, and calculating the biomass weighted coefficient of each leaf based on the real spectrum data; calculating the biomass index of a single tree using the location information of a single tree and the biomass weighted coefficient of each leaf.

[0079] Optionally, the model building module 220 is further adapted to: determine the point cloud geometric residual using the point cloud data of each scanning frame in the tree point cloud data, determine the first target value of the correlation factor corresponding to the hyperspectral laser radar by iteratively optimizing the point cloud geometric residual, transform the point cloud data of each scanning frame to the global coordinate to realize the initial mapping and form an initial environmental model; invert the measured spectral data of the tree object according to the pre-constructed output response model of the hyperspectral laser radar and the smoothness data of the tree object to obtain the real spectral data of the tree object; wherein the smoothness data of the tree object is calculated using the initial environmental model; determine the spectral residual using the real spectral data of the tree object, determine the second target value of the correlation factor by iteratively optimizing the point cloud geometric residual and the spectral residual, transform the point cloud data of each scanning frame and the real spectral data of the tree object to the global coordinate to realize the secondary mapping and form a fine environmental model.

[0080] Optionally, the segmentation module 230 is further suitable for: finding single tree seed points from the tree point cloud data, segmenting the point cloud data of a single tree from the tree point cloud data using a region growing algorithm, and calculating the position information of a single tree; using a deep neural network to separate the branches and leaves of the point cloud data of a single tree, and segmenting to obtain leaf point cloud data of each leaf in a single tree and branch point cloud data.

[0081] Optionally, the segmentation module 230 is further suitable for: constructing a covariance matrix of the neighborhood of any point in the point cloud data of a single tree according to the three-dimensional coordinates of the point, and calculating the geometric eigenvalues ​​of the point according to the covariance matrix; inputting the three-dimensional coordinates, geometric eigenvalues ​​and real spectral data of each point into a deep neural network, and using the deep neural network to classify each point; and segmenting the leaf point cloud data of each leaf and the branch point cloud data from the point cloud data of a single tree according to the category label of each point.

[0082] Optionally, the calculation module 240 is further adapted to: for each leaf, using the Monte Carlo method to randomly select a preset number of points on the leaf from the leaf point cloud data of the leaf, and determine the normal vector of each point; using the ray tracing method to calculate the number of vectors covered by the upper leaves and branches in the normal vectors of the preset number of points, and calculating the angle between the normal vector of the leaf and the solar zenith vector; calculating the solar light utilization coefficient of each leaf based on the preset number, the number of vectors covered by the upper leaves and branches, and the angle.

[0083] Optionally, the calculation module 240 is further adapted to: for each leaf, calculating the normalized vegetation index of the leaf based on the real spectral data of the leaf; calculating the biomass weighted coefficient of the leaf according to the solar light utilization coefficient of the leaf, the normalized vegetation index of the leaf and the area of ​​the leaf; wherein the area of ​​the leaf is calculated based on the leaf point cloud data of the leaf.

[0084] The description of each module above refers to the corresponding description in the method embodiment and will not be repeated here.

[0085] According to the tree biomass fine inversion device based on hyperspectral lidar provided in the embodiment of the present application, the forestry detection scene is detected by using the hyperspectral lidar, and the tree point cloud data of the tree objects in the forestry detection scene and the measured spectral data of the tree objects in the forestry detection scene can be obtained simultaneously, which effectively overcomes the disadvantage that the optical hyperspectral camera is easily affected by the change of natural light, and the hyperspectral lidar can work normally even in the environment of no light or weak light. According to the tree point cloud data and the measured spectral data, the real spectral data of the tree objects can be obtained through multiple mapping and spectral fine inversion, which improves the accuracy of the spectral data, effectively improves the accuracy and reliability of the registration, and realizes fine environmental modeling by using geometric-spectral joint matching, and obtains a fine environmental model that contains both the tree point cloud data and the real spectral data of the tree objects. The deep neural network performs fine segmentation on tree point cloud data and real spectral data of tree objects. By statistically calculating factors such as the three-dimensional size, orientation, sunlight utilization coefficient, and biomass weighting coefficient of the leaves, it can accurately and quickly invert the tree biomass index, realize the precise calculation of the biomass index of a single tree, and optimize the way of acquiring forestry resources. In addition, using real spectral data, it is also possible to quantitatively invert forest diseases and pests and growth conditions, effectively monitor the growth of trees and the occurrence and development of diseases and pests, etc. The flexible and convenient mobile mapping technology of this solution has great advantages in data acquisition efficiency and cost compared with traditional airborne scanning, ground-based fixed scanning, manual surveys, and the use of point cloud or image data alone. It can be applied to a variety of application needs of fine forestry, and will greatly promote the deep integration of informatization and intelligence with fine forestry.

[0086] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, which enables a processor to perform operations corresponding to the tree biomass fine inversion method based on hyperspectral lidar in any of the above method embodiments.

[0087] An embodiment of the present application provides a computer program product, which includes at least one executable instruction or computer program, which enables a processor to perform operations corresponding to the tree biomass fine inversion method based on hyperspectral lidar in any of the above-mentioned method embodiments.

[0088] Figure 3 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. The specific embodiment of the present application does not limit the specific implementation of the computing device.

[0089] like Figure 3 As shown, the computing device may include: a processor (processor) 302 , a communications interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .

[0090] The processor 302, the communication interface 304, and the memory 306 communicate with each other via the communication bus 308. The communication interface 304 is used to communicate with other devices such as a client or other server network elements. The processor 302 is used to execute the program 310, which can specifically execute the relevant steps in the embodiment of the tree biomass fine inversion method based on hyperspectral laser radar for computing devices.

[0091] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0092] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0093] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0094] The program 310 can be specifically used to enable the processor 302 to execute the tree biomass fine inversion method based on hyperspectral laser radar in any of the above-mentioned method embodiments. The specific implementation of each step in the program 310 can refer to the corresponding descriptions in the corresponding steps and units in the above-mentioned tree biomass fine inversion embodiment based on hyperspectral laser radar, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described equipment and modules can refer to the corresponding process description in the above-mentioned method embodiment, which will not be repeated here.

[0095] The algorithm and display provided herein are not inherently related to any specific computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application embodiment described herein, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application embodiment.

[0096] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0097] Similarly, it should be understood that in order to streamline the present disclosure and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed embodiments of the present application require more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the embodiment of the present application.

[0098] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0099] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the embodiments of the present application and to form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0100] The various component embodiments of the embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The embodiments of the present application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the embodiments of the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0101] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The embodiments of the present application may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A tree biomass fine inversion method based on hyperspectral lidar, characterized in that: include: Acquire tree point cloud data of tree objects in a forestry detection scene and measured spectral data of the tree objects using a hyperspectral laser radar; According to the tree point cloud data and the measured spectral data, multiple mapping and spectral inversion are performed to obtain a fine environmental model; wherein the fine environmental model includes the tree point cloud data and the real spectral data of the tree object; Segmenting the tree point cloud data according to the refined environment model to obtain location information of a single tree and leaf point cloud data of each leaf in the single tree; Calculating the solar light utilization coefficient of each leaf according to the leaf point cloud data of each leaf, and calculating the biomass weighted coefficient of each leaf based on the real spectrum data; Calculating the biomass index of the single tree using the location information of the single tree and the biomass weighting coefficient of each leaf; The step of calculating the sunlight utilization coefficient of each leaf according to the leaf point cloud data of each leaf further includes: For each leaf, a preset number of points on the leaf are randomly selected from the leaf point cloud data of the leaf using the Monte Carlo method, and a normal vector of each point is determined; Calculate the number of normal vectors of the preset number of points that are covered by upper leaves and branches using a ray tracing method, and calculate the angle between the normal vector of the leaf and the solar zenith vector; Calculate the solar light utilization coefficient of each leaf according to the preset number, the number of vectors covered by the upper leaves and branches, and the angle; The step of calculating the biomass weighting coefficient of each leaf based on the real spectral data further comprises: For each leaf, the normalized vegetation index of the leaf is calculated based on the real spectral data of the leaf; The biomass weighted coefficient of the leaf is calculated according to the solar light utilization coefficient of the leaf, the normalized vegetation index of the leaf and the area of ​​the leaf; wherein the area of ​​the leaf is calculated according to the leaf point cloud data of the leaf.

2. The method according to claim 1, characterized in that The performing multiple mapping and spectral inversion according to the tree point cloud data and the measured spectral data to obtain a fine environment model further includes: Using each scanning frame point cloud data in the tree point cloud data to determine the point cloud geometric residual, determining the first target value of the correlation factor corresponding to the hyperspectral laser radar by iteratively optimizing the point cloud geometric residual, transforming each scanning frame point cloud data into global coordinates to realize initial mapping and form an initial environment model; Inverting the measured spectral data of the tree object according to the pre-constructed output response model of the hyperspectral laser radar and the smoothness data of the tree object to obtain the real spectral data of the tree object; wherein the smoothness data of the tree object is calculated using the initial environment model; The spectral residual is determined using the real spectral data of the tree object, and the second target value of the correlation factor is determined by iteratively optimizing the point cloud geometric residual and the spectral residual. The point cloud data of each scanning frame and the real spectral data of the tree object are transformed into global coordinates to achieve secondary mapping and form the refined environment model.

3. The method according to claim 1, characterized in that The step of segmenting the tree point cloud data according to the refined environment model to obtain the location information of a single tree and the leaf point cloud data of each leaf in the single tree further includes: Finding a single tree seed point from the tree point cloud data, segmenting the point cloud data of the single tree from the tree point cloud data using a region growing algorithm, and calculating the position information of the single tree; The point cloud data of the single tree is separated into branches and leaves using a deep neural network, so as to obtain leaf point cloud data of each leaf in the single tree and branch point cloud data.

4. The method according to claim 3, characterized in that: The step of using a deep neural network to separate the point cloud data of the single tree from its branches and leaves, and segmenting the point cloud data of each leaf in the single tree and the point cloud data of the branch further includes: According to the three-dimensional coordinates of any point in the point cloud data of the single tree, a covariance matrix of the neighborhood of the point is constructed, and according to the covariance matrix, a geometric eigenvalue of the point is calculated; Inputting the three-dimensional coordinates, geometric feature values ​​and real spectral data of each point into the deep neural network, and using the deep neural network to classify each point; According to the category label of each point, the leaf point cloud data of each leaf and the branch point cloud data are segmented from the point cloud data of the single tree.

5. A tree biomass fine inversion device based on hyperspectral laser radar, characterized in that: include: An acquisition module, adapted to acquire tree point cloud data of tree objects in a forestry detection scene and measured spectral data of the tree objects using a hyperspectral laser radar; A model building module, adapted to perform multiple mapping and spectral inversion according to the tree point cloud data and the measured spectral data to obtain a fine environmental model; wherein the fine environmental model includes the tree point cloud data and the real spectral data of the tree object; A segmentation module, adapted to segment the tree point cloud data according to the fine environment model to obtain the location information of a single tree and the leaf point cloud data of each leaf in the single tree; A calculation module, adapted to calculate the solar light utilization coefficient of each leaf according to the leaf point cloud data of each leaf, and calculate the biomass weighted coefficient of each leaf based on the real spectrum data; and calculate the biomass index of the single tree using the location information of the single tree and the biomass weighted coefficient of each leaf; The calculation module is further adapted to: for each leaf, randomly select a preset number of points on the leaf from the leaf point cloud data of the leaf using the Monte Carlo method, and determine the normal vector of each point; calculate the number of vectors covered by upper leaves and branches in the normal vectors of the preset number of points using the ray tracing method, and calculate the angle between the normal vector of the leaf and the solar zenith vector; calculate the solar light utilization coefficient of each leaf according to the preset number, the number of vectors covered by upper leaves and branches, and the angle; The calculation module is further suitable for: for each leaf, calculating the normalized vegetation index of the leaf based on the real spectral data of the leaf; calculating the biomass weighted coefficient of the leaf according to the solar light utilization coefficient of the leaf, the normalized vegetation index of the leaf and the area of ​​the leaf; wherein the area of ​​the leaf is calculated based on the leaf point cloud data of the leaf.

6. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the tree biomass fine inversion method based on hyperspectral laser radar as described in any one of claims 1-4.

7. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and wherein the executable instruction enables a processor to execute operations corresponding to the tree biomass fine inversion method based on hyperspectral lidar as described in any one of claims 1 to 4.

8. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the tree biomass fine inversion method based on hyperspectral lidar as described in any one of claims 1 to 4.

Images