A tree species diversity extraction method based on multi-source remote sensing and deep learning

Abstract

The application discloses a kind of tree species diversity extraction methods of synergic multi-source remote sensing and deep learning, belong to the technical field of remote sensing ecological information.Collect multispectral image, airborne laser radar point cloud and ground measured data, and extract texture features based on gray level co-occurrence matrix;Digital elevation, digital surface model are generated after point cloud filtering classification, and crown height model is solved by difference;Sub-pixel unmixing is completed using full-constraint least square linear spectral mixture model, and normalization processing is carried out according to shadow correction formula;The corrected abundance map is registered and superimposed with the crown height model to form multi-channel features, and the sample set is constructed by combining ground sample plot diameter at breast height statistical rules to label tree species categories;Convolutional neural network containing multi-level convolution block is built, and iterative training is carried out, the network autonomously learns the nonlinear correlation of spectral and structural features, the regional tree species diversity spatial distribution map is output by model global pixel-by-pixel prediction, and the complex mountain forest area large-scale tree species information automatic extraction operation is adapted.

Description

Technical Field

[0001] This invention relates to the field of remote sensing ecological information technology, specifically to a method for extracting tree species diversity through collaborative multi-source remote sensing and deep learning. Background Technology

[0002] Tree species diversity is a core quantitative indicator for measuring the stability of forest ecosystems, carbon sequestration potential, and regional biodiversity levels. Accurate and efficient mapping of regional tree species diversity is a fundamental support for routine forest resource surveys, ecological protection planning, and ecological risk assessment of forest land along power lines. Traditional tree species diversity monitoring relies entirely on manual field quadrat surveys. Personnel need to record basic information such as tree species type, diameter at breast height (DBH), and canopy status in each quadrat. Although the identification accuracy of individual quadrats is high, the entire survey process involves huge investments in manpower, time, and economic costs. It is difficult to achieve full coverage in mountainous, canyonous, and other complex terrains with poor transportation. At the same time, the long field survey cycle cannot support the needs of quarterly and annual dynamic updates. With the iteration of remote sensing Earth observation technology, multispectral satellites and airborne lidar equipment have been gradually applied to the field of forestry identification. Relying on the advantages of large-scale synchronous imaging, they have broken through the spatial limitations of ground surveys. However, the information dimensions of a single remote sensing data source have inherent shortcomings. It is difficult to take into account the two key features of sub-pixel spectral components and the three-dimensional vertical structure of vegetation. The classification and identification error of complex mixed forest areas is relatively high, which cannot meet the needs of refined tree species diversity extraction projects.

[0003] Currently, numerous academic studies both domestically and internationally have explored tree species classification technologies based on multi-source remote sensing fusion. Representative studies include Hamdan et al.'s research on single-tree species identification based on multispectral airborne LiDAR and a two-branch Transformer network. This scheme completes feature input by shallowly stitching together point cloud 3D features and spectral features, but it only targets single-tree-scale segmentation and modeling, ignoring the industry reality of large-area mixed pixels in satellite multispectral images, and cannot be directly extended to regional continuous forest mapping scenarios. Feng et al. conducted a hyperspectral and LiDAR co-classification experiment in subtropical plantations, but the study relied solely on the NDVI single vegetation index as spectral input, without introducing texture features to expand the spectral dimensions. The model's generalization ability in mountainous forest areas is limited because it does not employ spectral unmixing techniques to decompose the tree species components within mixed pixels, and cannot effectively handle the spectral confusion caused by terrain shadows and canopy overlap. Meanwhile, the team led by Zhang Huaiqing at the Chinese Academy of Forestry proposed a dual-branch Transformer fusion recognition framework combining RGB imagery and LiDAR point clouds. This study uses a shallow feature overlay fusion strategy, simply splicing the two types of data at the model input without exploring the deep nonlinear correlation between spectral abundance sub-pixel information and canopy vertical structure. It lacks the ability to distinguish closely related tree species with highly similar spectral curves, and the entire process lacks standardized spectral correction and abundance normalization processing links, resulting in a significant decrease in classification accuracy in areas with dramatic terrain undulations.

[0004] Existing domestic invention patents also disclose multi-source remote sensing fusion tree species identification schemes. A representative patent, CN109492563A, discloses a tree species classification method based on UAV hyperspectral and LiDAR point cloud data. This patent directly concatenates the original hyperspectral pixels with LiDAR elevation features and inputs them into a machine learning model. It only uses a shallow support vector machine classifier to complete tree species classification, without introducing convolutional neural networks to automatically mine spatial context features. Furthermore, it lacks an abundance correction mechanism for mountainous terrain shadows, and the proportion of each tree species within the mixed pixels cannot be quantitatively output. It can only achieve a rough identification of a single dominant tree species, making it difficult to generate a quantitative spatial distribution map of tree species abundance. Another patent, CN116258971A, proposes a multi-source remote sensing image intelligent interpretation scheme. This patent fuses satellite multispectral and airborne point cloud data to construct a classification dataset, but its feature fusion method is only image-based. The system simply stacks pixels without utilizing a linear spectral mixing model to decompose sub-pixel tree species components. The input features lack subdivided tree species abundance channels, and the model can only perform recognition based on the average spectrum of the whole pixel. It is severely affected by interference from different spectra of the same species and different species with the same spectrum. Furthermore, it lacks a standardized extraction process for vegetation index and grayscale texture features, resulting in a single dimension of input features and low overall classification accuracy in complex forest areas. Patent CN119206499A discloses a forest tree recognition system based on attention convolutional neural networks. This scheme optimizes the network structure to improve feature extraction capabilities, but the data preprocessing stage lacks a fully constrained least squares spectral unmixing step, making it unable to separate multiple tree species, bare soil, and shadow end-member components within a pixel. It has poor stability in recognizing fragmented forest land with severe canopy overlap. The entire technical process does not form a standardized closed loop from data preprocessing to diversity thematic map output, resulting in a high threshold for practical engineering implementation. Although the aforementioned patents achieve the combined use of optical remote sensing and lidar data, none of them use linear spectral unmixing as a core preprocessing step. This makes it impossible to fully mine the spectral information of tree species at the sub-pixel scale, and multi-source features are only shallowly spliced. The model input discriminative features are insufficient, and the recognition effect of complex mountain mixed forests is significantly limited.

[0005] Existing field survey methods, mainstream academic research solutions, and published related patents all have significant technical shortcomings. They either rely on a single data source or simply stack surface features from multiple remote sensing sources. They lack a pre-processing workflow for unmixing linear spectral mixing models, and cannot simultaneously acquire two core features: sub-pixel tree species abundance and canopy 3D structure. They have not built a standardized deep learning classification workflow adapted to complex terrain and mixed pixel scenes. Problems such as poor differentiation of closely related spectral tree species, difficulty in eliminating mountain shadow interference, and insufficient accuracy in large-scale mapping have remained unresolved for a long time. Summary of the Invention

[0006] Based on the aforementioned technical problems, this application discloses a method for extracting tree species diversity through collaborative multi-source remote sensing and deep learning, comprising the following steps:

[0007] S1. Determine the study area and collect multi-source remote sensing data and ground-measured data; the multi-source remote sensing data includes multispectral imagery and airborne lidar point cloud data; the ground-measured data includes ground quadrat survey data and ground spectral acquisition data;

[0008] S2. Preprocess the multi-source remote sensing data and extract features; perform radiometric calibration and atmospheric correction on the multispectral image, calculate the vegetation index and extract texture features; generate digital elevation model and digital surface model from the airborne lidar point cloud data after denoising and classification, and obtain the canopy height model through differential operation.

[0009] S3. Based on the linear spectral mixing model, the preprocessed multispectral image is spectrally unmixed; the main tree species endmembers and non-vegetation endmembers are selected, and the fully constrained least squares method is used to unmix and obtain the preliminary abundance of each tree species. The shadow endmember is introduced to normalize and correct the preliminary abundance to obtain the corrected tree species abundance map.

[0010] S4. Construct a multi-source feature dataset; Spatially register and overlay the tree species abundance map obtained in S3 with the canopy height model obtained in step S2 to form multi-channel input data that integrates spectral features and vertical structure features, and use ground quadrat survey data to assign tree species category labels to the multi-channel input data;

[0011] S5. Construct and train a convolutional neural network model; using the multi-source feature dataset constructed in S4 as input and the tree species category label as output, train the convolutional neural network model through iterative optimization to enable it to learn the nonlinear mapping relationship between spectral abundance features and vertical structure features.

[0012] S6. Use the trained convolutional neural network model to perform pixel-by-pixel classification and prediction of the region to be predicted, and generate a spatial distribution map of tree species diversity.

[0013] Preferably, the specific process for extracting texture features from the multispectral image in S2 is as follows:

[0014] The radiometrically calibrated and atmospherically corrected multispectral images are converted into grayscale images. Then, the sliding window size and the step size are set, and the spatial co-occurrence probability of pixel grayscale values ​​is counted in each window to construct a grayscale co-occurrence matrix.

[0015] Based on the gray-level co-occurrence matrix, four texture parameters—contrast, entropy, correlation, and homogeneity—are calculated respectively.

[0016] Wherein, the contrast characterizes the degree of local gray-level difference within the window, the entropy characterizes the randomness of gray-level distribution within the window, the correlation characterizes the linear dependence of gray-level of adjacent pixels within the window, and the homogeneity characterizes the uniformity of local gray-level within the window.

[0017] Preferably, the process of acquiring the canopy height model from airborne lidar point cloud data in S2 is as follows:

[0018] Noise point filtering and ground / non-ground point classification processing are performed on the raw point cloud data;

[0019] A digital elevation model (DEM) is generated based on interpolation of classified ground points, and a digital surface model (DSM) is generated based on interpolation of the highest point among non-ground points.

[0020] The canopy height model is obtained through difference operations, using the following formula: ;

[0021] in, Represents the canopy height model; Represents a digital surface model; This represents a digital elevation model.

[0022] Preferably, when using the fully constrained least squares method for spectral unmixing in S3, both the abundance sum-of-one constraint and the non-negativity constraint must be satisfied simultaneously. The specific constraint formula is as follows: ;in, Indicates the first The abundance value is the area ratio of each end-member in the current pixel. This indicates the total number of endmembers involved in the unmixing process; This indicates that the sum of all endmember abundances equals 1, ensuring that the pixel is completely decomposed;

[0023] Under the above constraints, the preliminary abundance of each tree species and non-vegetation endmembers can be obtained by minimizing the sum of squared residuals between the observed spectrum and the linear combination spectrum of endmembers.

[0024] Preferably, the specific process of introducing shadow endmembers in S3 to normalize and correct the preliminary abundance is as follows:

[0025] Because shadows in complex terrain absorb some incident light, the abundance of various tree species in the spectral unmixing results is underestimated. Therefore, the abundance of shadow endmembers is used to compensate for and correct the initial abundance of tree species. The correction formula is as follows: ;in, Indicates the corrected number Tree species abundance; The first term obtained by unmixing using the fully constrained least squares method is... Preliminary abundance of tree species; Indicates the abundance of shadow endons; This indicates the percentage of the effective illuminated area after removing the influence of shadows.

[0026] when When this occurs, it means the pixel is completely covered by shadow and has no effective spectral information. In this case, let... To avoid division by zero errors, the sum of abundance of all tree species after correction is normalized back to 1, eliminating the interference of topographic shadows on the spectral unmixing results.

[0027] Preferably, the specific rule for assigning tree species category labels to the multi-channel input data using ground quadrat survey data in S4 is as follows:

[0028] For each ground quadrat, the diameter at breast height (DBH) of all living trees within the quadrat is counted, and the cross-sectional area at DBH is calculated for each tree. The formula is as follows: ;in, Indicates the first The cross-sectional area at breast height of a tree; Indicates the first The diameter at breast height (DBH) of a tree;

[0029] The total basal area at breast height (DBH) of a tree species is obtained by summing up the DBH of all individuals of the same species. The tree species with the largest proportion of total DBH to total DBH of all tree species in the quadrat is defined as the dominant tree species, and the category of the dominant tree species is used as the label of the corresponding pixel. If there are no living trees in the quadrat or the DBH of all tree species is equal, it is marked as another category.

[0030] Preferably, the specific structure of the convolutional neural network model described in S5 is as follows:

[0031] It includes two convolutional blocks connected in sequence. Each convolutional block consists of a convolutional layer, a batch normalization layer, a ReLU activation function layer and a max pooling layer connected in sequence.

[0032] The first convolutional block uses 32 convolutional kernels of size 3×3, and the second convolutional block uses 64 convolutional kernels of size 3×3.

[0033] Two convolutional blocks are followed by a flattening layer, a first fully connected layer, a Dropout layer, and a second fully connected layer in sequence.

[0034] The first fully connected layer has 128 neurons, the second fully connected layer has the same number of neurons as the total number of tree species, and the output layer uses the Softmax activation function to convert the output into the probability distribution of each species.

[0035] Preferably, in S5, the loss function used when training the convolutional neural network model is the cross-entropy loss function, with the following formula: ;in, This represents the loss value for the current sample; Indicates the total number of tree species categories; This represents the one-hot encoded value of the true label, when the sample belongs to the first... Class Time ,otherwise ; The first output of the model represents the... The class prediction probability, with a value range of (0,1); Represents the natural logarithm operation;

[0036] During training, the goal is to minimize the average loss of a batch of samples, and the network parameters are updated using the backpropagation algorithm.

[0037] Preferably, this application provides the vegetation index calculation in S2, specifically as follows:

[0038] The vegetation indices include the Normalized Difference Vegetation Index (NDVI), the Soil-Adjusted Vegetation Index (SAVI), and the Ratio Vegetation Index (RVI), calculated using the following formulas: , , ;

[0039] in, Indicates near-infrared reflectivity. Indicates the reflectivity in the red light band. The soil brightness correction factor is set to 0.5;

[0040] Preferably, this application provides the structure of the multi-channel input data in S4, specifically as follows:

[0041] The multi-channel input data generated in S4 includes the tree species correction abundance channels obtained in S3, the canopy height model channel obtained in S2, the NDVI channel, the SAVI channel, the RVI channel, and four texture feature channels: contrast, entropy, correlation, and homogeneity, totaling [amount missing]. One channel, among which The number of main tree species involved in the unmixing process.

[0042] Compared with the prior art, the technical solution of this application has the following technical effects:

[0043] This invention completes sub-pixel decomposition of multispectral images through a linear spectral mixing model, which can quantify the abundance information of various tree species within each pixel. It also completes abundance normalization processing with a matching shadow correction rule, uniformly eliminating spectral data deviations caused by terrain shadows. The entire spectral unmixing process completes endmember matching based on a localized tree species spectral library, which can completely extract the spectral component features of subdivided tree species, expand the spectral information dimension of the model input, enrich the feature expression of subdivided vegetation types within the image, and provide refined spectral basic data for subsequent feature fusion.

[0044] This invention generates a canopy height model based on airborne lidar point clouds, and simultaneously extracts multiple vegetation indices and gray-level co-occurrence matrix texture features to form a multi-dimensional feature system covering spectrum, texture, and vertical structure. After unified coordinate and resolution registration, various features are superimposed and combined into multi-channel input data, realizing the complete fusion of remote sensing information in different dimensions, fully preserving the surface spectral information of vegetation and the vertical spatial structure information of the forest understory, enriching the feature hierarchy of the input data, and improving the feature's ability to fully represent the state of forest vegetation.

[0045] This invention constructs a convolutional neural network with a multi-layer convolutional structure. It autonomously extracts the spatial correlation information of features based on the convolutional and pooling layers, and completes the network constraints by combining batch normalization and random deactivation layers. During the training phase, a dedicated loss function and an adaptive optimizer are used to iteratively update the network parameters. The network can autonomously learn the nonlinear correlation between spectral abundance and canopy structure. There is no need for manual selection of combined features. The network completes deep feature mining on its own and stably outputs standardized tree species classification results.

[0046] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0047] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0049] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0050] Figure 1 A flowchart illustrating the overall technical process for extracting tree species diversity information based on multi-source remote sensing and deep learning;

[0051] Figure 2 A hierarchical logical framework diagram for multi-source data processing, dataset construction, model training, and graphing;

[0052] Figure 3 A visualization of the spatial distribution of canopy height across the entire study area, obtained from the inversion of airborne LiDAR point clouds.

[0053] Figure 4 A raster map showing the abundance distribution of individual tree species generated by unmixing a linear spectral mixing model for nine different tree species.

[0054] Figure 5 The complete network architecture diagram of the convolutional neural network used to input the unmixed abundance features of multispectral images and the CHM fusion input;

[0055] Figure 6 A heatmap of the confusion matrix generated by the convolutional neural network classification model for the prediction results of nine tree species.

[0056] Figure 7 To train the model and complete the global pixel-by-pixel classification output, a spatial distribution map of the diversity of nine tree species in the region is generated. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0058] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0059] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0060] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0061] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0062] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0063] Example 1

[0064] This embodiment mainly describes a method for extracting tree species diversity through collaborative multi-source remote sensing and deep learning, such as... Figure 1 As shown, it includes the following steps:

[0065] S1: Determine the study area and collect multi-source remote sensing data and ground-measured data; the multi-source remote sensing data includes multispectral imagery and airborne lidar point cloud data; the ground-measured data includes ground quadrat survey data and ground spectral acquisition data.

[0066] S2: Preprocess the multi-source remote sensing data and extract features; perform radiometric calibration and atmospheric correction on the multispectral image, calculate the vegetation index and extract texture features; generate digital elevation model and digital surface model from the airborne lidar point cloud data after denoising and classification, and obtain the canopy height model through differential operation.

[0067] S3: Based on the linear spectral mixing model, the preprocessed multispectral image is spectrally unmixed; the main tree species endmembers and non-vegetation endmembers are selected, and the fully constrained least squares method is used to unmix and obtain the preliminary abundance of each tree species. The shadow endmember is introduced to normalize and correct the preliminary abundance to obtain the corrected tree species abundance map.

[0068] S4: Construct a multi-source feature dataset; Spatially register and overlay the tree species abundance map obtained in S3 with the canopy height model obtained in S2 to form multi-channel input data that integrates spectral features and vertical structure features, and use ground quadrat survey data to assign tree species category labels to the multi-channel input data;

[0069] S5: Construct and train a convolutional neural network model; using the multi-source feature dataset constructed in S4 as input and the tree species category label as output, train the convolutional neural network model through iterative optimization to enable it to learn the nonlinear mapping relationship between spectral abundance features and vertical structure features;

[0070] S6: Use the trained convolutional neural network model to perform pixel-by-pixel classification and prediction of the region to be predicted, and generate a spatial distribution map of tree species diversity.

[0071] Based on the above steps, this application can be divided into a multi-source data and feature extraction module, a sample dataset construction module, and a model training and graphing result generation module, such as... Figure 2 As shown;

[0072] In the initial stage of the multi-source data and feature extraction module, the multispectral imagery clearly selected PlanetScope 3B grade product. This imagery contains 8 discrete bands (coastal blue, blue, green, yellow, red, red edge, near-infrared 1, near-infrared 2), with a spatial resolution of 3 meters and a revisit cycle of once a day. Its high temporal resolution characteristics ensure that high-quality images with no or few clouds can be obtained during the peak vegetation growth season (June-September) in the study area, avoiding interference from phenological differences on spectral characteristics.

[0073] The data acquisition parameters for airborne lidar point cloud data are set as follows: flight altitude 800 meters, forward overlap rate 60%, lateral overlap rate 50%, average point cloud density greater than 10 points / square meter, and complete echo count recording (maximum 4 echoes). This density is sufficient to penetrate the multi-layered canopy of subtropical evergreen broad-leaved forests and accurately capture the understory topography and canopy vertical structure.

[0074] Regarding ground-based measured data, 323 fixed quadrats of 3m×3m were laid out in the study area according to the system grid. The center coordinates of the quadrats were determined using RTK-GPS, with a horizontal positioning error of less than 5 cm and an elevation error of less than 10 cm, to ensure the geometric registration accuracy with the remote sensing image.

[0075] Within each quadrat, the species name, diameter at breast height (DBH), height, and relative location of all living trees with a DBH ≥ 5 cm were recorded. Canopy spectra of the major tree species were collected using an ASD FieldSpec 4 handheld ground spectrometer during the 10:00-14:00 period on a clear, cloudless day. The spectral range was 350-2500 nm, with a sampling interval of 1 nm. At least 30 spectral curves were collected for each tree species. After whiteboard correction and dark current subtraction, a localized endmember spectral library was constructed to replace the general spectral library in order to improve the physical authenticity of subsequent spectral unmixing.

[0076] The above data acquisition specifications were not arbitrarily chosen, but rather the optimal configuration verified through pre-experiments: 3-meter resolution matches the sample plot size to avoid scale mismatch; 10 points / square meter density ensures CHM accuracy is better than 0.5 meters; 30 spectral curves ensure endmember representativeness; RTK positioning eliminates label misalignment caused by geometric offset.

[0077] In the specific implementation of feature extraction, this application abandons the traditional approach of relying solely on the original spectral bands, and instead constructs a composite feature space that includes spectral indices, texture statistics, and three-dimensional structural parameters. For multispectral images, after radiometric calibration and FLAASH atmospheric correction to eliminate sensor response differences and the effects of atmospheric scattering and absorption, six types of vegetation indices and eight types of texture features were calculated, and their specific mathematical models are shown in the table below. The calculation of these features is based on preprocessed surface reflectance data, and texture analysis is performed within a 3×3 sliding window. The window size matches the 3-meter pixel size, which preserves local details while suppressing noise.

[0078]

[0079] In the SAVI formula, 0.5 is an empirical constant used to balance soil and vegetation signals; in the Entropy formula... Specifically, this refers to the natural logarithm, not the base-10 logarithm, which directly affects the numerical range of the eigenvalues ​​and the convergence behavior of the subsequent model. Before calculating texture features, the red band of the multispectral image is converted into an 8-bit grayscale image, and then a normalized co-occurrence matrix is ​​constructed. The direction is taken as the average of four directions: 0°, 45°, 90°, and 135° to eliminate direction dependence.

[0080] For airborne LiDAR point cloud data, the processing flow is equally meticulous and rigorous. The Statistical OutlierRemoval algorithm is used to remove outlier noise points. This algorithm calculates the average distance and standard deviation of points in the neighborhood of the point cloud, identifies points that exceed the threshold range as noise and removes them, ensuring the accuracy of subsequent processing.

[0081] The point cloud was accurately divided into ground points and non-ground points using the support vector machine (SVM) classifier. The SVM model used the radial basis kernel function, and the training samples came from manually labeled typical ground and vegetation point cloud fragments. The classification accuracy reached more than 98% after cross-validation.

[0082] A digital elevation model (DEM) is generated based on the classified ground points using an inverse distance weighted interpolation algorithm. A digital surface model (DSM) is generated based on the highest point among the non-ground points using the same interpolation algorithm. The difference between the two models generates a canopy height model (CHM). The calculation formula is as follows: This CHM not only provides absolute tree height information, but more importantly, as an independent structural channel, it can effectively distinguish tree species with similar spectra but significant height differences, such as... Figure 3 As shown;

[0083] The CHM color map visually demonstrates the continuous variation in forest stand height in the study area, with colors gradually changing from blue (low vegetation) to red (tall trees). Its spatial details are perfectly aligned with the PlanetScope image, and the geometric registration error is controlled within 0.5 pixels, laying a reliable geometric foundation for subsequent feature fusion. Furthermore, the ground-acquired spectral data underwent Savitzky-Golay smoothing filtering to remove noise, and resampling was performed based on the center wavelength and half-width of each band of the multispectral image to form a localized endmember spectral library consistent with the multispectral image band response. This resampling process employed the Gaussian response function integral method to ensure the physical consistency between the endmember spectra and the image data.

[0084] In the spectral unmixing stage, this application introduces a shadow endmember correction mechanism to address the abundance underestimation problem caused by shadows in complex terrain. Endmember selection is not simply chosen from a spectral library; instead, a pure pixel index method is used to automatically extract candidate endmembers from the preprocessed multispectral imagery. These endmembers are then compared and screened against a ground-measured spectral library, ultimately determining seven main tree species endmembers (pine, fir, camphor, maple, *Symplocos edulis*, *Koelreuteria bipinnata*, and bamboo), one type of bare soil endmember, and one type of shadow endmember, for a total of nine endmembers to participate in the unmixing process. The fully constrained least squares method is used to solve for the conditions that satisfy… and The initial abundance, of which For the first The abundance of each endmember is determined. This optimization problem is solved using a quadratic programming algorithm, ensuring the uniqueness and physical plausibility of the solution. Shading correction is performed immediately after unmixing. ,in For the corrected tree species abundance, For shadow endmember abundance, the physical meaning of the formula is: to redistribute the spurious area occupied by shadows to other land features, restoring the abundance proportion under true lighting conditions. When When a pixel is determined to be in full shadow, all pixels are then... ;when Linear interpolation is used to smooth the transition and avoid abrupt changes in abundance after correction. After correction, a normalization check is performed on all pixels. (Due to floating-point operation errors), the result is scaled proportionally to a sum of 1. For example... Figure 4 As shown, the corrected abundance maps of (pine, fir, camphor, maple, Michelia champaca, Koelreuteria bipinnata, bamboo forest, other woodlands, and others) exhibit reasonable spatial continuity, with no abnormally low-value patches caused by topographic shading, and the abundance value range is consistent with the actual vegetation cover, proving the effectiveness and necessity of the correction strategy.

[0085] The key to the sample construction phase lies in the objectivity, representativeness, and spatial independence of the labels. Each 3m×3m quadrat corresponds to one of nine labels (pine, fir, camphor, maple, *Symplocos macrantha*, *Koelreuteria bipinnata*, bamboo forest, other woodlands, and others). Label assignment strictly follows the principle of dominance based on the basal area at breast height (DBH): the DBH basal area of ​​each tree is calculated. ,in For the first The diameter at breast height (DBH) of a single tree; summing the DBH of all individuals of the same species. The total basal area at breast height (DBH) of the tree species was obtained. Finally, the tree species with the largest proportion of DBH among all tree species within the quadrat was defined as the dominant species, and its category was used as the label for the corresponding pixel. This biomass-based labeling reflects ecological dominance and community structure better than simple tree counts. If there were no living trees in the quadrat or all tree species had equal DBH, they were labeled as "Other." When dividing the samples into the training set (258 samples) and the test set (65 samples) at an 8:2 ratio, a spatial stratified sampling strategy was specifically adopted to ensure the consistency of the distribution of the training and test sets across environmental gradients such as altitude, aspect, and vegetation type, avoiding data leakage and overfitting risks caused by neighboring quadrats entering the same set. In the final sample set, each sample is a 16-channel feature vector (7 corrected abundances, 1 CHM, 6 vegetation indices, and 2 core texture features Contrast and Entropy), with compact dimensions and complete information.

[0086] Model construction is the core technology of this application, and its design fully considers the characteristics of multi-source remote sensing data and the requirements of tree species classification tasks. For example... Figure 5 As shown, the convolutional neural network used is not a general architecture, but a CBR-Res1D dual-branch structure designed specifically for multi-source remote sensing features.

[0087] The left branch uses the standard Conv1D-BatchNorm-ReLU-MaxPool sequence. The first convolutional block uses 32 convolutional kernels of size 3×3 to extract shallow spectral-texture patterns; the second convolutional block uses 64 convolutional kernels of size 3×3 to extract deep semantic features.

[0088] The right branch connects to the AdaptiveAvgPool1D layer after the same two convolutional blocks, compressing the variable-length features into a fixed-length representation to capture global context information and long-range dependencies. The outputs of the two branches are concatenated along the channel dimension, regularized by a Dropout layer (ratio 0.5), and then fed into a two-stage fully connected classifier: the first fully connected layer has 128 neurons and uses the ReLU activation function for feature integration and nonlinear transformation; the second fully connected layer has 9 neurons (corresponding to 9 tree species categories), and the output layer uses the Softmax activation function to convert the output into the probability distribution of each category.

[0089] The advantages of this dual-branch design are: the left branch retains fine-grained discriminative features, which is suitable for distinguishing tree species with similar spectra; the right branch provides coarse-grained semantic support, which is suitable for recognizing large-scale spatial patterns; the two complement each other, enhancing the model's robustness to scale changes and its adaptability to complex scenes.

[0090] Cross-entropy loss function is used during training. ,in The total number of categories, One-hot encoding of the real label. The first output of the model Class prediction probabilities. The optimizer used was Adam, with an initial learning rate of 0.001, a batch size of 32, and a maximum training epoch of 100. Experimental results showed that placing the Dropout layer between two fully connected layers was more effective than placing it after a convolutional layer, as fully connected layers have denser parameters and are more prone to overfitting. During training, the classification accuracy on the validation set was continuously monitored. Training was terminated early when the validation set accuracy did not improve for 10 consecutive epochs, and the model parameters with the highest validation set accuracy were saved as the final model to avoid overfitting and wasting computational resources.

[0091] like Figure 6As shown, the confusion matrix of the model on the test set clearly reveals the recognition ability and misclassification patterns of each category. The diagonal elements are generally high, with pine reaching 0.92, indicating that its combination of spectral and structural features is highly discriminative; cedar is 0.88 and camphor is 0.85, both at relatively high levels; bamboo is 0.75, slightly lower because its spectrum overlaps with some broad-leaved trees, but the unique height information provided by CHM still makes it significantly better than pure spectral classification. The overall accuracy of 86% (56 / 65) is not only evident in the numerical value itself, but also in the sparsity and rationality of the off-diagonal elements in the confusion matrix. The classification is mainly concentrated in ecological transition zones or ambiguous quadrat boundaries, rather than systematic confusion, indicating that the model has learned the true species discrimination boundaries rather than data spurious correlations. The Kappa coefficient of 0.83 further confirms the reliability and consistency of the classification results.

[0092] like Figure 7 As shown, the generated spatial distribution map of tree species diversity presents an ecological pattern consistent with field observations: pines and firs are concentrated on ridges and sunny slopes, adapted to arid and barren environments; camphor trees are distributed in shady and damp areas of valleys, preferring moist and fertile soil; bamboo forests are patchy in areas with frequent human activity; other woodlands fill abandoned or degraded forest areas. This map is not only a classification result but also quantifiable ecological baseline data, directly supporting biodiversity assessment, forest resource inventory, and management decisions, demonstrating the practical application value of the technical solution.

[0093] This implementation achieves high-precision extraction of tree species diversity in complex terrain through multi-source data collaboration, physical model correction, customized network design, and rigorous verification closed loop.

[0094] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for extracting tree species diversity through collaborative multi-source remote sensing and deep learning, characterized in that, Includes the following steps: S1. Determine the study area and collect multi-source remote sensing data and ground-measured data; the multi-source remote sensing data includes multispectral imagery and airborne lidar point cloud data; the ground-measured data includes ground quadrat survey data and ground spectral acquisition data; S2. Preprocess the multi-source remote sensing data and extract features; After radiometric calibration and atmospheric correction of the multispectral image, vegetation index is calculated and texture features are extracted. The airborne lidar point cloud data is processed by denoising and classification to generate digital elevation models and digital surface models, and the canopy height model is obtained through differential operations. S3. Perform spectral demixing on the preprocessed multispectral image based on a linear spectral mixing model; The main tree species endmembers and non-vegetation endmembers were selected, and the preliminary abundance of each tree species was obtained by unmixing using the fully constrained least squares method. The preliminary abundance was then normalized and corrected by introducing the shadow endmembers to obtain the corrected tree species abundance map. S4. Construct a multi-source feature dataset; Spatially register and overlay the tree species abundance map obtained in S3 with the canopy height model obtained in step S2 to form multi-channel input data that integrates spectral features and vertical structure features, and use ground quadrat survey data to assign tree species category labels to the multi-channel input data; S5. Construct and train a convolutional neural network model; using the multi-source feature dataset constructed in S4 as input and the tree species category label as output, train the convolutional neural network model through iterative optimization to enable it to learn the nonlinear mapping relationship between spectral abundance features and vertical structure features. S6. Use the trained convolutional neural network model to perform pixel-by-pixel classification and prediction of the region to be predicted, and generate a spatial distribution map of tree species diversity.

2. The method according to claim 1, characterized in that, The specific process for extracting texture features from multispectral images as described in S2 is as follows: The radiometrically calibrated and atmospherically corrected multispectral images are converted into grayscale images. Then, the sliding window size and the step size are set, and the spatial co-occurrence probability of pixel grayscale values ​​is counted in each window to construct a grayscale co-occurrence matrix. Based on the gray-level co-occurrence matrix, four texture parameters—contrast, entropy, correlation, and homogeneity—are calculated respectively. Wherein, the contrast characterizes the degree of local gray-level difference within the window, the entropy characterizes the randomness of gray-level distribution within the window, the correlation characterizes the linear dependence of gray-level of adjacent pixels within the window, and the homogeneity characterizes the uniformity of local gray-level within the window.

3. The method according to claim 2, characterized in that, The process of acquiring the canopy height model from airborne lidar point cloud data, as described in S2, is as follows: Noise point filtering and ground / non-ground point classification processing are performed on the raw point cloud data; A digital elevation model (DEM) is generated based on interpolation of classified ground points, and a digital surface model (DSM) is generated based on interpolation of the highest point among non-ground points. The canopy height model is obtained through difference operations, using the following formula: ; in, Represents the canopy height model; Represents a digital surface model; This represents a digital elevation model.

4. The method according to claim 3, characterized in that, When using the fully constrained least squares method for spectral unmixing in S3, both the abundance sum-of-one constraint and the non-negativity constraint must be satisfied simultaneously. The specific constraint formulas are as follows: ;in, Indicates the first The abundance value is the area ratio of each end-member in the current pixel. This indicates the total number of endmembers involved in the unmixing process; This indicates that the sum of all endmember abundances equals 1, ensuring that the pixel is completely decomposed; Under the above constraints, the preliminary abundance of each tree species and non-vegetation endmembers can be obtained by minimizing the sum of squared residuals between the observed spectrum and the linear combination spectrum of endmembers.

5. The method according to claim 4, characterized in that, The specific process of introducing shadow endmembers in S3 to normalize and correct the preliminary abundance is as follows: Because shadows in complex terrain absorb some incident light, the abundance of various tree species in the spectral unmixing results is underestimated. Therefore, the abundance of shadow endmembers is used to compensate for and correct the initial abundance of tree species. The correction formula is as follows: ;in, Indicates the corrected number Tree species abundance; The first term obtained by unmixing using the fully constrained least squares method is... Preliminary abundance of tree species; Indicates the abundance of shadow endons; This indicates the percentage of the effective illuminated area after removing the influence of shadows. when When this occurs, it means the pixel is completely covered by shadow and has no effective spectral information. In this case, let... To avoid division by zero errors, the sum of abundance of all tree species after correction is normalized back to 1, eliminating the interference of topographic shadows on the spectral unmixing results.

6. The method according to claim 5, characterized in that, The specific rules for assigning tree species category labels to the multi-channel input data using ground quadrat survey data in S4 are as follows: For each ground quadrat, the diameter at breast height (DBH) of all living trees within the quadrat is counted, and the cross-sectional area at DBH is calculated for each tree. The formula is as follows: ;in, Indicates the first The cross-sectional area at breast height of a tree; Indicates the first The diameter at breast height (DBH) of a tree; The total basal area at breast height (DBH) of a tree species is obtained by summing up the DBH of all individuals of the same species. The tree species with the largest proportion of total DBH to total DBH of all tree species in the quadrat is defined as the dominant tree species, and the category of the dominant tree species is used as the label of the corresponding pixel. If there are no living trees in the quadrat or the DBH of all tree species is equal, it is marked as another category.

7. The method according to claim 6, characterized in that, The specific structure of the convolutional neural network model described in S5 is as follows: It includes two convolutional blocks connected in sequence. Each convolutional block consists of a convolutional layer, a batch normalization layer, a ReLU activation function layer and a max pooling layer connected in sequence. The first convolutional block uses 32 convolutional kernels of size 3×3, and the second convolutional block uses 64 convolutional kernels of size 3×3. Two convolutional blocks are followed by a flattening layer, a first fully connected layer, a Dropout layer, and a second fully connected layer in sequence. The first fully connected layer has 128 neurons, the second fully connected layer has the same number of neurons as the total number of tree species, and the output layer uses the Softmax activation function to convert the output into the probability distribution of each species.

8. The method according to claim 7, characterized in that, In S5, the loss function used when training the convolutional neural network model is the cross-entropy loss function, with the following formula: ;in, This represents the loss value for the current sample; Indicates the total number of tree species categories; This represents the one-hot encoded value of the true label, when the sample belongs to the first... Class Time ,otherwise ; The first output of the model represents the... The class prediction probability, with a value range of (0,1); Represents the natural logarithm operation; During training, the goal is to minimize the average loss of a batch of samples, and the network parameters are updated using the backpropagation algorithm.

Images