Intelligent identification method and system for tree species of urban forest

Abstract

The invention provides an intelligent identification method and system for tree species of an urban forest, and relates to the technical field of image processing, and the method comprises the steps: carrying out the preprocessing of original hyperspectral image data and original inclined image data; performing characteristic wave band extraction on the target hyperspectral image data through a minimum noise separation algorithm to obtain characteristic wave band extraction data; performing vegetation extraction on the target hyperspectral image data according to the characteristic wave band extraction data to obtain a vegetation mask; according to an urban forest canopy height model, through a multi-scale segmentation algorithm, single tree segmentation is carried out on tree species in the target oblique photography image data; a tree species intelligent identification model based on a double-flow HybrdSN network is constructed; using the training data set to train the tree species intelligent identification model; and inputting the to-be-identified hyperspectral image data and the to-be-identified oblique photography image data into the trained tree species intelligent identification model for intelligent identification, and outputting an identification result of the tree species of the urban forest.

Description

A method and system for intelligent tree species identification in urban forests Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for intelligent tree species identification in urban forests. Background Technology

[0002] As a core component of the urban ecosystem, urban forests not only regulate regional climate, purify air, and conserve water resources, but also provide green spaces for residents' leisure and recreation, playing an irreplaceable role in enhancing the city's ecological carrying capacity and residents' quality of life. Accurately identifying the dominant tree species in urban forests and understanding their spatial distribution characteristics are prerequisites for carrying out urban forest resource management, ecological planning, and conservation work.

[0003] Existing methods for tree species identification mainly fall into two categories: one is based on single hyperspectral remote sensing data, which extracts spectral features from hyperspectral images and combines them with traditional machine learning algorithms such as support vector machines and random forests or basic convolutional neural network models to complete classification; the other is based on the fusion of optical images and elevation data, which uses orthophotos generated by UAV oblique photography and digital surface models to extract spatial features such as canopy morphology and texture of trees to assist in classification. Some studies have also introduced classic hybrid convolutional models such as HybridSN, which combine 3D and 2D convolutions to achieve joint extraction of spectral-spatial features from hyperspectral data, achieving certain results in forest tree species identification tasks.

[0004] However, existing technologies do not fully integrate hyperspectral spectral features with canopy three-dimensional structural features, the multi-source data fusion method is unreasonable, it is difficult to distinguish tree species with similar spectra but significant morphological differences, and there is a lack of feature optimization mechanisms and refined training strategies. The models are prone to overfitting due to noise band interference, and the generalization ability of the models is insufficient. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide an intelligent tree species identification method for urban forests, which can solve the technical problems of the prior art, such as insufficient integration of hyperspectral spectral features and canopy three-dimensional structural features, unreasonable multi-source data fusion method, difficulty in distinguishing tree species with similar spectra but significant morphological differences, lack of feature optimization mechanism and refined training strategy, easy overfitting due to noise band interference, and insufficient model generalization ability.

[0006] A first aspect of this invention proposes a method for intelligent tree species identification in urban forests, comprising:

[0007] S1: Acquire raw hyperspectral image data and raw oblique photogrammetric image data of tree species in urban forests;

[0008] S2: Preprocess the raw hyperspectral image data and raw oblique image data to obtain target hyperspectral image data and target oblique photographic image data;

[0009] S3: The feature bands of the target hyperspectral image data are extracted using the minimum noise separation algorithm to obtain the feature band extraction data;

[0010] S4: Based on the data extracted from the characteristic bands, vegetation is extracted from the target hyperspectral image data to obtain a vegetation mask;

[0011] S5: Generate an urban forest canopy height model based on the target oblique photographic image data and vegetation mask;

[0012] S6: Based on the urban forest canopy height model, the tree species in the target oblique photographic image data are segmented into individual trees using a multi-scale segmentation algorithm to obtain individual tree segmentation data;

[0013] S7: Perform data augmentation on the single-tree segmentation data to construct a training dataset;

[0014] S8: Construct a tree species intelligent identification model based on a dual-stream HybridSN network;

[0015] S9: Train the tree species intelligent identification model using the training dataset;

[0016] S10: Acquire the hyperspectral image data and oblique photogrammetric image data of the tree species to be identified in the urban forest, and input the hyperspectral image data and oblique photogrammetric image data to be identified into the trained tree species intelligent identification model for intelligent identification, and output the identification results of the tree species in the urban forest.

[0017] A second aspect of this invention provides an intelligent tree species identification system for urban forests, comprising: a processor and a memory;

[0018] The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the intelligent tree species identification method for urban forests as described in the first aspect.

[0019] A third aspect of the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the intelligent tree species identification method for urban forests as described in the first aspect.

[0020] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0021] In this embodiment of the invention, by constructing a tree species intelligent identification model based on a dual-stream HybridSN network, hyperspectral spectral features and canopy three-dimensional structural features can be fully integrated, further optimizing the multi-source data fusion mode and achieving accurate differentiation of tree species with similar spectra but significant morphological differences. At the same time, feature band extraction is completed by using the minimum noise separation algorithm, combined with data augmentation processing of single tree segmentation data and refined training of the tree species intelligent identification model, effectively reducing noise interference, avoiding model overfitting, and significantly improving the model's generalization ability. Attached Figure Description

[0022] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0023] Figure 1 is a flowchart illustrating a method for intelligent tree species identification in urban forests according to an embodiment of the present invention.

[0024] Figure 2 is a model diagram of a dual-stream HybridSN network provided in an embodiment of the present invention.

[0025] Figure 3 shows the training accuracy curve of a dual-stream HybridSN network provided in an embodiment of the present invention.

[0026] Figure 4 is a schematic diagram of the structure of an intelligent tree species identification system for urban forests provided in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] The intelligent tree species identification method for urban forests provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0029] Referring to Figure 1 in the specification, a flowchart illustrating an intelligent tree species identification method for urban forests provided by an embodiment of the present invention is shown.

[0030] Referring to Figure 2 in the specification, a model diagram of a dual-stream HybridSN network provided in an embodiment of the present invention is shown.

[0031] Referring to Figure 3 in the specification, a training accuracy curve of a dual-stream HybridSN network provided in an embodiment of the present invention is shown.

[0032] This invention provides a method for intelligent tree species identification in urban forests, which may include the following steps:

[0033] S1: Acquire raw hyperspectral image data and raw oblique photogrammetric image data of tree species in urban forests.

[0034] It should be noted that the data on trees in the urban forest were obtained by using hyperspectral and oblique photogrammetry sensors mounted on the UAV. Specifically, the hyperspectral and oblique photogrammetry imaging sensors were used to collect data over the study area by using a DJI matrix 300 RTK UAV. The hyperspectral imaging sensor was a Specim AFX10, and the oblique photogrammetry imaging sensor was a PSDK 102S V3 camera.

[0035] Furthermore, the hyperspectral data has a spatial resolution of 0.03 m and a wavelength range of [missing information]. The spectral resolution is 5.5 nm, and the number of spectral bands is 224. The oblique photography data resolution is 6144×4096, and the pixel size is 3.76μm. Flight path planning was performed using DJI Pilot 2, and the entire operation adopted flight path mode with automatic equidistant photo capture to ensure efficient image acquisition.

[0036] S2: Preprocess the original hyperspectral image data and the original oblique image data to obtain the target hyperspectral image data and the target oblique photographic image data.

[0037] It should be noted that the acquired raw hyperspectral image data undergoes preprocessing such as radiometric calibration, geometric correction, and atmospheric correction. The acquired raw oblique image data undergoes preprocessing such as aerial triangulation to generate a true-color orthophoto (DOM) and a digital surface model (DSM).

[0038] Furthermore, using the Smooth function to perform spectral smoothing on the raw hyperspectral image data helps reduce noise in the data, making it easier to analyze and interpret.

[0039] In this embodiment of the invention, radiometric calibration, geometric correction, atmospheric correction, and spectral smoothing preprocessing are performed on the original hyperspectral image data. Aerial triangulation is performed on the original oblique image data to generate true-color orthophotos and digital surface models. This not only eliminates radiometric distortion, geometric deviation, and atmospheric scattering interference in the hyperspectral data, reduces spectral noise, and improves the spectral purity and spatial consistency of the hyperspectral data, but also transforms the oblique image data into orthophotos and elevation models with accurate spatial coordinate information. This provides standardized and high-precision basic data for subsequent feature band extraction, vegetation mask generation, canopy height model construction, and individual tree segmentation. It effectively avoids the adverse effects of errors and noise in the original data on the training and classification accuracy of subsequent tree species identification models, ensuring the stability and reliability of the entire urban forest dominant tree species identification process.

[0040] S3: The feature bands of the target hyperspectral image data are extracted using the minimum noise separation algorithm to obtain the feature band extraction data.

[0041] The minimum noise separation algorithm is a preprocessing algorithm for dimensionality reduction and noise separation of hyperspectral image data. Through statistical analysis and linear transformation, it decouples the effective information from noise in the hyperspectral data, thereby extracting feature bands with concentrated information and low noise content. First, the hyperspectral data is organized into a matrix of "number of bands × number of pixels," and a covariance matrix reflecting the correlation and information distribution between bands is calculated. Then, the covariance matrix is ​​decomposed into eigenvalues, yielding a set of eigenvalues ​​and corresponding eigenvectors. The magnitude of the eigenvalue represents the amount of information carried by the corresponding eigenvector (the larger the eigenvalue, the higher the information content and the lower the noise content). Next, the eigenvalues ​​are sorted from smallest to largest, and a transformation matrix is ​​constructed using the eigenvectors corresponding to the minimum eigenvalues. This matrix is ​​used to project the original hyperspectral data into a new feature space, minimizing the noise variance. Finally, feature bands with high information content are selected in the new space, preserving the core spectral-spatial features of the hyperspectral data while effectively eliminating noise interference, providing high-quality data support for subsequent tasks such as vegetation extraction and tree species identification.

[0042] In one possible implementation, S3 specifically includes sub-steps S301 to S305:

[0043] S301: Construct the covariance matrix of the target hyperspectral image data.

[0044] The covariance matrix is ​​specifically:

[0045]

[0046]

[0047] Where C represents the covariance matrix, X represents the target hyperspectral image data, T represents the transpose operation, M represents the number of bands, and N represents the number of pixels.

[0048] S302: Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and eigenvectors.

[0049] S303: Sort the feature values ​​in ascending order and select a preset number of minimum feature values.

[0050] Those skilled in the art can set the preset quantity according to actual needs, and the present invention does not limit this.

[0051] S304: Construct the transformation matrix of the minimum noise separation algorithm based on the minimum eigenvalue.

[0052] S305 projects the target hyperspectral image data into the space of the transformation matrix to obtain feature band extraction data.

[0053] It should be noted that projecting the target hyperspectral image data onto the new space of the Minimum Noise Separation (MNF) transformation matrix yields the data after minimum noise separation. In this new space, the data in each band are more independent of each other, thus effectively reducing the impact of noise. This helps to highlight the key information in the data.

[0054] Furthermore, the 30 feature bands with the highest eigenvalues ​​were extracted using the Minimum Noise Separation (MNF) algorithm, with a cumulative variance contribution rate exceeding 99%.

[0055] In this embodiment of the invention, the feature bands of the preprocessed target hyperspectral image data are extracted using a minimum noise separation algorithm. Through a series of operations such as covariance matrix construction, eigenvalue decomposition, and transformation matrix projection, the effective information and noise of the hyperspectral data can be decoupled and separated. This not only allows for the selection of feature bands with concentrated information, low noise content, and independent characteristics (e.g., the cumulative variance contribution rate of the 30 extracted feature bands exceeds 99%), but also removes redundant noise interference while preserving the core spectral-spatial features to the greatest extent. Furthermore, it significantly reduces the computational complexity of subsequent vegetation extraction and model training, avoiding the dimensionality curse caused by excessively high dimensionality of hyperspectral data bands. This provides high-quality and cost-effective data support for subsequent vegetation mask generation, spectral feature extraction of the dual-stream HybridSN model, and accurate tree species identification, thereby improving the efficiency and accuracy of the entire identification process.

[0056] S4: Based on the data extracted from the characteristic bands, vegetation is extracted from the target hyperspectral image data to obtain a vegetation mask.

[0057] Vegetation mask is a type of binary image used in remote sensing image processing to accurately distinguish between vegetated and non-vegetated areas in an image. It is usually generated based on the vegetation index calculated from the image data and by setting a reasonable classification threshold. Vegetated areas are marked as 1, while non-vegetated areas such as buildings, bare land, and roads are marked as 0.

[0058] In one possible implementation, S4 specifically includes sub-steps S401 and S402:

[0059] S401: Calculate the red-edge normalized vegetation index based on data extracted from characteristic bands.

[0060] Specifically, the red-edged normalized vegetation index is as follows:

[0061]

[0062] Wherein, NDVI represents the red-edge normalized vegetation index, ρ 750 ρ represents the wavelength reflectance in the 750nm band. 705 This indicates the wavelength reflectance in the 705nm band.

[0063] It should be noted that using the red-edge normalized vegetation index (NRVI) to extract vegetation from hyperspectral images is a slight modification of the traditional NRVI and is quite sensitive to chlorophyll content.

[0064] S402: Based on the red-edge normalized vegetation index, vegetation is extracted from the target hyperspectral image data to obtain a vegetation mask.

[0065] The specific formula for calculating the vegetation cover is as follows:

[0066]

[0067] Where M(x,y) represents the vegetation mask, (x,y) represents the pixel coordinates of the feature band extracted data, and τ represents the classification threshold between vegetation and background.

[0068] It should be noted that vegetated areas are marked as foreground (value=1), and other areas are marked as background (value=0).

[0069] In this embodiment of the invention, based on data extracted from feature bands, a vegetation mask is generated by calculating the red-edge normalized vegetation index, which is sensitive to chlorophyll content, and setting a classification threshold. This can accurately and efficiently distinguish vegetated areas from non-vegetated areas such as buildings and bare land in remote sensing images. The binarization method, which marks vegetated areas as 1 and non-vegetated areas as 0, can provide accurate regional selection criteria for subsequent digital terrain model construction, eliminate the interference of vegetation canopy on surface elevation calculation, narrow the processing range of subsequent single tree segmentation and tree species identification, reduce the amount of computation of invalid data, improve the targeting and efficiency of the entire urban forest dominant tree species identification process, and ensure the accuracy of subsequent canopy height model generation and model training.

[0070] S5: Generate an urban forest canopy height model based on target oblique photographic image data and vegetation mask.

[0071] The urban forest canopy height model is an elevation model reflecting the vertical height from the top of the tree canopy to the ground surface in urban forests. It is obtained by subtracting the difference between a Digital Surface Model (DSM) and a Digital Terrain Model (DTM). The DSM includes the highest elevations of all features, including tree canopies and buildings, while the DTM reflects only the true undulations of the ground surface after removing all interference from features. Subtracting the two models accurately yields the canopy height for each pixel. The urban forest canopy height model is crucial intermediate data in the urban forest tree species identification process. It not only provides height-dimensional feature support for individual tree segmentation, helping to distinguish adjacent trees with similar spectral characteristics, but also serves as the core input to the structural feature stream of the dual-stream HybridSN model. Combined with the DSM, it extracts three-dimensional morphological features of the canopy, thereby improving the accuracy of tree species identification and providing data support for urban forest planning and management.

[0072] It should be noted that, based on the characteristics of flat urban terrain and small changes in adjacent surface height, a digital terrain model (DTM) is generated by interpolating to preserve the surface elevation values ​​to reflect the actual terrain characteristics of the study area. The urban forest canopy height model is then generated by combining the digital surface model (DSM) and the digital terrain model (DTM).

[0073] In one possible implementation, S5 specifically includes sub-steps S501 to S503:

[0074] S501: Generate an initial digital terrain model based on the vegetation mask.

[0075] The initial digital terrain model is specifically as follows:

[0076]

[0077] Among them, DTM initial(x,y) represents the initial digital terrain model, DSM non-veg (x,y) represents the elevation value of the non-vegetated area, and T represents the threshold.

[0078] It should be noted that a mask is applied to non-surface areas to preserve the surface elevation values ​​of other areas. First, the elevation values ​​of non-vegetated areas are preserved in the digital surface model using the vegetation extraction results. Then, a threshold is set. When the elevation value of a non-vegetated area is greater than the threshold, it is regarded as a non-surface elevation and is masked, while the remaining digital surface model is regarded as the surface elevation value. This generates the initial digital terrain model.

[0079] S502: By using the inverse distance weighted interpolation method, the occluded areas in the initial digital terrain model are estimated and filled to obtain the digital terrain model.

[0080] Among them, the inverse distance weighted interpolation method is a deterministic interpolation algorithm based on the spatial correlation principle that "the closer the known point is, the greater its influence on the interpolation point." It is often used to supplement missing or occluded areas in spatial data such as terrain and elevation. In the process of building urban forest canopy height models, it is mainly used to improve the initial digital terrain model. First, the straight-line distance between the interpolation point and all surrounding known surface elevation points is calculated. Then, the reciprocal of the distance is used as the weight, with closer known points having higher weights. Subsequently, the elevation value of the interpolation point is estimated by weighted averaging, ultimately generating a complete and continuous digital terrain model without gaps, providing a foundation for accurate calculation of canopy height.

[0081] Specifically, the digital terrain model includes:

[0082]

[0083]

[0084] Where DTM(x,y) represents the digital terrain model, h j This represents the j-th known elevation value, where n represents the number of known points, and d... pj Let k represent the distance between the p-th interpolation point and the j-th known point, and k represent the weighting exponent. p ,y p (x) represents the p-th interpolation point. j ,y j ) represents the j-th known point.

[0085] S503: Generate an urban forest canopy height model based on the digital terrain model and the digital surface model in the target oblique photogrammetric image data.

[0086] Among them, the Digital Surface Model (DSM) is a spatial model that reflects the absolute elevation of the ground surface. It records the elevation information of the tops of all ground features (including tree canopies, buildings, roads, and bare ground) within the study area, fully presenting the three-dimensional morphological characteristics of the ground surface and ground features. In the process of identifying dominant tree species in urban forests, the DSM is usually generated from oblique photogrammetry sensor data preprocessing and serves as the foundation for subsequent core data production.

[0087] The urban forest canopy height model is as follows:

[0088]

[0089] Wherein, CHM(x,y) represents the urban forest canopy height model, and DSM(x,y) represents the digital surface model.

[0090] In this embodiment of the invention, by combining target oblique photographic image data with vegetation masks, an initial digital terrain model is generated by filtering the elevation of non-vegetated areas using vegetation masks. Inverse distance weighted interpolation is then used to complete the occluded areas, resulting in a complete digital terrain model. This model is then compared with the digital surface model to generate an urban forest canopy height model. This not only accurately separates the elevation information of tree canopies from the ground surface, obtaining spatial distribution data reflecting the true height of each tree, thus overcoming the limitation of a single digital surface model in distinguishing between canopies and the ground surface, but also provides height-dimensional feature support for subsequent individual tree segmentation. This effectively distinguishes adjacent trees with similar spectral characteristics but significant height differences. Furthermore, as the core input of the structural feature stream of the dual-stream HybridSN model, it is combined with the digital surface model to extract three-dimensional morphological features of the tree canopy, significantly improving the accuracy of tree species identification and providing reliable elevation data support for the refined management and planning of urban forest resources.

[0091] S6: Based on the urban forest canopy height model, the tree species in the target oblique photographic image data are segmented into individual trees using a multi-scale segmentation algorithm to obtain individual tree segmentation data.

[0092] Multi-scale segmentation algorithms are a region segmentation technique for remote sensing images. Based on the similarity of spectral, spatial, and texture features of the images, they aggregate pixels into irregular object units with homogeneous attributes. Furthermore, different granularity segmentation results can be generated by adjusting scale parameters. In the process of identifying dominant tree species in urban forests, true-color orthophotos generated by oblique photography, digital surface models, and canopy height models are typically used as joint inputs. By setting appropriate spectral weights, shape weights, and compactness weights, adjacent pixels with similar spectral features and consistent height and shape are aggregated into individual tree objects. This accurately delineates the canopy boundaries of each tree, obtaining individual tree segmentation data. This provides precise spatial unit support for subsequent visual labeling of tree species, spectral spatial cube clipping, and model training sample construction.

[0093] It should be noted that Trimble eCognition Developer 10.3 software uses a multi-scale segmentation algorithm to segment individual trees.

[0094] Furthermore, using true-color orthophotos, digital surface model (DSM), and canopy height model (CHM) from the target oblique photographic data as input layers, individual trees are segmented by combining color, texture, and height information. The multi-resolution segmentation algorithm includes four important parameters: layer weights, scale, shape index, and compactness. The setting of segmentation parameters has a significant impact on the segmentation results. After repeated experiments and visual comparisons with different parameter combinations, the optimal combination was determined to be: layer weights = [1,1,1,2], scale parameter = 500, shape index = 0.2, and compactness = 0.7.

[0095] In this embodiment of the invention, a multi-scale segmentation algorithm is used to segment individual trees in target oblique photographic image data by combining an urban forest canopy height model and optimizing parameter combinations. This integrates the spectral texture features of true-color orthophotos, the elevation features of digital surface models, and the canopy height features of canopy height models, accurately aggregating adjacent and homogeneous pixel units to delineate the canopy boundary of each tree, resulting in high-precision individual tree segmentation data. This step not only solves the problems of easily confusing adjacent trees and blurring boundaries in traditional segmentation methods, but also provides accurate spatial unit support for subsequent tree species visual labeling, spectral spatial cube clipping, and training set construction, significantly improving the efficiency and accuracy of sample labeling and laying a data foundation for high-quality training of the HybridSN model.

[0096] S7: Based on the single-tree segmentation data, construct the training dataset through data augmentation processing.

[0097] It should be noted that the data after individual tree segmentation is labeled with tree species and then cropped. The original image is segmented into a series of spectral space cubes with labels of size S×S×L, using the geometric center pixel of the sample as the center. S×S represents the size of the neighborhood, and L represents the number of bands. Data augmentation is used to increase the number of training samples by rotating them by 90°, 180°, and 270° respectively, and mirroring the rotated samples and the original samples along the diagonal. This increases the number of training samples to eight times the original.

[0098] In one possible implementation, S7 specifically refers to:

[0099] Tree species were labeled on the individual tree segmentation data using visual interpretation, and the number of training samples was increased by data augmentation to obtain the training dataset.

[0100] Visual interpretation is a commonly used manual interpretation and annotation method in remote sensing image and geospatial data processing. It refers to the operator's professional knowledge of vegetation classification and tree species morphology, combined with the spectral information of hyperspectral imagery, the spatial texture information of oblique photography, and the canopy outline data after individual tree segmentation. Through human-computer interaction, each segmented tree object is examined and identified one by one, and its corresponding tree species category label is assigned. At the same time, the labeled individual tree areas are cropped according to the preset sample specifications, and finally labeled training, verification and test samples are formed, which provide accurate category reference for the subsequent training of the dual-stream HybridSN model.

[0101] In this embodiment of the invention, the tree species are accurately labeled and the spectral space cube is cropped to a preset specification using a visual interpretation method for the single tree segmentation data. The number of samples is increased to eight times the original number through geometric transformations such as rotation and mirroring. This not only provides high-quality training samples with accurate category labels and uniform specifications for the dual-stream HybridSN model, but also effectively expands the sample size and enriches the spatial morphological diversity of the samples through data augmentation. This alleviates the overfitting problem caused by insufficient sample quantity or uneven distribution during model training. At the same time, the accurate cropping based on the single tree segmentation boundary can avoid interference from non-target region pixels, greatly improving the model's targeting and effectiveness in learning tree species features. This lays a solid data foundation for the high accuracy and strong robustness of subsequent model training.

[0102] S8: Construct a tree species intelligent identification model based on a dual-stream HybridSN network.

[0103] The HybridSN network is a hybrid convolutional deep learning architecture optimized for hyperspectral remote sensing image classification. It extracts spectral and spatial information collaboratively through two parallel feature streams, then fuses the outputs to improve classification accuracy, making it particularly suitable for identifying dominant tree species in urban forests. One stream is the spectral-spatial stream, which takes the spectral space cube of the hyperspectral image as input and extracts pixel spectral features and two-dimensional spatial neighborhood features through 3D convolutional layers, capturing the spectral response differences and local spatial correlations of tree canopies. The other stream is the structural feature stream, which takes elevation data from a Digital Surface Model (DSM) or Canopy Height Model (CHM) as input and extracts structural features such as three-dimensional canopy morphology and height distribution through 2D convolutional layers, compensating for the shortcomings in distinguishing similar spectral features. The two streams of features are pooled and flattened before being fed into a fully connected layer. The softmax function outputs the tree species probability of a single tree object, achieving complementary fusion of spectral and structural features. This significantly improves the robustness and accuracy of tree species identification in complex urban forest scenarios, providing precise classification basis for urban forest planning and management.

[0104] It should be noted that the original HybridSN model adopts a two-stream architecture: a spectral-spatial feature stream processes the spectral and spatial features of hyperspectral images, while a structural feature stream processes the height and morphological features of tree canopies in oblique photography. Furthermore, band attention is added, allowing the network to automatically learn the importance of each band and thus assign different weights to different bands.

[0105] Furthermore, the network employs a dual-stream parallel architecture to process spectral-spatial features and canopy structure features separately. The spectral-spatial feature stream, based on an improved HybridSN framework, first introduces a band attention mechanism to adaptively learn the weights of important spectral bands, then extracts joint spectral-spatial features through three layers of 3D convolutional kernels, and further extracts spatial features through 2D convolutions, finally outputting a feature vector. The structure feature stream specifically processes canopy height models (CHM), digital surface models (DSM), and their derived features as input, extracting three-dimensional morphological features of the canopy through a multi-layer 2D convolutional network, outputting a feature vector. The features from the two branches are concatenated and fused in an intermediate layer, avoiding information interference from early fusion and insufficient feature interaction from late fusion, and finally, tree species classification is completed through a fully connected layer.

[0106] In this embodiment of the invention, a tree species intelligent identification model based on a dual-stream HybridSN network is constructed. By innovatively designing a parallel architecture of spectral spatial flow and structural feature flow, it not only utilizes the band attention mechanism and 3D / 2D hybrid convolution of the spectral spatial flow to accurately capture the core spectral-spatial joint features of hyperspectral data, but also relies on the 2D convolution of the structural feature flow to extract the three-dimensional morphological features of the canopy from the canopy height model and digital surface model. At the same time, by using the intermediate layer feature splicing and fusion method, it effectively avoids the problems of information interference in early fusion and insufficient feature interaction in late fusion, making up for the shortcomings of single spectral features in distinguishing morphologically similar tree species. This significantly improves the accuracy and robustness of dominant tree species identification in complex urban forest scenarios, providing efficient and reliable model support for the intelligent and refined identification of tree species in urban forests.

[0107] S9: Use the training dataset to train the tree species intelligent recognition model.

[0108] It should be noted that relevant parameters are defined and initialized with a fixed learning rate of 0.001. A learning rate scheduling strategy is employed: if the validation loss does not decrease for eight consecutive epochs, the learning rate is halved. The maximum number of iterations is set to 80, the number of images processed in a batch is set to 32, the input image size is 15×15 pixels, and the early stopping tolerance value is 15; if the validation accuracy does not improve for 15 consecutive epochs, training is terminated early. The network uses the multi-class cross-entropy loss function as the optimization objective. The optimizer uses the AdamW algorithm to optimize the weights. After each training epoch, the network calculates the gradient and updates the weight parameters through backpropagation to gradually reduce the loss value, saving the weight parameter model with the minimum loss.

[0109] In one possible implementation, S9 specifically includes sub-steps S901 to S903:

[0110] S901: Input the extracted feature band data into the tree species intelligent identification model to obtain the hyperspectral feature vector.

[0111] In one possible implementation, S901 specifically includes steps S9011 to S9014:

[0112] S9011: Weighted processing of feature band extracted data is performed through a band attention mechanism.

[0113] The band attention mechanism is a deep learning module for optimizing hyperspectral image features. Its core function is to enable the model to adaptively identify and strengthen spectral bands that are more critical to the task (such as tree species classification) while suppressing interference from noisy or irrelevant bands. In the spectral-spatial feature stream of the dual-stream HybridSN network, it typically contains two fully connected layers. First, the input hyperspectral band features are transformed. Then, the output is mapped to a weight range of [0,1] using a Sigmoid activation function, generating attention weights for each spectral band. The closer the weight value is to 1, the more effective classification information the band contains; the closer it is to 0, the higher the noise content or redundancy of the band. The generated attention weights are then multiplied band-by-band by the original hyperspectral data to perform weighted processing, allowing subsequent convolutional layers to focus on high-value spectral features, thereby improving the model's ability to distinguish spectrally similar tree species.

[0114] S9012: Through a three-dimensional convolutional layer, scale features are extracted from the weighted feature band extraction data to obtain a three-dimensional feature map.

[0115] S9013: Reshape a 3D feature map into a 2D feature map.

[0116] S9014: Spatial features are extracted and flattened from the two-dimensional feature map through a two-dimensional convolutional layer to obtain a hyperspectral feature vector.

[0117] It should be noted that the feature band extracted data is input into the spectral-spatial feature stream. First, a band attention mechanism is used, which consists of two fully connected layers and generates attention weights in the range [0,1] using a sigmoid activation function. The weighted spectral data undergoes three layers of 3D convolution processing: the first layer uses eight 7×3×3 convolutional kernels, the second layer uses sixteen 5×3×3 convolutional kernels, and the third layer uses thirty-two 3×3×3 convolutional kernels to extract features at different scales. Subsequently, the 3D features are reconstructed into a 2D feature map, and spatial features are extracted through a 2D convolutional layer (64 3×3 convolutional kernels). Finally, the features are flattened.

[0118] S902: Input the target oblique photographic image data and the data from the urban forest canopy height model into the tree species intelligent identification model to obtain the structural feature vector.

[0119] In one possible implementation, S902 specifically includes steps S9021 and S9022:

[0120] S9021: Through a two-dimensional convolutional layer, features are extracted from the target oblique photographic image data and the data in the urban forest canopy height model to obtain the initial structural feature vector.

[0121] S9022: The initial structural feature vector is compressed and flattened by global average pooling to obtain the structural feature vector.

[0122] Global average pooling is a feature dimensionality reduction and integration operation used in deep learning models. Its core function is to compress the high-dimensional feature map output by convolutional layers, reducing the number of parameters and avoiding overfitting while retaining key feature information. In the structural feature stream of the dual-stream HybridSN network, specifically, it calculates the average value of all pixels in each channel of the feature map output by the two-dimensional convolutional layer, transforming the original two-dimensional feature map with spatial dimension into a single numerical value, and then integrating the values ​​of all channels into a one-dimensional feature vector. Compared with the dimensionality reduction of traditional fully connected layers, global average pooling does not require learning additional weight parameters and can better preserve the spatial invariance of features. In canopy structure feature extraction tasks, it can effectively aggregate global features of canopy height and morphology, providing compact and robust structural feature support for subsequent feature fusion and tree species classification.

[0123] It should be noted that the target oblique photographic image data and the data from the urban forest canopy height model are input into the structural feature stream. The input data is passed through a two-layer 2D convolutional network. The first layer uses 16 3×3 convolutional kernels, and the second layer uses 32 3×3 convolutional kernels. Then, global average pooling is used to obtain the feature vector, and finally the features are flattened.

[0124] S903: The hyperspectral feature vector and structural feature vector are concatenated and their dimensions reduced to obtain the trained tree species intelligent recognition model.

[0125] Furthermore, the fusion layer combines two flattened features and then uses a fully connected layer to progressively reduce the dimensionality of the features, finally outputting a predicted classification map for 10 categories (including background, grass, and other categories).

[0126] In this embodiment of the invention, optimized parameter configuration is adopted, and the dual-stream HybirdSN tree species intelligent identification model is trained by combining the multi-class cross-entropy loss function and the AdamW optimizer. Hyperspectral feature vectors are extracted by band attention weighting and 3D / 2D convolution of the spectral spatial flow, and canopy structure feature vectors are extracted by 2D convolution and global average pooling of the structural feature flow. Then, the model is optimized by feature concatenation and dimensionality reduction. This not only enables the model to accurately learn the complementary relationship between the spectral-spatial joint features of hyperspectral data and the three-dimensional morphological features of the canopy, but also effectively avoids overfitting and improves training efficiency through learning rate scheduling and early stopping strategies. At the same time, the weights are iteratively updated based on backpropagation and the model parameters with the minimum loss are saved to ensure that the trained model has high accuracy and strong robustness, and can efficiently complete the accurate identification of dominant tree species in urban forests.

[0127] S10: Acquire the hyperspectral image data and oblique photogrammetric image data of the tree species to be identified in the urban forest, and input the hyperspectral image data and oblique photogrammetric image data to be identified into the trained tree species intelligent identification model for intelligent identification, and output the identification results of the tree species in the urban forest.

[0128] It should be noted that the model with the highest accuracy on the validation set was used to identify tree species in the hyperspectral and oblique photographic images of the test set. The model predicted all images containing tree species information, generating corresponding predicted images and forming a tree species prediction map for the entire study area. The overall accuracy was 90.87%, and the Kappa coefficient was 0.89. Finally, the same operation was performed on the original HybirdSN model, with an overall accuracy of 86.43% and a Kappa coefficient of 0.85.

[0129] Furthermore, using the dual-stream HybridSN model compared to the original model resulted in a 4.44% increase in accuracy and a 0.04 increase in the Kappa coefficient. After obtaining tree species identification images, information can be extracted from the tree species images to output the approximate location and species type of urban forest trees, providing a reference for urban planning and management.

[0130] Referring to Figure 4 in the specification, a structural schematic diagram of an intelligent tree species identification system for urban forests provided by an embodiment of the present invention is shown.

[0131] This invention provides an intelligent tree species identification system 20 for urban forests, comprising: a processor 201 and a memory 202;

[0132] The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-described intelligent tree species identification method for urban forests and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0133] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0134] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0135] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0136] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0137] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0139] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0142] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] This invention provides a readable storage medium comprising: storing a program or instructions on the readable storage medium, wherein when the program or instructions are executed by a processor, the program or instructions implement the steps of the above-described intelligent tree species identification method for urban forests and achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent tree species identification in urban forests, characterized in that, include: S1: Acquire raw hyperspectral image data and raw oblique photographic image data of tree species in urban forests; S2: Preprocess the raw hyperspectral image data and the raw oblique image data to obtain target hyperspectral image data and target oblique photographic image data; S3: Extract feature bands from the target hyperspectral image data using a minimum noise separation algorithm to obtain feature band extracted data. S4: Based on the data extracted from the characteristic bands, vegetation is extracted from the hyperspectral image data of the target to obtain a vegetation mask; S5: Based on the oblique photographic image data of the target and the vegetation mask, an urban forest canopy height model is generated; S6: Based on the urban forest canopy height model, a multi-scale segmentation algorithm is used to segment individual tree species in the oblique photographic image data of the target to obtain individual tree segmentation data; S7: Based on the single-tree segmentation data, construct a training dataset through data augmentation processing; S8: Construct a tree species intelligent identification model based on a dual-stream HybridSN network; S9: Use the training dataset to train the tree species intelligent identification model; S10: Obtain the hyperspectral image data and oblique photographic image data of the tree species to be identified in the urban forest, and input the hyperspectral image data and oblique photographic image data of the tree species to be identified into the trained tree species intelligent identification model for intelligent identification, and output the identification result of the tree species in the urban forest.

2. The method for intelligent tree species identification in urban forests according to claim 1, characterized in that, S3 specifically includes: S301: Constructing the covariance matrix of the target hyperspectral image data; S302: Performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and eigenvectors; S303: Sort the eigenvalues ​​in ascending order and selecting a preset number of minimum eigenvalues; S304: Constructing the transformation matrix of the minimum noise separation algorithm based on the minimum eigenvalues; S305: Projecting the target hyperspectral image data onto the space of the transformation matrix to obtain the feature band extraction data.

3. The intelligent tree species identification method for urban forests according to claim 1, characterized in that, S4 specifically includes: S401: Calculating the red-edge normalized vegetation index based on the data extracted from the characteristic bands; S402: Extracting vegetation from the target hyperspectral image data based on the red-edge normalized vegetation index to obtain the vegetation mask.

4. The intelligent tree species identification method for urban forests according to claim 1, characterized in that, S5 specifically includes: S501: generating an initial digital terrain model based on the vegetation mask; S502: estimating and filling the occluded areas in the initial digital terrain model using an inverse distance weighted interpolation method to obtain a digital terrain model; S503: generating the urban forest canopy height model based on the digital terrain model and the digital surface model in the target oblique photographic image data.

5. The intelligent tree species identification method for urban forests according to claim 1, characterized in that, Specifically, S7 involves: visually interpreting the individual tree segmentation data to label the tree species, and using data augmentation to increase the number of training samples to obtain the training dataset.

6. The intelligent tree species identification method for urban forests according to claim 1, characterized in that, S9 specifically includes: S901: inputting the extracted feature band data into the tree species intelligent identification model to obtain a hyperspectral feature vector; S902: inputting the target oblique photographic image data and the data from the urban forest canopy height model into the tree species intelligent identification model to obtain a structural feature vector; S903: concatenating the hyperspectral feature vector and the structural feature vector and reducing their dimensionality to obtain the trained tree species intelligent identification model.

7. The method for intelligent tree species identification in urban forests according to claim 6, characterized in that, S901 specifically includes: S9011: weighting the feature band extraction data through a band attention mechanism; S9012: extracting scale features from the weighted feature band extraction data through a three-dimensional convolutional layer to obtain a three-dimensional feature map; S9013: reshaping the three-dimensional feature map into a two-dimensional feature map; S9014: extracting spatial features from the two-dimensional feature map and flattening it through a two-dimensional convolutional layer to obtain the hyperspectral feature vector.

8. The intelligent tree species identification method for urban forests according to claim 6, characterized in that, S902 specifically includes: S9021: using a two-dimensional convolutional layer to extract features from the target oblique photographic image data and the data in the urban forest canopy height model to obtain an initial structural feature vector; S9022: using global average pooling to compress and flatten the initial structural feature vector to obtain the structural feature vector.

9. A tree species intelligent identification system for urban forests, characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the intelligent tree species identification method for urban forests as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the intelligent tree species identification method for urban forests as described in any one of claims 1 to 8.

Images