Urban tree species identification method, system and device and storage medium

Abstract

The invention discloses an urban tree species identification method, system and device and a storage medium, and the method comprises the steps: carrying out the shadow identification of an urban region according to multi-channel satellite remote sensing image data, and extracting the information data of the shadow region; performing shadow area spectrum recovery according to the shadow area information to obtain an image after shadow recovery; and carrying out tree species identification on the image after shadow recovery. Furthermore, an improved MSASDNet-M model is provided for shadow recognition, a fusion method of multiband radiation correction and boundary smoothing is provided for shadow region spectrum recovery, and an NAM-DeepLabV3 + network model is provided for tree species recognition. According to the method, the shadow region identification precision, the spectrum compensation effect and the overall tree species identification precision can be remarkably improved. Meanwhile, the method has high cross-domain adaptability and can be widely applied to tree species monitoring and ecological planning of other urban green lands.

Description

Technical Field

[0001] This invention belongs to the field of intelligent recognition and deep learning of remote sensing images, and particularly relates to a method, system, device and storage medium for identifying urban tree species. Background Technology

[0002] With the acceleration of global urbanization, urban forests, as an important component of urban ecosystems, play an irreplaceable role in improving urban climate, mitigating the heat island effect, enhancing air quality, and improving the living environment for residents. Accurately obtaining information on the distribution of urban tree species is fundamental to urban ecological planning, carbon sequestration assessment, and landscape design.

[0003] Currently, urban tree species identification mainly relies on high-resolution aerial imagery or satellite remote sensing data, employing traditional machine learning or deep learning methods for automatic identification. However, in high-resolution urban remote sensing imagery, the presence of complex structures such as tall buildings and overlapping tree canopies makes shadow interference a key issue affecting the accuracy of tree species identification. In shadowed areas, pixel brightness is significantly reduced, and the spectral curve shifts downwards overall, blurring the spectral differences between vegetation areas and between vegetation and the background. This results in a significant decrease in the model's ability to identify tree species within shadowed regions.

[0004] In existing technologies, shadow processing mainly includes the following categories:

[0005] (1) Masking method: The shaded areas are directly removed, and classification is performed only in the illuminated areas. However, this method results in incomplete classification results and fails to reflect the continuity of urban vegetation.

[0006] (2) Histogram matching and physical radiation model: The shadow area is restored by brightness correction and physical modeling, but in complex urban environments, the boundary transition is unnatural or over-enhanced.

[0007] (3) Shadow recognition based on deep learning: using convolutional neural networks to recognize shadows in urban areas, but most models are based on only RGB three-channel input and lack multispectral feature support, making it difficult to accurately distinguish shadows from dark vegetation under complex lighting conditions.

[0008] Furthermore, there are significant differences in spectral response between different sensor data, which causes the performance of deep learning models to degrade when applied across domains. Even with pre-trained models, a complex domain adaptation process is still required to maintain accuracy.

[0009] In summary, existing urban tree species identification technologies have shortcomings in terms of shadow recognition accuracy, spectral compensation effectiveness, and cross-domain adaptability. Therefore, there is an urgent need for a deep learning framework that can fully utilize multispectral information, possess shadow perception and recovery capabilities, and achieve high-precision cross-domain classification. Summary of the Invention

[0010] Objective of this invention: The objective of this invention is to provide a method for identifying urban tree species. Through multiple innovations at the data layer, model structure layer, and optimization layer, it significantly improves the identification accuracy of shaded areas and the overall classification performance, achieving high-precision, continuous, and cross-domain adaptive classification of urban tree species. Another objective of this invention is to provide an urban tree species identification system, electronic device, and computer-readable storage medium.

[0011] Technical solution: The urban tree species identification method includes the following steps:

[0012] Shadow identification in urban areas is performed based on multi-channel satellite remote sensing image data, and shadow area information data is extracted;

[0013] Based on the shadow region information, spectral restoration of the shadow region is performed to obtain the shadow-restored image;

[0014] Tree species identification is performed on the image after shadow restoration.

[0015] The multi-channel satellite remote sensing image data in this invention includes cases with more than three channels, which means that compared with traditional RGB remote sensing images, it supports the use and processing of higher resolution remote sensing image data.

[0016] Preferably, the multi-channel satellite remote sensing image data uses six channels including deep blue, blue, green, red, red-edge, and near-infrared bands. Among these, the deep blue band, due to its penetrating power, can effectively reveal details within shadows, aiding in more accurate shadow boundary judgment and internal information extraction. This invention uses the entire band for tree species identification experiments, which better reflects the spectral characteristics of vegetation compared to traditional RGB data. In particular, the red-edge and near-infrared bands play a crucial role in vegetation health monitoring and tree species identification.

[0017] Preferably, the satellite remote sensing image data uses Pléiades high-resolution imagery.

[0018] Optionally, the shadow recognition of the urban area employs a deep convolutional neural network to extract features from multi-channel satellite remote sensing image data and to locate shadow areas through network learning.

[0019] Preferably, the deep convolutional neural network adopts the MSASDNet-M (MSASDNet-Multispectralenhanced) model. This model expands the input structure from three-channel data to six-band data based on the original MSASDNet model, that is, the input includes six bands: deep blue, blue, green, red, red edge, and near-infrared. Using pre-trained weights from the AISD (Aerial Imagery dataset for Shadow Detection) dataset, a channel duplication and mean-filling strategy is used to initialize the pre-trained weights of the three newly added channels. A two-stage transfer learning strategy is used to transfer the model weights pre-trained on the AISD dataset to the target dataset.

[0020] Optionally, during the training of the MSASDNet-M model, the loss function used is weighted cross-entropy, in which different weights are assigned to the shadow class and the non-shadow class.

[0021] Optionally, the loss function used is weighted cross-entropy, where different weights are assigned to the shadow class and the non-shadow class.

[0022] Optionally, the pre-trained weights of the three newly added channels are initialized as shown in the following formula:

[0023]

[0024] in, Here are the weights of the original RGB channels of the convolution kernel, and i is the index variable used for summation. i = 1, 2, 3 correspond to the three channel dimensions of the original 3-channel convolution kernel, respectively. The initial weights for the three newly added channels are given, where c is the index of the new channel, and c∈{4,5,6} represents the newly added 4th, 5th, and 6th channels.

[0025] Optionally, the two-stage transfer learning strategy includes: in the first stage, freezing the lower layers of the backbone network and training only the newly added input layer and classification head to enable the model to initially adapt to multispectral input; in the second stage, gradually unfreezing the network and combining cosine annealing learning rate scheduling to gradually adapt to the target domain data distribution and achieve full network adaptation to the target domain.

[0026] Optionally, the weighted cross-entropy loss function is expressed as follows:

[0027]

[0028] in, , These are the weights for shadow and non-shadow classes, respectively. For real labels, To predict probabilities, The total number of samples.

[0029] Optionally, the spectral restoration of the shadow region adopts a general radiometric correction model that can flexibly support any number of bands. It is a multi-band generalization based on the original radiometric correction model, including: calculating the radiometric gain coefficient of each band according to the shadow region information and the maximum gain constraint, using the radiometric gain coefficient to perform multi-band radiometric correction on the multi-channel satellite remote sensing image data, and then performing shadow boundary smoothing processing to obtain the shadow restored image.

[0030] Optionally, the shadow area information uses a shadow mask. For each band, the mean radiation ratio of the shadowed area to the unshadowed area in the shadow mask is calculated, and the radiation gain coefficient of each band is calculated according to the following general radiation correction model. :

[0031]

[0032] in, This represents the radiation correction gain coefficient for band b. This represents the mean radiation value of the unshaded area in band b. The mean radiation value representing the shadow region of band b;

[0033] Add maximum gain constraint To prevent excessive recovery;

[0034] Gaussian convolution is used to blur the shadow mask after multi-band radiometric correction, converting it into a continuous grayscale mask, thereby generating a smooth transition region at the boundary between shadow and non-shadow areas, ensuring spectral consistency between shadow and non-shadow areas.

[0035] Preferably, the maximum gain constraint .

[0036] Optionally, the tree species identification adopts the NAM-DeepLabV3+ network model, which expands the input channels to six bands based on the DeepLabV3+ classifier; an improved multi-scale atrous spatial pyramid pooling (ASPP) module is embedded in the deep layer of the encoder, combined with a two-layer asymmetric attention mechanism; the decoder adopts a residual-enhanced shallow feature fusion structure for high-fidelity restoration of texture and edge details.

[0037] Optionally, the encoder combined with a two-layer asymmetric attention mechanism includes: embedding NAM modules in the shallow and deep layers of the encoder respectively, wherein the shallow NAM module retains only the channel attention path and generates channel weights through global average pooling and two fully connected networks; the deep NAM module retains only the spatial attention path and is used to model long-distance dependencies between pixels.

[0038] Optionally, the deep NAM module introduces a spatial attention mechanism based on the Query–Key–Value mechanism.

[0039] Optionally, the improved multi-scale dilated spatial pyramid pooling ASPP module is a lightweight ASPP module, retaining only the core branches from the original ASPP module: 1×1 convolution, 3×3 dilated convolution (dilation rates of 6, 12, and 18), and image-level global pooling.

[0040] Optionally, the decoder employs a residual-enhanced shallow feature fusion structure, which, during the decoding stage, introduces an upsampling process into the shallow features after weighting by the NAM (Normalized Attention Module) module through the residual path, effectively improving the spatial consistency of the tree canopy boundary and small target regions.

[0041] Optionally, the loss function used to train the NAM-DeepLabV3+ network model adopts a hybrid form of weighted cross-entropy, Dice loss, and Focal loss.

[0042] Optionally, the loss function is defined as:

[0043]

[0044] in, This represents the final mixed loss value, which is the objective function that needs to be minimized during model training. , , : Represents the balancing weight coefficient, which is a hyperparameter between [0, 1] used to adjust the contribution ratio of the three loss terms to the total loss. : Represents weighted cross-entropy loss. It is an improved form of cross-entropy loss, calculating a weight coefficient for each class (tree species) (usually related to the inverse of the class frequency). Its purpose is to solve the global class imbalance problem, preventing the model from over-biasing predictions to the majority class (such as common tree species) with more pixels, while assigning higher weights to the minority classes with fewer pixels, thereby improving the model's basic ability to identify rare tree species. : Represents Dice loss. Inspired by the Dice coefficient, it directly optimizes the overlap area between the predicted and ground truth regions. It is highly sensitive to the boundaries of small objects. In tree classification, it effectively improves the classification accuracy for tree crown edge pixels and small areas of rare tree species, preventing these areas from being ignored by the model. : Represents Focal loss. It is an improved form of cross-entropy loss, adjusted by a factor. This automatically reduces the contribution of easily classifiable samples (such as large areas of solid color background or common simple tree canopies) to the loss function, allowing the model to focus more on difficult-to-classify samples (such as tree canopies obscured by shadows, regions with blurred spectral features, or regions similar to surrounding tree species). This helps the model discover difficult samples and further improve classification performance.

[0045] Preferably, = 0.5, =0.3, =0.2. This setting indicates that the training relies more on weighted cross-entropy in the initial stage to stabilize the training, and the effects of Dice and Focal losses are gradually increased to optimize the details.

[0046] The urban tree species identification system includes:

[0047] The shadow recognition module is used to identify shadows in urban areas based on multi-channel satellite remote sensing image data and extract shadow area information data.

[0048] The shadow area spectral restoration module is used to perform spectral restoration of the shadow area based on the shadow area information data extracted by the shadow recognition module, so as to obtain the shadow-restored image;

[0049] The tree species identification module is used to identify the tree species in the image after the shadow is restored.

[0050] The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements some or all of the steps of the above-described urban tree species identification method.

[0051] The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements some or all of the steps of the above-described urban tree species identification method.

[0052] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0053] 1. Based on high-resolution satellite remote sensing image data, the shadow areas of the entire image are directly restored and removed. After shadow restoration, the entire image is classified, eliminating the need for separate training for shadow and non-shadow areas. Through feature-level and spectral-level coupling between modules, an end-to-end optimizable shadow perception learning system is formed. This not only saves computing power but also solves the technical problem of low recognition accuracy in traditional tree species identification methods due to significant shadow interference in high-resolution remote sensing images, which makes it difficult to accurately distinguish between shadow areas and tree species. Experiments show that the urban tree species distribution map generated using this invention significantly outperforms existing methods in terms of overall coherence and shadow area restoration quality.

[0054] 2. This invention proposes the innovative MSASDNet-M model, achieving a breakthrough improvement in shadow recognition accuracy. Traditional MSASDNet models focus on salient target detection in RGB images, while this invention uses multi-channel satellite remote sensing imagery as input, expanding the data beyond traditional RGB data to better reflect vegetation spectral characteristics. However, its core advantage does not merely stem from using multispectral input, but rather from its original solution to the key bottleneck in combining multispectral input with transfer learning, thereby unlocking the potential of multispectral data. Experimental results show that the model using this invention achieves a precision value of 98.18%, an F1 score of 83.28%, and a BER reduced to 11.27%.

[0055] 3. Through multi-band radiometric correction and boundary smoothing, the problem of spectral distortion in the shadow area is effectively solved. The spectral compensation in the shadow area is natural, the boundary transition is smooth, and the PSNR (peak signal-to-noise ratio) is improved by 4.2 dB.

[0056] 4. Based on the improved DeepLabV3+ network and dual NAM attention mechanism, the overall accuracy of tree species classification is improved, with an overall mIoU improvement of 6.57% and mPA improvement of nearly 6%.

[0057] 5. This invention has strong cross-domain adaptability and can be widely applied in urban ecological monitoring, carbon sink estimation, urban heat island effect analysis, and urban landscape planning. Through parameter fine-tuning and transfer learning, it can be extended to other cities and images from different sensors, achieving high-precision tree species identification across platforms. Attached Figure Description

[0058] Figure 1 This is a schematic flowchart of the method of the present invention;

[0059] Figure 2 To illustrate the differences in shadow recognition performance across different experimental schemes;

[0060] Figure 3The image shows the overall shadow recognition results of MSASDNet-M and a magnified local area (red box).

[0061] Figure 4 The shadow restoration effects of the two methods in a typical area;

[0062] Figure 5 This demonstrates the overall shadow area restoration effect of the proposed method on the image.

[0063] Figure 6 For the improved DeepLabV3+ network architecture (NAM-DeepLabV3+);

[0064] Figure 7 A comparison of the impact of shadow restoration processing on tree species identification in high-resolution remote sensing images;

[0065] Figure 8 This is a final tree species classification map of a certain urban area, generated using the optimal model (6-channel + dual NAM attention mechanism + improved loss function). Detailed Implementation

[0066] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0067] This invention provides a method for identifying urban tree species, comprising the following steps:

[0068] Shadow identification in urban areas is performed based on multi-channel satellite remote sensing image data, and shadow area information data is extracted;

[0069] Based on the shadow region information, spectral restoration of the shadow region is performed to obtain the shadow-restored image;

[0070] Tree species identification is performed on the image after shadow restoration.

[0071] Accordingly, the present invention also provides an urban tree species identification system, comprising:

[0072] The shadow recognition module is used to identify shadows in urban areas based on multi-channel satellite remote sensing image data and extract shadow area information data.

[0073] The shadow area spectral restoration module is used to perform spectral restoration of the shadow area based on the shadow area information data extracted by the shadow recognition module, so as to obtain the shadow-restored image;

[0074] The tree species identification module is used to identify the tree species in the image after the shadow is restored.

[0075] Example 1

[0076] The method framework of this invention is applicable to multispectral satellite remote sensing images with varying numbers of channels. However, considering the data source and the balance between computing power and performance, a six-channel high-resolution Pléiades multispectral image (Pléiades-6C) is preferred as the input source image, including deep blue, blue, green, red, red-edge, and near-infrared band data. The image resolution is 0.3 meters, and the coordinate system is WGS_1984_UTM_Zone_50N.

[0077] As a preferred implementation, the images are standardized, histogram matched, and band registered before shadow recognition in urban areas to eliminate spectral differences.

[0078] The MSASDNet-M model is used for shadow region identification, and a shadow mask is output.

[0079] The spectral restoration of the shadow region includes: performing radiometric restoration on the shadow region in the shadow mask to generate a shadow-restored image.

[0080] The NAM-DeepLabV3+ model was used to identify tree species in the shadow-restored images, and the distribution map of each tree species was output.

[0081] The multi-index evaluation module calculates classification performance metrics and automatically saves the optimal weight model.

[0082] Example 2

[0083] In urban remote sensing scenarios, shadow sources mainly include the occlusion of tall buildings and the shadows formed by overlapping tree canopies. These areas typically exhibit reduced spectral brightness, loss of texture detail, and inconsistent performance across different wavelengths. Traditional three-channel RGB models, which only utilize the red, green, and blue visible light bands to represent ground features, have significant limitations in shadow detection.

[0084] 1. The visible light reflectance of the shadow area is greatly reduced, making it easy to misjudge the shadow as a dark object such as asphalt road or dark green vegetation;

[0085] 2. The effect of shadows is uneven across different bands, and the RGB model cannot utilize the ground feature difference information carried in near-infrared or red edge bands;

[0086] 3. Shadows in high-resolution images often cover complex textures and structures, such as building facades and tree canopy layers, making it difficult for traditional models to recreate their true forms.

[0087] To achieve high-precision shadow detection for Pléiades high-resolution multispectral imagery, it is necessary to overcome the spectral limitations of the three-channel model. Since the original MSASDNet model and its pre-trained weights (e.g., based on the AISD dataset) are only adapted to three-channel (R, G, B) input, when directly applied to six-channel Pléiades imagery containing new bands such as deep blue, red edges, and near-infrared, the convolutional kernel weights for the new channels lack effective initialization criteria. Random initialization would destroy the low-level feature extraction capabilities already learned by the pre-trained model, leading to severe oscillations and convergence difficulties in the early stages of model training, making it difficult to effectively fuse multispectral information.

[0088] Furthermore, the source domain (AISD, RGB aerial imagery) and the target domain (Pléiades, six-channel satellite imagery) differ significantly in spatial resolution, spectral response function, and imaging conditions, leading to a sharp drop in model generalization performance due to these cross-sensor domain differences. The experiments later in this embodiment also demonstrate (as shown in Exp1 of Table 1) that direct zero-sample transfer learning severely degrades model performance (F-Score only 39.22%), indicating a significant mismatch between the feature extraction pattern of the pre-trained model and the target domain data distribution, rendering it unsuitable for direct application.

[0089] Therefore, to address the challenges of weight initialization and training stability caused by channel expansion, this invention proposes an improved MSASDNet-M model for shadow recognition and shadow recognition modules in urban areas. This model systematically improves upon the original MSASDNet model, specifically in terms of input structure, parameter initialization, transfer learning strategy, and loss function design, to adapt to the spectral characteristics of Pléiades multispectral images and the actual needs of urban shadow recognition tasks.

[0090] (1) Input structure expansion

[0091] Traditional models only support three-channel input, which limits the utilization of multispectral information. To fully exploit the spectral features in images, this embodiment expands the input to six bands: Deep Blue, Blue, Green, Red, Red Edge, and Near-Infrared (NIR). This significantly enhances the model's sensitivity to differences in spectral reflectance between shaded and unshaded areas. Furthermore, existing research indicates that the Red Edge and Near-Infrared bands are extremely sensitive to the spectral response of vegetation; therefore, the multi-channel input of this invention can also significantly enhance the separability between shaded and dark vegetation. This expansion effectively improves the model's shadow recognition capability in complex urban greening environments.

[0092] (2) Parameter initialization strategy

[0093] Since commonly used AISD pre-trained weights only support three-channel input, when the input is expanded to a six-channel image containing bands with unique physical significance such as deep blue, red edges, and near-infrared, how to effectively and physically reasonable initialize the convolutional kernels of the three newly added channels becomes the primary challenge. If fully random initialization is used, the weights of the newly added channels are completely disconnected from the pre-trained RGB channel weights in terms of distribution. This not only fails to utilize the pre-trained knowledge but also destroys the model's ability to extract low-level visual features (such as edges and textures), leading to abnormal gradients, slow convergence, or even divergence in the early stages of training.

[0094] To address this, this invention proposes a "channel replication and mean filling" strategy for initializing the weights of newly added channels. On one hand, the weight distribution of the RGB channels is inherited through mean filling, implicitly assuming that the new spectra are statistically similar to RGB, thus improving the rationality of the initialization. On the other hand, driven by physical meaning, mean filling preserves the physical correlation between the spectral characteristics of the new channels (such as NIR and SWIR) and RGB, rather than random initialization. Independent random initialization of new channels (such as Xavier or He initialization) could disrupt the physical consistency of the input features. Simultaneously, this avoids training oscillations caused by the random distribution of new channel weights, providing stability guarantees, especially in small dataset scenarios.

[0095] In one specific implementation, initialization is performed according to the following formula:

[0096]

[0097] in, Here are the weights of the original RGB channels of the convolution kernel, and i is the index variable used for summation. i = 1, 2, 3 correspond to the three channel dimensions of the original 3-channel convolution kernel, respectively. The initial weights for the newly added channels are defined by c, where c ∈ {4, 5, 6} represents the newly added 4th, 5th, and 6th channels. The newly added bands are initialized by taking the average of the RGB channel weights, maintaining spectral physical consistency, effectively reducing instability in the early stages of transfer learning, and accelerating convergence.

[0098] (3) Adopt a two-stage transfer learning strategy

[0099] The AISD dataset (aerial RGB imagery) used for pre-training and the Pléiades imagery (satellite multispectral imagery) for the target application scenario belong to different "domains," exhibiting systematic differences in imaging mechanisms, spatial resolution, illumination conditions, and spectral responses of ground features. Directly fine-tuning the entire pre-trained model can easily lead to the model rapidly forgetting general features learned in the source domain (AISD) (such as shadow geometry and spatial relationships with buildings), while overfitting to noise or specific artifacts in the target domain (Pléiades). This phenomenon is known as "catastrophic forgetting," which severely impairs the model's generalization ability and robustness. Therefore, this invention designs a two-stage progressive transfer learning strategy:

[0100] The first stage freezes the lower layers of the backbone network, trains only the newly added input layer and classification head parameters, fixes the backbone network weights, avoids destroying existing feature representations, and allows the model to initially adapt to multispectral inputs.

[0101] The second stage involves gradually unfreezing the network and combining it with cosine annealing learning rate scheduling. Specifically, during model training, all pre-trained layers are frozen initially, and only newly added layers are trained. Then, throughout the training process, the parameters of the previously frozen pre-trained layers are gradually made trainable in batches, from low-level (shallow) to high-level (deep) networks, and the learning rate is gradually reduced for fine-tuning. That is, within each training phase (or cycle), the learning rate smoothly decreases from its initial value to near zero according to a cosine function curve. This helps the model converge more stably to a better local optimum after unfreezing. The core innovation of this strategy lies in independently configuring a complete cosine annealing learning rate cycle for each network unfreezing phase, thereby providing a fully exploratory and smoothly converging adaptive optimization environment for each batch of newly released parameters.

[0102] Therefore, the two-stage transfer learning strategy of this invention can gradually adapt to the target domain data distribution, achieving full network adaptation to the target domain. This strategy effectively avoids the common problems of "catastrophic forgetting" and "underfitting" in cross-domain transfer.

[0103] (4) Use weighted cross-entropy loss function

[0104] In high-resolution urban imagery, shadowed areas constitute a small proportion, creating a severe class imbalance compared to the large areas of non-shadowed background. Using a standard loss function, the model tends to predict these areas as background, leading to a higher false negative rate for shadowed areas (positive samples) and hindering refined shadow boundary detection. This embodiment addresses the class imbalance problem in shadow detection within complex urban scenes by providing a weighted cross-entropy loss function:

[0105]

[0106] in, , These are the weights for shadow and non-shadow classes, respectively. For real labels, To predict probabilities, The total number of samples. This weighting strategy effectively balances the sample distribution and improves the model's ability to identify the minority class (shaded).

[0107] To verify the effectiveness of the shadow region spectral restoration and shadow region spectral restoration module of this invention, and to compare the performance of different network models and training strategies, the following five sets of comparative experiments were designed:

[0108] Exp1: AISD → Pléiades: Directly transfer the AISD pre-trained weights to Pléiades using zero samples;

[0109] Exp2: Pléiades-3C-From Scratch: Training from Scratch with Three-Channel Images;

[0110] Exp3: AISD → Pléiades-3C: Three-channel transfer learning based on AISD weights;

[0111] Exp4: Pléiades-6C-From Scratch: Training from Scratch with Six-Channel Images;

[0112] Exp5: AISD → Pléiades-6C: Six-channel transfer learning based on AISD weights.

[0113] Model performance is comprehensively evaluated using metrics such as F1 score, overall accuracy (OA), Kappa coefficient, balanced error rate (BER), and IoU.

[0114] To prevent overfitting to a single metric, the following comprehensive performance evaluation metric (Composite Score) is proposed:

[0115]

[0116] The system automatically saves the model weights corresponding to the highest metric score during training to ensure optimal performance of the final model.

[0117] The quantification results of the five experiments on the Pléiades high-resolution image are shown in Table 1:

[0118]

[0119] The results show that the direct transfer AISD model (Exp1) performed the worst on the Pléiades test set, with an accuracy of only 38.16% and a BER as high as 44.31%, indicating that cross-domain differences significantly affect the model's generalization performance.

[0120] Compared with the three-channel model, the six-channel model (Exp4) improved the F1 score by 15.78% and reduced the BER by 7.23%, verifying the important role of the near-infrared and red-edge bands in shadow feature extraction.

[0121] The six-channel model (Exp5) using transfer learning performed best, with an F1 score of 83.28%, an OA score of 93.31%, and a BER score of 11.27%, outperforming all other groups.

[0122] Further visual effect verification was conducted. Experiments were performed using the original MSASDNet model, including direct transfer, RGB input + direct transfer, and RGB input + two-stage transfer, yielding results for Exp1-Exp3 as follows: Figure 2 As shown in Figure ac. Based on this, channel number expansion and a two-stage transfer learning mechanism are then sequentially superimposed to obtain the experimental results for Exp5-Exp6, as shown in the figure. Figure 2 As shown in the middle.

[0123] Exp1: Numerous false positives occurred in shadow detection, misidentifying dark vegetation as shadows;

[0124] Exp2: Although it can capture the main shadow, it has obvious missed detections in the edge and small shadow areas;

[0125] Exp3: The shadow coverage is good, but the false detection rate is high, such as roads or building facades being mistaken for shadows;

[0126] Exp4: The shadow area is continuous and the boundaries are smooth, but there are still a few false background detections;

[0127] Exp5: Almost completely covers real shadows, with clear and consistent edges, and very few false detections in non-shadow areas.

[0128] Figure 3 The overall shadow recognition results are shown, along with a magnified local area (the part within the red box).

[0129] It is evident that the MSASDNet-M model of this invention accurately covers the entire shadow region (i.e., Exp5) at the macroscopic level, while maintaining clear edges and consistent shapes at the local scale. This avoids the "fragmentation" and "diffusion" problems common in traditional methods, verifying the superior performance of the shadow recognition module of this invention in multispectral high-resolution images.

[0130] Example 3

[0131] The shadow region spectral restoration technology and corresponding shadow region spectral restoration module provided in this embodiment are systematically improved based on the shadow restoration algorithm proposed by Silva et al. (2018). The improvements mainly include three aspects: multispectral generalization of the radiometric correction model, smoothing of shadow boundaries, and computational optimization structure for large-scale images, thereby improving the spectral restoration quality and computational efficiency of shadow regions in multispectral remote sensing images.

[0132] (1) Multispectral generalization of the radiation correction model

[0133] To address the limitation of traditional algorithms being applicable only to RGB three-band images, this invention constructs a universal radiometric correction model that can flexibly support any number of bands.

[0134] The model calculates the ratio of the mean radiation in the shaded area to the unshaded area for each band to obtain the radiation gain coefficient for each band. :

[0135]

[0136] in, This represents the radiation correction gain coefficient for band b. This represents the mean radiation value of the unshaded area in band b. This represents the mean radiation value of the shadowed region in band b.

[0137] To avoid overexposure of radiation in deep shadow areas after correction, the model also incorporates a maximum gain constraint. This ensures that the correction gain for each band does not exceed this threshold. Compared to the linear correction mechanism of the original method, the new model improves the radiance level in the shadow area while better preserving the spectral characteristics of ground features, significantly enhancing the recovery quality of multispectral data.

[0138] (2) Smoothing of shadow boundaries

[0139] To address the problem that the original method is prone to radiation abrupt changes and jagged artifacts at the shadow boundary, this invention proposes a boundary smoothing mechanism based on continuous masking.

[0140] This mechanism uses Gaussian convolution to blur the binary shadow mask, converting the discrete binary mask value range of [0,1] into a continuous grayscale mask. The standard deviation of the Gaussian convolution is set to... This is used to generate a smooth transition area at the boundary between shadow and non-shadow areas. Through this continuous mask weighted blending, the brightness changes at the shadow boundary are more natural, effectively eliminating the hard boundary phenomenon that occurs in the original method.

[0141] Experiments have shown that this method improves the peak signal-to-noise ratio (PSNR) by more than 4.2 dB in the shadow edge region, which is more consistent with the spatial continuity characteristics of natural lighting.

[0142] (3) Optimization of computation for large-format images

[0143] To address the significant memory bottleneck issue in traditional algorithms when processing ultra-large-scale remote sensing images, this invention designs a block-based streaming computation structure based on a sliding window.

[0144] The system's default window size is 512×512 pixels. It divides the entire image into multiple independent processing units and performs radiometric correction calculations on a block-by-block basis.

[0145] After each block is processed, the system immediately releases the corresponding memory resources, achieving dynamic memory reuse.

[0146] Compared to the original method that requires loading the entire image at once, this architecture increases the maximum processable image size from about 5GB to the TB level, while reducing peak memory usage from 12GB to less than 2GB when processing 10GB images, a reduction of about 83%.

[0147] This mechanism significantly improves the engineering feasibility and operational efficiency of the algorithm, ensuring stable shadow restoration operations under multispectral high-resolution data.

[0148] (4) Performance verification and comparative analysis

[0149] To verify the effectiveness of the shadow recovery module, this invention designed a comparative experiment of two recovery strategies:

[0150] A. Traditional shadow removal method based on region group matching (2024);

[0151] B. The improved shadow restoration method proposed in this invention.

[0152] In a comparison of typical areas, the spectral values ​​of the shadowed areas in the original image are significantly lower, and the details of trees and the ground are almost completely lost, resulting in poor visual continuity.

[0153] Traditional region group matching methods can partially recover shadow brightness, but they have the following two prominent problems:

[0154] ① Uneven spectral recovery (e.g.) Figure 4 (As shown in the red circle), the vegetation in the shaded area is dark and lacks the typical vegetation response of the red edge and near-infrared band;

[0155] ② The transition of the shadow boundary is abrupt (e.g.) Figure 4As shown in the yellow circle, there is a significant color difference between the restored area and the non-shaded area, forming a "patchwork effect" that disrupts the consistency of the overall image.

[0156] In contrast, the improved method of the present invention exhibits higher spectral consistency and visual smoothness within the same area, fully restores ground feature details in shaded areas, and displays red edges and near-infrared reflectance characteristics in vegetated areas that conform to physical properties.

[0157] like Figure 5 As shown, the shadow areas of the overall image are restored to have uniform brightness and continuous texture, with natural transitions at the edges and no obvious artifacts or color differences.

[0158] The comprehensive experimental results show that the method of the present invention achieves better shadow brightness recovery and radiometric consistency under multispectral high-resolution images, providing a more stable spectral input basis for subsequent tree species identification.

[0159] (5) Overall effect

[0160] Quantitative and qualitative verifications have shown that the improved algorithm in this module achieves significant improvements in radiative enhancement in shadow areas, smooth boundary transitions, and processing efficiency. Compared with traditional methods, this invention effectively improves the spectral consistency and visual continuity of shadow areas and achieves accurate restoration of vegetation reflectance features in multi-band images, providing high-precision input for subsequent tree species identification.

[0161] Example 4

[0162] like Figure 6 As shown, after completing shadow recognition and shadow restoration, fine tree species identification is required for the multispectral image. This embodiment achieves this through a tree species identification module.

[0163] This module structurally improves upon the classic DeepLabV3+ network, proposing the NAM-DeepLabV3+ model. By introducing a two-layer asymmetric attention module (NAM), multi-channel input adaptation, and a hybrid loss function design, it comprehensively enhances the robustness and accuracy of tree species identification. The coupling of the NAM module with DeepLabV3+ essentially involves deeply embedding a dynamic, adaptive attention mechanism into a static, fixed-architecture segmentation network. It guides and calibrates the entire process of DeepLabV3+ feature extraction, context aggregation, and detail restoration in real time from two dimensions: "feature channel importance" (shallow NAM) and "inter-pixel semantic correlation" (deep NAM). This coupling enables the network to flexibly address the challenges posed by shadows, occlusion, scale variations, and spectral similarity between tree species in urban imagery, ultimately achieving highly consistent and accurate semantic segmentation results while maintaining high-resolution details.

[0164] (1) Multispectral input adaptation

[0165] To fully utilize the multispectral characteristics of Pléiades imagery, the model expands its input channels to six bands: Deep Blue, Blue, Green, Red, Red Edge, and Near Infrared (NIR). The newly added Red Edge and NIR bands provide the deep learning model with a "lens" that directly reflects the internal biochemical components (chlorophyll) and macroscopic physical structures (cellular tissue, canopy) of leaves. These not only significantly increase the dimensionality of spectral features but, more importantly, provide discriminative features directly related to the physiological essence of tree species. This allows the model to learn more profound and stable tree species differentiation patterns than the human eye (RGB) or traditional three-channel models, thereby achieving higher-precision classification among tree species under complex lighting conditions (especially shade) and with similar appearances.

[0166] The backbone network adopts the ResNet-101 structure. The weights of its first convolutional layer are initialized through weight remapping and channel-wise normalization to incorporate ImageNet pre-trained knowledge and avoid training instability caused by multispectral input. The weight remapping refers to "copying-averaging" the original RGB weights to initialize new bands, so that the new channels have physical response characteristics similar to visible light in the initial state, thus realizing knowledge transfer.

[0167] The encoder deeply embeds an improved multi-scale hollow spatial pyramid pooling (ASPP) module, combined with a two-layer attention mechanism: deep NAM is used to understand relevance, lightweight ASPP is used to perceive the relevance range, and the two are combined through dynamic fusion. Finally, shallow NAM is used to ensure the purity of details. This coupled structure cleverly shifts computational resources from blind multi-scale feature extraction to guided, precise feature focusing and fusion, thereby achieving a better balance of accuracy, robustness, and efficiency in complex urban tree species classification tasks.

[0168] The decoder employs a residual-enhanced shallow feature fusion structure to achieve high-fidelity restoration of texture and edge details, thereby improving tree species recognition performance in complex urban environments.

[0169] (2) Hierarchical NAM module design

[0170] This invention constructs a hierarchical attention framework, embedding NAM modules in both the shallow and deep layers of the encoder:

[0171] The shallow NAM module retains only the channel attention path and generates channel weights through global average pooling and two fully connected networks, highlighting spectral differences and edge texture features, thereby enhancing the model's ability to respond to fine vegetation targets.

[0172] The calculation formula is as follows:

[0173]

[0174]

[0175] Where H and W are the height and width of the feature map, respectively. Spatial position in the c-th channel eigenvalues ​​at that location This represents the global average pooling feature of the c-th channel. , Let be the weight matrix of the fully connected layer, and δ and σ be the ReLU and Sigmoid activation functions, respectively.

[0176] Compared to symmetric attention mechanisms such as SENet, the shallow NAM module only uses channel attention paths, avoiding spatial interference and more effectively highlighting vegetation edges and spectral differences.

[0177] The deep NAM module introduces a spatial attention mechanism based on the Query–Key–Value mechanism to model long-distance dependencies between pixels.

[0178]

[0179]

[0180] in, , , These are the query and key-value matrices generated from the input features, respectively. This represents the correlation weight between pixels i and j. This represents the final weighted output feature at position i, with indices i, j, and j. Each represents an index of all spatial locations (pixels) in the feature map.

[0181] This mechanism effectively captures the semantic associations of tree canopies under shadow and occlusion conditions, improving the model's discrimination ability in complex lighting environments.

[0182] The shallow and deep NAM modules work together to enhance local detail features and supplement global contextual information, thereby significantly improving the model's classification accuracy in shaded areas and rare tree species distributions.

[0183] (3) Lightweight ASPP and residual shallow layer fusion

[0184] To address the issue of redundant multi-branch convolutions in the original ASPP module, this invention retains the core branches: 1×1 convolution, 3×3 dilated convolution (with dilation rates of 6, 12, and 18), and image-level global pooling, thereby achieving multi-scale feature extraction while reducing the number of parameters.

[0185] During the decoding stage, shallow features are weighted by NAM and then introduced into the upsampling process through the residual path, which effectively improves the spatial consistency of the tree canopy boundary and small target areas.

[0186] (4) Design of hybrid loss function

[0187] To address the issues of uneven tree species distribution and scarce samples in a few categories in high-resolution images, this invention designs a hybrid loss function to simultaneously balance overall accuracy, boundary overlap rate, and the learning process of easy and difficult samples.

[0188] The total loss function is defined as:

[0189]

[0190] in, This represents the final mixed loss value, which is the objective function that needs to be minimized during model training. , , : Represents the balancing weight coefficient, a hyperparameter between [0, 1] used to adjust the contribution ratio of the three loss terms to the total loss. In this study, after performance tuning on the validation set, it is set to . = 0.5, =0.3, =0.2. This setting indicates that the training relies more on weighted cross-entropy in the initial stage to stabilize the training, and the effects of Dice and Focal losses are gradually increased to optimize the details. : Represents weighted cross-entropy loss. It is an improved form of cross-entropy loss, calculating a weight coefficient for each class (tree species) (usually related to the inverse of the class frequency). Its purpose is to solve the global class imbalance problem, preventing the model from over-biasing predictions to the majority class (such as common tree species) with more pixels, while assigning higher weights to the minority classes with fewer pixels, thereby improving the model's basic ability to identify rare tree species. : Represents Dice loss. Inspired by the Dice coefficient, it directly optimizes the overlap area between the predicted and ground truth regions. It is highly sensitive to the boundaries of small objects. In tree classification, it effectively improves the classification accuracy for tree crown edge pixels and small areas of rare tree species, preventing these areas from being ignored by the model. : Represents Focal loss. It is an improved form of cross-entropy loss, adjusted by a factor. This automatically reduces the contribution of easily classifiable samples (such as large areas of solid color background or common simple tree canopies) to the loss function, allowing the model to focus more on difficult-to-classify samples (such as tree canopies obscured by shadows, regions with blurred spectral features, or regions similar to surrounding tree species). γ represents the predicted probability of the current sample in its correct category; γ is the focusing parameter, which controls the model's attention to samples that are difficult to classify (such as tree canopies obscured by shadows, regions with blurred spectral features, or regions with features similar to those of surrounding tree species), thereby making the model more focused on such samples that are difficult to classify accurately.

[0191] The aforementioned hybrid loss function design is a "three-in-one" precision strategy: weighted cross-entropy ensures "fair learning," Dice loss ensures "shape accuracy," and Focal loss ensures "overcoming difficulties." All three are indispensable, working together to ensure that this invention achieves significantly better recognition results than traditional or simple combined loss functions when facing complex urban scenarios. This helps the model to discover difficult samples, further improving classification performance.

[0192] (5) Ablation experiment and performance verification

[0193] To verify the independent and synergistic effects of multi-channel input, NAM module and hybrid loss design, this invention conducted a series of ablation experiments based on DeepLabV3+.

[0194] The experimental sequence is as follows:

[0195] ① Six-channel input expansion;

[0196] ② Single NAM module;

[0197] ③ Dual NAM modules;

[0198] ④ Introduce a hybrid loss function.

[0199] All experiments maintained the same hyperparameters and data partitioning, and mIoU, mPA, mPrecision, and mRecall were used as evaluation metrics.

[0200] Table 2 presents the quantitative results of the ablation experiments:

[0201] The results show that:

[0202] The six-channel expansion improved mIoU by 3.28%, validating the complementary value of the red-edge and near-infrared bands. After adding a single NAM module, mIoU reached 77.24%, indicating that shallow attention effectively enhances edge and texture representation. The dual NAM module further improved semantic consistency, with mPA and mRecall both exceeding 91%. Finally, the improved loss function achieved optimal performance, with mIoU of 78.90%, mPA of 92.15%, and mRecall of 92.75%, validating the synergistic advantages of the complete architecture.

[0203] (6) Overall tree species identification performance and effect

[0204] To further evaluate the combined effect of shadow restoration and multispectral enhancement in tree species identification, this invention compares the classification results of two configurations:

[0205] ① Three-channel input, shadowless recovery (before improvement);

[0206] ② Six-channel input, including shadow recovery (improved).

[0207] Table 3 shows a comparison of the overall recognition accuracy of the two schemes:

[0208]

[0209] Experimental results show that after shadow restoration and multispectral channel enhancement, the model significantly improved in all four metrics: mIoU increased by 6.57%, mPA by 5.95%, mPrecision by 3.39%, and the Kappa coefficient increased by 0.02. This indicates that the present invention can effectively restore spectral consistency in severely shadowed areas and improve tree species identification capabilities.

[0210] like Figure 7 The figure shows a comparison of the impact of shadow restoration processing on tree species identification in high-resolution remote sensing images. In the figure, a represents the original high-resolution image without shadow restoration processing; b represents the tree species identification results on the three-channel image without shadow restoration processing; c represents the high-resolution image after shadow restoration processing; and d represents the tree species identification results on the six-channel image after shadow restoration processing. The comparison results in a typical urban area shown in the figure indicate that the unimproved three-channel model suffers from severe deficiencies in tree species identification in shadow areas, with discontinuous boundaries; while the improved six-channel model accurately identifies previously invisible trees after shadow restoration, with more coherent classification boundaries and an overall result closer to the true distribution.

[0211] like Figure 8As shown, the optimal model of this invention (6-channel + dual NAM attention mechanism + improved loss function) accurately depicts the spatial distribution pattern of various tree species in the tree species identification map generated within a certain urban area, reflecting the composition and structural characteristics of the urban forest. The classification results reveal the spatial pattern of urban tree species on a macro scale, and also demonstrate high recognition and recovery capabilities in local shaded areas.

[0212] (7) Overall effect

[0213] In summary, the NAM-DeepLabV3+ model of this invention achieves adaptive suppression of shadow interference and precise tree species identification in high-resolution urban remote sensing images through multi-channel input, a two-layer attention mechanism, and synergistic optimization of hybrid loss functions. Compared with traditional three-channel models, this invention significantly improves overall accuracy, edge coherence, and shadow robustness, enabling highly reliable urban tree species identification mapping.

[0214] Example 5

[0215] An experiment was conducted on a six-lane Pléiades image of a central urban area of ​​a certain city, covering an area of ​​approximately 20 km², with a total of 1183 sampling plots. The experimental results are as follows:

[0216] Shadow recognition accuracy: Precision = 98.18%, BER = 11.27%, F1 = 83.28%;

[0217] Shadow restoration effect: Shadows in the original image severely damage spectral information; the method proposed in this paper is significantly better than the comparison method in terms of spectral consistency and boundary smoothness.

[0218] Tree species identification accuracy: average mIoU = 78.9%, mPA = 92.15%, mPrecision = 84.54%;

[0219] Improvements: Compared to the model without shadow correction, mIoU is improved by 6.57% and mPA by 5.95%.

[0220] The generated classification map ( Figure 8 The results showed that after the shadows were restored, the tree species boundaries were clearer and local confusion was reduced, especially in the building shadows and forest edges, where the classification results were significantly improved.

[0221] Example 6

[0222] To implement the above-mentioned urban tree species identification method, the present invention also provides an electronic device and a computer-readable storage medium.

[0223] The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements some or all of the steps of the above-described urban tree species identification method.

[0224] The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements some or all of the steps of the above-described urban tree species identification method.

[0225] This invention can be extended to the following scenarios:

[0226] 1. Refined management of urban greening: Using tree species distribution maps to guide watering paths and pruning plans;

[0227] 2. Carbon storage estimation: Calculate urban carbon sinks by combining tree species volume models;

[0228] 3. Urban landscape planning: assisting in the design of road greening and park layout;

[0229] 4. Cross-platform application: The model structure can be reused on satellite imagery such as WorldView and GF-2, requiring only minor parameter adjustments.

[0230] In summary, this invention significantly improves the accuracy and reliability of tree species identification in shadow areas of high-resolution remote sensing images by constructing an integrated deep learning framework encompassing shadow recognition, shadow restoration, and tree species identification. Through the synergistic effect of multispectral input, dual NAM attention mechanisms, and transfer learning strategies, this invention achieves high robustness and cross-domain generalization capabilities in complex urban environments, providing a feasible solution for intelligent remote sensing monitoring of urban ecology, and possesses high scientific research value and potential for widespread application.

Claims

1. A method for identifying urban tree species, characterized in that, Includes the following steps: Shadow identification in urban areas is performed based on multi-channel satellite remote sensing image data, and shadow area information data is extracted; Based on the shadow region information, spectral restoration of the shadow region is performed to obtain the shadow-restored image; Tree species identification is performed on the image after shadow restoration.

2. The urban tree species identification method according to claim 1, characterized in that, The multi-channel satellite remote sensing image data selected includes data in the deep blue, blue, green, red, red-edge, and near-infrared bands.

3. The urban tree species identification method according to claim 1, characterized in that, The shadow recognition in the urban area uses a deep convolutional neural network to extract features from multi-channel satellite remote sensing image data and to locate shadow areas through network learning.

4. The urban tree species identification method according to claim 3, characterized in that, The deep convolutional neural network adopts the MSASDNet-M model, which expands the input structure from three-channel data to six-band data based on the original MSASDNet model. It uses a channel duplication and mean padding strategy to initialize the pre-trained weights of the three newly added channels. It uses a two-stage transfer learning strategy to transfer the model weights pre-trained on the AISD dataset to the target dataset.

5. The urban tree species identification method according to claim 1, characterized in that, The spectral restoration of the shadow region includes: calculating the radiation gain coefficient of each band based on the shadow region information and the maximum gain constraint; using the radiation gain coefficient to perform multi-band radiometric correction on the multi-channel satellite remote sensing image data; and then performing shadow boundary smoothing processing to obtain the shadow-restored image.

6. The urban tree species identification method according to claim 1, characterized in that, The tree species identification adopts the NAM-DeepLabV3+ network model, which expands the input channels to six bands based on the DeepLabV3+ classifier; an improved multi-scale hollow spatial pyramid pooling (ASPP) module is embedded in the deep layer of the encoder, combined with a two-layer asymmetric attention mechanism; the decoder adopts a residual-enhanced shallow feature fusion structure.

7. The urban tree species identification method according to claim 6, characterized in that, The loss function used to train the NAM-DeepLabV3+ network model is a hybrid of weighted cross-entropy, Dice loss, and Focal loss.

8. A city tree species identification system, characterized in that, The system includes: The shadow recognition module is used to identify shadows in urban areas based on multi-channel satellite remote sensing image data and extract shadow area information data. The shadow area spectral restoration module is used to perform spectral restoration of the shadow area based on the shadow area information data extracted by the shadow recognition module, so as to obtain the shadow-restored image; The tree species identification module is used to identify the tree species in the image after the shadow is restored.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the urban tree species identification method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the urban tree species identification method as described in any one of claims 1-7.

Images