Tree detection method and system based on pre-training large model and gradient side optimization strategy

Abstract

The application discloses a single tree detection method and system based on a pre-training large model and a gradient side optimization strategy, relates to the technical field of forestry remote sensing intelligent monitoring, and comprises the following steps: acquiring a remote sensing image, preprocessing and constructing a reference sample data set; taking the pre-training large model as a global feature encoder, constructing a side branch model as a local feature encoder based on the gradient side optimization strategy, using the encoder to extract and fuse global and local features of the preprocessed remote sensing image in stages, and obtaining multi-level features; presetting a segmentation threshold and a crown layer processing strategy for different scenes; constructing a double-flow feature decoder, processing the multi-level features according to the segmentation threshold and the crown layer processing strategy, further processing the multi-level features by using a full convolution network, and obtaining a tree crown layer segmentation result; and outputting single tree detection information in a target region. The application improves the detection adaptability to complex scenes, reduces the dependence on large-scale labeled data, reduces the cost of single tree detection and improves the efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of forestry remote sensing intelligent monitoring, more particularly to a single tree detection method and system based on pre-training large model and gradient side optimization strategy. BACKGROUND

[0002] Urban trees play an important role in carbon sequestration, climate change mitigation, and improving living environment. With the accelerating process of urbanization, there is an increasing demand for precise monitoring of urban forests. Traditional manual survey methods are costly and time-consuming, making it difficult to achieve high-precision single-tree-level monitoring at a large scale. The rise of high-resolution remote sensing images provides a new means for urban tree detection. However, due to the complex shape and uneven distribution of tree canopies in urban environments, as well as the presence of buildings, roads, and water bodies, automatic detection of single trees using traditional algorithms still faces challenges.

[0003] In recent years, the development of deep learning technology has made significant progress in feature extraction and target detection of remote sensing images. Traditional remote sensing processing methods rely on manually designed feature extraction strategies, which are often limited to fixed rules and assumptions and show poor adaptability in complex and variable environments. Existing research shows that deep learning models can automatically learn multi-level features from a large amount of training data, reducing the dependence on manually designed features. In urban tree monitoring, deep learning can efficiently handle the complex shape and uneven distribution of tree canopies through its adaptive feature learning mechanism, and outperforms traditional methods in automated detection and high-precision extraction. Deep learning, especially models based on deep convolutional networks, has shown its potential in forestry remote sensing applications by effectively separating targets from noise in complex backgrounds, thus achieving accurate tree detection.

[0004] At the same time, the emergence of visual large models provides a broader innovation space for remote sensing image analysis. These large models are pre-trained on super-large-scale datasets, and this pre-training process enables them to learn shared features from different domains and tasks, providing stable and reliable performance in various scenarios. They have strong versatility and generalization ability, enabling cross-domain migration and rapid adaptation under conditions of few or even zero samples, reducing dependence on manual annotation and specific scene data. Segment Anything Model (SAM) as a representative achievement of visual large models, not only suitable for natural images, but also effectively dealing with remote sensing field, providing innovative solutions for urban tree detection, ecological monitoring, and resource management in high-resolution remote sensing images.

[0005] However, the deep learning and large model single tree extraction method still has the following problems to be properly solved, including:

[0006] (1) The deep learning method usually relies on a large amount of labeled data sets, and has insufficient cross-scene generalization ability;

[0007] (2) When directly applying a large model to a specific single tree extraction task, the model output does not contain specific categories, and effective fine-tuning of the model structure is still needed to meet the application requirements of specific fields;

[0008] (3) The tree features in different scenes (such as cities, suburban forests, and mixed areas) are significantly different, and targeted tuning strategies and scene adaptation methods are needed. SUMMARY

[0009] The purpose of the present application is to overcome the defects in the prior art and provide a single tree detection method and system based on a pre-trained large model and a gradient side tuning strategy.

[0010] To achieve the above purpose, the technical scheme of the present application is as follows:

[0011] In a first aspect, the present application provides a single tree detection method based on a pre-trained large model and a gradient side tuning strategy, comprising the following steps:

[0012] Obtain a remote sensing image and preprocess it, and construct a reference sample data set according to the remote sensing image;

[0013] Use the global feature encoder and the local feature encoder to extract the global features and local features of the preprocessed remote sensing image in stages and perform stage-by-stage fusion to obtain multiple levels of features;

[0014] Different segmentation thresholds and crown processing strategies are preset for different scenes;

[0015] Construct a double-flow feature decoder, use the double-flow feature decoder to process the multiple levels of features according to the segmentation thresholds and crown processing strategies, and then use a full convolutional network for further processing to obtain a tree crown segmentation result;

[0016] Output single tree detection information within the target area according to the tree crown segmentation result.

[0017] Preferably, the pre-trained large model includes a network model of a mask autoencoder (MAE) trained Vision Transformer structure, and the parameters of the pre-trained large model remain unchanged during the single tree detection process;

[0018] Preferably, the local feature encoder comprises a multi-scale convolutional neural network structure, which comprises a convolutional layer of a relatively small size of convolution kernel and a convolutional layer of a relatively large size of convolution kernel.

[0019] Preferably, in the multi-scale convolutional neural network structure, the convolutional layer of the relatively small size of convolution kernel is used to extract local structure, edge and texture information of the tree, and the convolutional layer of the relatively large size of convolution kernel is used to extract background information including overall shape of the tree crown and relationship between adjacent trees or other objects.

[0020] Preferably, the fusion process is specifically weighting fusion of the feature map of the global feature and the feature map of the local feature through a learnable gating unit parameter, and the weighting fusion process is represented by the following expression:

[0021] X sam = SAM_encoder(x) e R H×W×C

[0022] X add = Additional_encoder(x) e R H×W×C

[0023] X` = a X sam + (1-a) X add

[0024] wherein X sam is the feature map of the global feature obtained through the SAM_encoder encoder, i.e. the global feature encoder, X add is the feature map of the local feature obtained through the Additional_encoder encoder, i.e. the local feature encoder, X` is the feature map obtained by fusion using the learnable gating unit parameter, HxWxC represents the dimension of the feature map, i.e. height, width and channel number, respectively, a is the learnable gating unit parameter, which is a variable coefficient, representing the weight in the weighting fusion, and the learnable gating unit parameter a is trained and optimized in the branch model training process according to the reference sample data set through back propagation, so that the global and local information in multiple levels of features achieve dynamic balance.

[0025] Preferably, in the process of extracting global features and local features stage by stage and performing fusion stage by stage, firstly, a blocking operation is performed to make the input tile keep consistent in size, feature extraction is performed in each stage in the process of extracting and fusing stage by stage, and down-sampling is performed in the convolution module of the local feature encoder, the extracted global features and local features are fused in each stage, and the fused features are input to the next stage for fusion, thereby obtaining multiple levels of features.

[0026] Preferably, the multiple levels of features are processed using the dual-stream feature decoder, and further processed using a fully convolutional network, specifically including:

[0027] The dual-stream feature decoder performs level-by-level upsampling and restoration on the multiple levels of features, and the dual-stream feature decoder further includes a feature pyramid network and a region proposal network, the feature pyramid network fuses the multiple levels of features after level-by-level upsampling and restoration to obtain fused features, and the region proposal network generates candidate region proposals according to the fused features to determine potential tree locations, and the fully convolutional network performs fine segmentation on the candidate regions to finally obtain tree crown segmentation results.

[0028] Preferably, the reference sample dataset is constructed according to the remote sensing image, including:

[0029] The remote sensing image data of the partial region is labeled, and the tree crown range is marked as the reference sample dataset;

[0030] The labeled image and the corresponding label are cropped into fixed-size tiles, and a part of them is used as a training set to train the local feature encoder and the gating parameter a, and another part is used as a validation set for performance evaluation and parameter tuning.

[0031] Preferably, different segmentation thresholds and crown processing strategies are used for different scenes, specifically including:

[0032] In an urban scene, the segmentation threshold is increased to reduce misjudgment caused by building shadows and water bodies, and multiple small crown fragments are fused;

[0033] In a suburban forest scene, the segmentation threshold is reduced to capture more potential tree crowns, and large crowns are moderately split into small crown regions to more accurately identify dense and overlapping trees;

[0034] In a mixed scene, a general segmentation threshold and a standard crown processing strategy are used to make the results remain relatively stable in a multi-element environment.

[0035] In a second aspect, the present application provides a single tree detection system based on a pre-trained large model and a gradient side tuning strategy, which is used for the single tree detection method based on the pre-trained large model and the gradient side tuning strategy described in the above technical solution, and the system includes:

[0036] A data acquisition and processing module is used to acquire and preprocess remote sensing images, and to construct a reference sample dataset according to the remote sensing images;

[0037] The multi-level feature generation module is configured to use a pre-trained large model as a global feature encoder, construct a side branch model based on a gradient side tuning strategy and use the side branch model as a local feature encoder, use the global feature encoder and the local feature encoder to extract global features and local features of the preprocessed remote sensing image in stages and fuse the global features and the local features in stages to obtain multi-level features.

[0038] The scene adaptation module is configured to preset different segmentation thresholds and crown layer processing strategies for different scenes.

[0039] The crown layer segmentation result generation module is configured to construct a double-flow feature decoder, use the double-flow feature decoder to process the multi-level features according to the segmentation thresholds and the crown layer processing strategies, and further process the multi-level features by using a full convolutional network to obtain a tree crown layer segmentation result.

[0040] The single tree detection information output module is configured to output single tree detection information in a target region according to the tree crown layer segmentation result.

[0041] Compared with the prior art, the method has the following advantages:

[0042] The method uses a pre-trained large model to extract global features, improves cross-scene adaptability, extracts local features based on a gradient side tuning strategy, and uses a learnable gating unit to realize dynamic fusion of the global features and the local features, thereby enhancing the expression of complex tree crown information, formulating specific adaptation strategies for different scenes, significantly improving the detection adaptability to complex scenes, reducing the dependence on large-scale labeled data, reducing the cost of single tree detection and improving the efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 FIG. 1 is a schematic diagram of a single tree detection method based on a pre-trained large model and a gradient side tuning strategy according to Embodiment 1 of the present application;

[0044] Figure 2 FIG. 4 is a schematic diagram of a global feature encoder according to Embodiment 1 of the present application;

[0045] Figure 3 FIG. 5 is a schematic diagram of a local feature encoder according to Embodiment 1 of the present application;

[0046] Figure 4 FIG. 6 is a schematic diagram of extracting global features and local features in stages and fusing the global features and the local features in stages according to Embodiment 1 of the present application;

[0047] Figure 5 FIG. 7 is a schematic diagram of single tree detection before and after according to Embodiment 1 of the present application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0049] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0050] Embodiment 1

[0051] Please refer to Figure 1 The embodiment 1 of the present application provides a single tree detection method based on pre-training large model and gradient side optimization strategy, including the following steps S1-S5:

[0052] S1: Obtain a remote sensing image and perform preprocessing, and construct a reference sample dataset according to the remote sensing image.

[0053] In the embodiment 1 of the present application, a high-resolution optical remote sensing image of a region to be detected is used as a data source, wherein the spatial resolution of the high-resolution remote sensing image is about 0.1m-0.15m, including three bands of red, green and blue. Necessary preprocessing is performed on the image to meet the input requirements of the subsequent model, and the preprocessing can include radiation correction, geometric correction, cropping and enhancement.

[0054] The remote sensing image data of part of the region is manually or semi-automatically labeled, and the tree canopy range is marked as a reference sample dataset; the labeled image and the corresponding label are cropped into fixed-size patches, and part of them are used as a training set to train the local feature encoder and the gating parameter a, and the other part is used as a validation set for performance evaluation and parameter optimization.

[0055] S2: Use a pre-training large model as a global feature encoder, construct a side branch model based on a gradient side optimization strategy and use it as a local feature encoder, use the global feature encoder and the local feature encoder to extract global features and local features of the preprocessed remote sensing image stage by stage and fuse them stage by stage to obtain multiple levels of features.

[0056] The pre-trained large model adopted by the present application is a network model of VIT (Vision Transformer) structure trained by Masked Autoencoders (MAE). In the present embodiment 1, the pre-trained large model is preferably a Segment Anything Model (SAM). The Segment Anything large model SAM is composed of three components: an image encoder, a prompt encoder and a mask decoder. The image encoder uses a pre-trained Masked Autoencoders Vision Transformer network to extract image features. The prompt encoder can process four types of inputs: points, boxes, text and masks, which are embedded into the model in different ways. The mask decoder is responsible for effectively combining the features of the image and the prompt, and interacting through the cross-attention mechanism, one as the query and the other as the key and value vectors. Finally, the features are upsampled by transposed convolution. The mask decoder can generate multiple results according to the provided prompt. Please refer to Figure 2 , Figure 2 is a schematic diagram of the global feature encoder of the present embodiment 1. The present application uses a SAM image encoder with unchanged parameters to extract global features of remote sensing images. In the present embodiment 1, a Ladder-Side Tuning (LST) strategy is used to introduce a convolutional neural network to construct a side branch model as a local feature encoder for tree crown related local feature extraction. This strategy uses a lightweight network architecture different from the main network as a downstream guide branch to extract local features required by downstream tasks. LST aims to enhance feature representation by combining different network paths that work together to refine and supplement the extracted features. Specifically, the main network (i.e. the SAM as the global feature encoder) focuses on obtaining global features, while the downstream guide branch (i.e. the side branch model as the local feature encoder) that extracts more detailed local features is supplemented. This multi-branch structure allows hierarchical representation, with different levels of features being integrated continuously, similar to a ladder-like configuration. The downstream guide branch helps to side-tune the features, thereby maintaining a balance between global and local feature extraction. This method can capture global features while extracting local features of image segmentation instances using a deep convolutional network within a local perception unit. In this part, the convolutional neural network serves as a guide model for downstream tasks, mainly outputting multi-scale local features to assist the global feature encoder in processing images. During the training process, the parameters of the pre-trained SAM model remain unchanged, while the side branch model based on LST is trained and optimized using the training set in the reference sample dataset through backpropagation to enhance feature extraction, so that the global and local information in multiple levels of features achieve dynamic balance.

[0057] SAM demonstrates high generalization ability in diverse scenarios and can effectively segment almost never-seen targets, but its performance in specific domains (e.g., accurately identifying individual trees in remote sensing images and other downstream tasks) still needs to be improved. To overcome this limitation, Embodiment 1 of the present application introduces a convolutional neural network to construct a side branch model as a local feature encoder to enhance the model's feature extraction and representation ability in specific domains, thereby improving the performance of the corresponding tasks. Please refer to Figure 3 , Figure 3 Figure 1 is a schematic diagram of the local feature encoder of Embodiment 1 of the present application. The local feature encoder is composed of a multi-scale convolutional neural network structure, which includes convolutional layers with relatively small size convolutional kernels and convolutional layers with relatively large size convolutional kernels. In Embodiment 1, the multi-scale convolutional neural network structure includes convolutional layers with 3*3 size convolutional kernels and convolutional layers with 7*7 size convolutional kernels, which have the following specific functions:

[0058] (1) The 3*3 convolutional layer with a smaller convolutional kernel focuses on the fine features of the image, such as the edges and textures of the trees, which are crucial for extracting the tree crown.

[0059] (2) The 7*7 convolutional layer with a larger convolutional kernel captures features within a wider field of view, such as the overall shape of the tree crown and the relationship between adjacent trees. Large size convolutional kernels can extract more background information, which is particularly important for distinguishing trees from complex backgrounds and helps identify the location of trees in the environment.

[0060] (3) The features extracted by the 3*3 convolutional layer and the 7*7 convolutional layer are integrated into spatial prior features, which are used for subsequent feature fusion and analysis processing

[0061] Please refer to Figure 4 , Figure 4Figure 1 is a schematic diagram of the embodiment 1 of the present application for extracting global features and local features in stages and fusing them in stages. The embodiment 1 of the present application divides the global feature encoder into four stages (stage), each corresponding to a local feature encoder, and fuses the global features and local features in depth at each different encoding stage. The embodiment 1 of the present application inputs the global features and local features into a learnable gating unit, optimizes the learnable gating unit parameter a through back propagation, and realizes dynamic weighted fusion of global and local features. The function of a is to dynamically adjust the contribution of each network feature in the composite feature representation. This adaptive weight parameter is optimized according to the contribution of the feature map to the overall task performance during the training stage. Using the gating unit allows a to vary between 0 and 1, where a = 1 indicates complete dependence on the features of SAM, and a = 0 indicates complete dependence on the features extracted by the convolutional neural network CNN of the local encoder. During the entire training process, a is refined through back propagation, so that the global and local information in multiple levels of features achieve dynamic balance to ensure that the integration of features from the two sources is optimally balanced, reflecting their relevance and effectiveness in affecting the model output. The process of weighted fusion is specifically that the feature map of the global features and the feature map of the local features are weighted and fused through the learnable gating unit parameter, and the weighted fusion process is represented by the following expression:

[0062] X sam = SAM_encoder(x) e R H×W×C

[0063] X add = Additional_encoder(x) e R H×W×C

[0064] X` = a · X sam + (1 - a) · X add

[0065] wherein X sam is the feature map of the global features obtained by the SAM_encoder encoder, i.e., the global feature encoder, X add is the feature map of the local features obtained by the Additional_encoder encoder, i.e., the local feature encoder, X` is the feature map obtained by fusion using the learnable gating unit parameter, H x W x C represents the dimensions of the feature map, i.e., height, width, and number of channels, a is the learnable gating unit parameter, which is a variable coefficient representing the weight in the weighted fusion, and the learnable gating unit parameter a is trained and optimized through back propagation according to the reference sample dataset during the training process of the side branch model, so that the global and local information in multiple levels of features achieve dynamic balance.

[0066] The above learnable gating unit and cross attention mechanism are used in the examples of the present application to facilitate feature interaction between global feature encoders and local feature encoders. In the process of extracting global features and local features stage by stage and fusing them stage by stage, the image is first subjected to patch partitioning so that the input patches remain consistent. Each encoding stage contains different numbers of Transformer Blocks for extracting global features and different convolutional modules (Convolutional Blocks) for extracting local features. Each stage performs feature extraction and down-sampling in the convolutional module to align the scale of the Transformer Block input. After the first encoding stage, the PatchMerging module between each stage fuses the extracted global features and local features and inputs the fused features to the next stage through the cross attention mechanism. The feature map of the input global features in each stage is the feature map combined from the global feature map and the local feature map of the previous stage. Global features and local features are extracted and fused stage by stage. Through this process, multiple levels of features from shallow to deep are obtained. The cross attention mechanism effectively integrates spatial prior information into the features, thereby enhancing the understanding of spatial relationships. This not only improves the accuracy of feature extraction, but also enhances adaptability and flexibility, making it suitable for processing complex and variable image data.

[0067] S3: Different segmentation thresholds and crown treatment strategies are preset for different scenes.

[0068] In the embodiment 1 of the present application, the detection process is dynamically adjusted by implementing a customized scene adaptation strategy to adapt the final output detection result to different urban scenes. These adjustments mainly include determining the segmentation threshold and adopting different crown coverage treatment strategies.

[0069] S4: A double-stream feature decoder is constructed, and the double-stream feature decoder is used to process the multiple levels of features according to the segmentation threshold and the crown treatment strategy, and a full convolutional network is further used for processing to obtain a tree crown segmentation result.

[0070] The double-flow feature decoder is constructed and the multiple levels of features obtained in step S2 are input into the double-flow feature decoder to perform instance-level segmentation and positioning on the single tree target, so as to generate a high-precision single tree detection result. The double-flow feature decoder processes feature maps of the multiple levels of features and then outputs a candidate box of instance segmentation, i.e., a tree canopy segmentation result. The double-flow feature decoder is composed of a feature pyramid network (FPN) and a region proposal network (RPN). The FPN is a network structure for target detection. It realizes the fusion and utilization of multi-scale features by constructing a feature pyramid, and can improve the detection accuracy. After the multiple levels of features obtained in step S2 are input into the double-flow feature decoder, the double-flow feature decoder performs step-by-step upsampling and restoration on the multiple levels of features, performs feature fusion on the multiple levels of features by using the feature pyramid network FPN, obtains fused features, and then performs feature screening and inversion on the basis of the fused features by using the double-flow feature image decoder. After that, the fused features after the feature screening and inversion are input into the region proposal network RPN to generate a single tree candidate region proposal, so as to determine a potential single tree position. After the double-flow feature decoder outputs the candidate region proposal, a fully convolutional network (FCN) is used to further refine the candidate region to perform accurate bounding box prediction, finely segment the candidate region, and finally obtain a tree canopy segmentation result.

[0071] S5: Output single tree detection information in the target region according to the tree canopy segmentation result.

[0072] Please refer to Figure 5 , Figure 5 Fig. 1 is a schematic diagram of single tree detection before and after the embodiment 1 of the present application. According to the tree canopy segmentation result generated in step S4, single tree detection information in the target region is output, i.e., single tree distribution and quantity information in the target region is output.

[0073] Embodiment 2

[0074] The embodiment 2 of the present application is based on the embodiment 1, and further describes that different segmentation thresholds and canopy processing strategies are preset for different scenes in the embodiment 1, as follows:

[0075] The scenes are divided into three different types: suburban forest, urban, and mixed areas. The implementation of the scene adaptation strategy in Embodiment 2 of the present application is based on the unique characteristics of urban and suburban forest scenes, aiming to optimize the detection process to adapt to different scene characteristics. The region proposal network (RPN) can effectively identify potential target regions in the image and assign a confidence score to these regions. The segmentation threshold, i.e., the preset confidence score, is considered valid if it is greater than or equal to this confidence score.

[0076] In urban areas, the segmentation threshold is set at a higher level to effectively distinguish trees from other man-made elements, as urban scenes have diverse and complex background elements such as buildings, roads, and other artificial structures, which may have similar colors and shapes to trees. In contrast, a lower threshold is more suitable for suburban forest areas. Due to the high tree density, smaller canopies and understory vegetation need to be included to ensure complete suburban forest coverage. Additionally, when canopy overlap occurs, different strategies will be employed depending on the scene. In suburban forest areas, a large canopy is replaced by multiple smaller canopies, which helps to more accurately represent the structure of individual trees and more accurately analyze the ecological relationships between trees. In urban areas, multiple small canopies are merged into a larger canopy to reduce redundancy and improve consistency, as urban trees are typically pruned and have more regular canopy shapes. In mixed areas, where elements are complex and diverse, a moderate universal segmentation threshold and standardized canopy processing scheme (i.e., using the model or network's default parameters without modification) are employed to avoid increasing complexity and uncertainty due to over or under correction, thereby reducing overall computational efficiency, adaptability, and accuracy. In environments where buildings, roads, and water bodies coexist, i.e., in cities, the segmentation threshold is increased to reduce false positives, and multiple smaller tree canopy fragments are merged to make the results more consistent with the characteristics and patterns of urban tree distribution. In the case of dense canopy overlap and fuzzy boundaries in forests, the segmentation threshold is reduced to capture more potential canopies, and the large canopy is broken down into multiple smaller canopies to refine the results, thereby more accurately depicting the patterns of individual trees in high-density vegetation environments. For mixed areas with multiple land cover types, a moderate segmentation threshold and standardized canopy processing scheme are used to balance the influence of different object characteristics and avoid over or under correction, thereby maintaining high detection accuracy under the influence of multiple factors. Based on statistical analysis and accuracy evaluation of detection results in different scenes, the regional threshold and canopy processing parameters are dynamically adjusted to achieve better cross-regional and cross-temporal single tree detection performance.

[0077] The other technical details and steps of Embodiment 2 are the same as those of Embodiment 1, which will not be repeated here.

[0078] Embodiment 3

[0079] The embodiment 3 of the application provides a single tree detection system based on a pre-training large model and a ladder side optimization strategy on the basis of the embodiment 1 and the embodiment 2, and is used for applying the single tree detection method based on the pre-training large model and the ladder side optimization strategy.

[0080] A data acquisition and processing module is configured to acquire a remote sensing image and perform preprocessing, and to construct a reference sample data set according to the remote sensing image.

[0081] A multi-level feature generation module is configured to use a pre-training large model as a global feature encoder, to construct a side branch model based on a ladder side optimization strategy and use the side branch model as a local feature encoder, to use the global feature encoder and the local feature encoder to extract global features and local features of the remote sensing image after preprocessing and to perform fusion stage by stage, and to obtain features at multiple levels.

[0082] A scene adaptation module is configured to preset different segmentation thresholds and crown layer processing strategies for different scenes.

[0083] A crown layer segmentation result generation module is configured to construct a double-flow feature decoder, to use the double-flow feature decoder to process the features at the multiple levels according to the segmentation thresholds and the crown layer processing strategies, and to further process the features at the multiple levels by using a full convolutional network to obtain a tree crown layer segmentation result.

[0084] A single tree detection information output module is configured to output single tree detection information in a target region according to the tree crown layer segmentation result.

[0085] Other technical details and steps of the embodiment 3 are the same as those of the embodiment 1 or the embodiment 2, and will not be described here again.

[0086] As described in the embodiments, the global feature encoder from the SAM is combined, the parameters of the global feature encoder are kept unchanged, and the local feature encoder based on the LST strategy and the convolutional neural network is added, so that the pre-training segmentation capability of the SAM is effectively utilized, and the parameters of the local feature encoder based on the LST strategy are allowed to be updated to adapt to the single tree recognition task. In order to determine the importance of the features extracted by the two networks, a learnable gating unit is introduced when the extracted feature maps are fused, to evaluate the importance of different feature maps, to construct a double-flow feature decoder, to segment single tree crowns according to a preset scene adaptation strategy, and to output single tree detection information. Compared with other classification methods, the method proposed in the scheme effectively improves the overall precision of single tree segmentation, and indicates that the combined method is an effective solution for single tree segmentation in different urban backgrounds.

Claims

1. A single tree detection method based on a pre-trained large model and gradient side optimization strategy, characterized in that, The method comprises the following steps: acquiring a remote sensing image and pre-processing, constructing a reference sample dataset according to the remote sensing image; using a pre-trained large model as a global feature encoder, constructing a side branch model based on a gradient side optimization strategy and using the side branch model as a local feature encoder, using the global feature encoder and the local feature encoder to extract global features and local features of the pre-processed remote sensing image in stages and performing fusion in stages to obtain multiple levels of features; presetting different segmentation thresholds and crown processing strategies for different scenes; constructing a double-flow feature decoder, using the double-flow feature decoder to process the multiple levels of features according to the segmentation thresholds and crown processing strategies, and further processing using a full convolutional network to obtain a tree crown segmentation result; outputting single tree detection information in a target region according to the tree crown segmentation result; the fusion process specifically comprises weighting and fusing feature maps of global features and feature maps of local features through learnable gating unit parameters, and the weighting and fusing process is represented by the following expression: wherein, is a feature map of global features obtained by a SAM_encoder, i.e., a global feature encoder, is a feature map of local features obtained by an Additional_encoder, i.e., a local feature encoder, is a feature map obtained by fusion using learnable gating unit parameters, denotes the dimensions of the feature map, respectively, height, width, and the number of channels, is a learnable gating unit parameter, which is a variable coefficient, representing the weight in the weighted fusion, and the learnable gating unit parameter In the side branch model training process, the global and local information in the multiple levels of features are dynamically balanced through back propagation training and optimization according to the reference sample data set. In the process of extracting global features and local features in stages and performing fusion in stages, firstly, block operation is performed to make the input blocks consistent in size, feature extraction is performed in each stage in the process of extracting and fusing in stages, and down-sampling is performed in the convolution module of the local feature encoder, the extracted global features and local features are fused in each stage, and the fused features are input into the next stage for fusion, thereby obtaining multiple levels of features; the process of using the double-flow feature decoder to process the multiple levels of features and further processing using a full convolutional network specifically comprises: the double-flow feature decoder performs level-by-level up-sampling and restoration on the multiple levels of features, the double-flow feature decoder further comprises a feature pyramid network and a region proposal network, the feature pyramid network fuses the multiple levels of features after level-by-level up-sampling and restoration to obtain fused features, the region proposal network generates candidate region proposals according to the fused features to determine potential single tree positions, and the full convolutional network performs fine segmentation on the candidate regions to finally obtain a tree crown segmentation result.

2. The single tree detection method based on pre-training large model and gradient side optimization strategy according to claim 1, characterized in that, The pre-trained large model comprises a network model of a Masked Autoencoder (MAE) trained Vision Transformer structure, and the parameters of the pre-trained large model remain unchanged in the single tree detection process.

3. The single tree detection method based on pre-training large model and gradient side optimization strategy according to claim 1, characterized in that, The local feature encoder comprises a multi-scale convolutional neural network structure, and the multi-scale convolutional neural network structure comprises a convolution layer with a relatively small size of convolution kernel and a convolution layer with a relatively large size of convolution kernel.

4. The single tree detection method based on pre-training large model and gradient side optimization strategy according to claim 3, characterized in that, In the multi-scale convolutional neural network structure, the convolution layer with a relatively small size of convolution kernel is used to extract local structure, edge and texture information of trees, and the convolution layer with a relatively large size of convolution kernel is used to extract background information including overall shape of a tree crown and relationship between adjacent trees or other objects.

5. The single tree detection method based on pre-training large model and gradient side optimization strategy according to claim 1, characterized in that, The reference sample dataset is constructed according to the remote sensing image, which comprises: Part of the remote sensing image data in the region is labeled, and the tree canopy range is marked as a reference sample data set; The labeled image and the corresponding label are cropped into fixed-size tiles, and part of them are used as a training set to train the local feature encoder and the gating parameter α, and the other part is used as a validation set for performance evaluation and parameter tuning.

6. The single tree detection method based on pre-training large model and gradient side optimization strategy according to any one of claims 1-5, characterized in that, The different segmentation thresholds and canopy processing strategies are adopted for different scenes, specifically including: In the urban scene, the segmentation threshold is increased to reduce misjudgment caused by building shadows and water bodies, and multiple small canopy fragments are fused; In the suburban forest scene, the segmentation threshold is reduced to capture more potential tree canopies, and large canopies are moderately split into small canopy regions to more accurately identify dense and overlapping trees; In the mixed scene, a general segmentation threshold and standard canopy processing strategy are used to make the results remain relatively stable in a multi-element environment.

7. A single tree detection system based on pre-trained large model and gradient side fine-tuning strategy, applied to the single tree detection method based on pre-trained large model and gradient side fine-tuning strategy of any one of claims 1-6, characterized in that, The system comprises: A data acquisition and processing module for acquiring and preprocessing remote sensing images, and constructing a reference sample data set based on the remote sensing images; A multi-level feature generation module for using a pre-trained large model as a global feature encoder, constructing a side branch model based on a gradient side tuning strategy as a local feature encoder, and using the global feature encoder and the local feature encoder to extract global features and local features of the preprocessed remote sensing images in stages and fuse them in stages to obtain multiple levels of features; A scene adaptation module for presetting different segmentation thresholds and canopy processing strategies for different scenes; A canopy segmentation result generation module for constructing a double-flow feature decoder, using the double-flow feature decoder to process the multiple levels of features according to the segmentation threshold and the canopy processing strategy, and further processing using a full convolutional network to obtain tree canopy segmentation results; A single tree detection information output module for outputting single tree detection information in a target region according to the tree canopy segmentation results.

Images