Urban green space segmentation method and device based on deep learning

Abstract

The invention discloses an urban green space segmentation method and device based on deep learning, and relates to the technical field of image classification. The method comprises the following steps: acquiring a four-waveband image, and determining vegetation features and texture features; based on the RGB image, a vegetation probability graph and a binary mask graph are obtained through a pre-trained coarse segmentation model; based on the four-waveband image, the vegetation probability map, the vegetation features and the texture features, outputting a three-classification segmentation map through a pre-trained fine classification model; and based on the binary mask pattern, carrying out fusion processing on the three-classification segmentation image to obtain an urban green space segmentation result corresponding to the four-band image. According to the method, the three-classification segmentation image is output through the coarse segmentation model and the fine classification model, and finally the binary mask image fusion processing is carried out, so that the calculation overhead is effectively reduced, the problems of boundary blur, local structure loss, low vegetation and other subclass missing detection caused by edge information weakening are solved, and the accuracy of edge information weakening is improved. And the overall precision and practicability of urban green space segmentation are obviously improved.

Description

Technical Field

[0001] This application relates to the field of image classification technology, and in particular to a method and apparatus for segmenting urban green spaces based on deep learning. Background Technology

[0002] Urban green space (UGS) is a core component of the urban ecosystem, referring to areas within a city primarily covered by vegetation. This includes both low-lying vegetation (such as grasslands, lawns, and low shrubs) and dense vegetation (such as woodlands and thickets). UGS plays an irreplaceable role in regulating urban climate, mitigating the urban heat island effect, sequestering carbon and releasing oxygen, conserving water resources, and improving the living environment. It also serves as a crucial source of foundational data for ecological planning, environmental monitoring, and public space management in smart city construction. Accurate urban green space segmentation provides pixel-level quantitative support for urban carbon sequestration assessment, green space system planning, ecological corridor design, and dynamic environmental monitoring, significantly contributing to enhancing urban ecological governance capabilities.

[0003] Current common methods for segmenting urban green spaces are based on deep learning. This method adopts a single-stage end-to-end network structure, such as an architecture that connects the main decoder and the auxiliary decoder in parallel through a single encoder. One main decoder is responsible for completing the standard pixel category prediction, while the other auxiliary decoder focuses on generating the edge prediction map. The outputs of the two decoders are fused in a so-called "fine segmentation head" to achieve the segmentation of urban green spaces.

[0004] However, in current technologies, common urban green space segmentation methods often involve large computational overhead for single-stage deep learning models. Furthermore, in complex urban contexts, edge information weakens, leading to blurred boundaries, loss of local structures, and missed detection of small categories (low vegetation), resulting in poor overall segmentation accuracy and practicality of green urban spaces. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a deep learning-based method and apparatus for urban green space segmentation. This method acquires four-band satellite remote sensing images and extracts vegetation and texture features. A pre-trained coarse segmentation model is used to initially separate vegetation from the background, generating a vegetation probability map and a binary mask image. Then, combining the four-band images, the vegetation probability map, and the vegetation and texture features, a pre-trained fine classification model outputs a three-class segmentation map. Finally, the binary mask image is fused, effectively reducing the computational overhead of single-stage deep learning models. This solves the problems of blurred boundaries, lost local structures, and missed detections of low-lying vegetation caused by weakened edge information in complex urban environments, significantly improving the overall accuracy and practicality of urban green space segmentation.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a deep learning-based method for urban green space segmentation, including:

[0008] Four-band satellite remote sensing images are acquired, and corresponding vegetation features and texture features are obtained based on the four-band images; wherein, the four-band images include: near-infrared (NIR) band, red (R) band, green (G) band, and blue (B) band images;

[0009] Based on the RGB images in the four-band imagery, a pre-trained coarse segmentation model is used to obtain the corresponding vegetation probability map and binary mask image. The vegetation probability map is used to indicate the probability that each pixel in the RGB image belongs to a vegetation category, and the binary mask image is used to distinguish the background area and vegetation area in the RGB image in a binarized manner.

[0010] Based on the four-band imagery, the vegetation probability map, the vegetation features, and the texture features, a pre-trained fine classification model outputs a corresponding three-class segmentation map; wherein, the three-class segmentation map is used to indicate whether each pixel in the four-band imagery is classified as background, low vegetation, or dense vegetation.

[0011] Based on the binary mask image, the three-class segmentation image is fused to obtain the urban green space segmentation result corresponding to the four-band image: wherein the urban green space segmentation result includes: the category label of each pixel is background, low vegetation or dense vegetation.

[0012] In one possible implementation, the step of obtaining the corresponding vegetation probability map and binary mask map based on the RGB images in the four-band imagery using a pre-trained coarse segmentation model includes:

[0013] The RGB images from the four-band imagery are input into a pre-trained coarse segmentation model, which encodes the RGB images to generate a first feature map corresponding to the RGB images. Based on the first feature map, the coarse segmentation model calculates the probability of each pixel in the RGB images belonging to a vegetation category using the softmax function, generating a corresponding vegetation probability map. Based on the corresponding vegetation probability map, the argmax function is used to determine the category of each pixel's probability of belonging to a vegetation category, generating a corresponding binary mask map, and outputting the vegetation probability map and the binary mask map.

[0014] In one possible implementation, the pre-trained coarse segmentation model is trained through transfer learning.

[0015] The pre-trained coarse segmentation model is obtained through transfer learning and includes:

[0016] Obtain a preset general large-scale dataset and an urban green space dataset; wherein, the urban green space dataset includes: multiple four-band images and urban green space segmentation results corresponding to the multiple four-band images respectively, and the urban green space segmentation results include the category label of each pixel as background, low vegetation or dense vegetation;

[0017] The background, low vegetation, and dense vegetation labels in the urban green space dataset are binarized into vegetation labels and background labels to obtain a coarse segmentation dataset. The coarse segmentation dataset includes: RGB images in each of the four-band images, and binary classification results corresponding to multiple RGB images respectively. The binary classification results include the category label of each pixel as vegetation or background.

[0018] The coarse segmentation model is trained by transfer learning using the preset general large-scale dataset to obtain a coarse segmentation model after transfer learning. The coarse segmentation model after transfer learning is then fine-tuned using the coarse segmentation dataset to obtain a pre-trained coarse segmentation model.

[0019] In one possible implementation, the step of outputting a corresponding three-class segmentation map based on the four-band imagery, the vegetation probability map, the vegetation features, and the texture features using a pre-trained fine-classification model includes:

[0020] The four-band image, the vegetation probability map, the vegetation features, and the texture features are input into a pre-trained fine classification model, so that the fine classification model can integrate the features of the four-band image, the vegetation probability map, the vegetation features, and the texture features to obtain initial feature data. The initial feature data is then processed by an encoder to perform multi-scale encoding to generate a multi-scale feature map. The multi-scale feature map includes multiple scale feature maps of different scales.

[0021] The fine classification model uses a decoder to perform upsampling operations layer by layer on the smallest scale feature map to restore the resolution and obtain the corresponding second feature map. Each upsampling operation integrates an edge attention upsampling module.

[0022] Specifically, for each upsampling operation, the upsampling operation includes: the fine classification model upsampling the input feature map to obtain an upsampled feature map, and then performing edge attention processing on the upsampled feature map to generate an edge attention map that matches the scale of the upsampled feature map; the upsampled feature map and the corresponding edge attention map are fused element-wise using residual connections to obtain a fused feature map, and then the fused feature map and a scale feature map that matches the scale of the fused feature map are fused using skip connections to obtain the feature map output by the upsampling operation, thus completing one upsampling operation; wherein, the second feature map is the feature map output by the last upsampling operation;

[0023] The fine classification model is based on the second feature map. It uses the softmax function to calculate the probability that each pixel in the four-band image belongs to the three categories of background, low vegetation, and dense vegetation, and generates a corresponding three-class probability map. Based on the corresponding three-class probability map, the argmax function is used to determine the category of each pixel, and a three-class segmentation map is generated and output.

[0024] In one possible implementation, the step of performing edge attention processing on the upsampled feature map to generate an edge attention map that matches the scale of the upsampled feature map includes:

[0025] By using a fixed edge detection operator, the gradient magnitude of the upsampled feature map is extracted to obtain the corresponding gradient magnitude map;

[0026] A lightweight convolutional neural network consisting of two convolutional layers and a batch normalization (BatchNorm) and ReLU activation function between the two convolutional layers is used to optimize the corresponding gradient magnitude map, and an edge attention map matching the scale of the upsampled feature map is generated by the sigmoid function.

[0027] In one possible implementation, the pre-trained fine-classification model is trained as follows:

[0028] Obtain an urban green space dataset; wherein, the urban green space dataset includes: multiple four-band images and urban green space segmentation results corresponding to the multiple four-band images respectively, and the urban green space segmentation results include the category label of each pixel as background, low vegetation or dense vegetation;

[0029] Based on the urban green space dataset, vegetation features and texture features corresponding to the multiple four-band images are obtained respectively. Based on the RGB images in the multiple four-band images in the urban green space dataset, vegetation probability maps corresponding to the multiple four-band images are obtained through a pre-trained coarse segmentation model.

[0030] Based on the multiple four-band images, the vegetation features and texture features corresponding to the multiple four-band images, the vegetation probability maps corresponding to the multiple four-band images, and the urban green space segmentation results corresponding to the multiple four-band images, a fine classification training set is obtained.

[0031] Based on the fine-classification training set, the fine-classification model is trained using a weighted multi-task loss function to obtain a pre-trained fine-classification model. The weighted multi-task loss function is obtained by weighted summation of a weighted cross-entropy loss function and a vegetation class Dice loss function. In the weighted cross-entropy loss function, the class weights are inversely proportional to the class frequencies. In the vegetation class Dice loss function, the weight of the low-growing vegetation class is set to 3, and the weight of the dense vegetation class is set to 0.5.

[0032] In one possible implementation, the step of fusing the three-class segmentation map based on the binary mask image to obtain the urban green space segmentation result corresponding to the four-band image includes:

[0033] Based on the binary mask image, the categories of pixels in the three-class segmentation image corresponding to the vegetation areas in the binary mask image are retained as category labels for pixels in the four-band image corresponding to the vegetation areas in the binary mask image. The category labels of pixels in the four-band image corresponding to the background areas in the binary mask image are uniformly updated to background, thus obtaining the urban green space segmentation result corresponding to the four-band image.

[0034] In one possible implementation, the vegetation features include: Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index (EVI), or Soil Adjusted Vegetation Index (SAVI).

[0035] The texture features include: Local Binary Pattern (LBP), Gray-Level Co-occurrence Matrix (GLCM), or Histogram of Oriented Gradients (HOG).

[0036] In one possible implementation, after acquiring the four-band satellite remote sensing imagery, the method further includes:

[0037] Radiometric correction, atmospheric correction, orthorectification, panchromatic and multispectral fusion, and image registration are performed on the four-band images to obtain preprocessed four-band images.

[0038] Secondly, embodiments of this application provide a deep learning-based urban green space segmentation device, comprising:

[0039] The acquisition module is used to acquire four-band images from satellite remote sensing, and obtain corresponding vegetation features and texture features based on the four-band images; wherein, the four-band images include: near-infrared (NIR) band, red (R) band, green (G) band, and blue (B) band images;

[0040] The coarse segmentation module is used to obtain the corresponding vegetation probability map and binary mask map based on the RGB image in the four-band image using a pre-trained coarse segmentation model; wherein, the vegetation probability map is used to indicate the probability that each pixel in the RGB image belongs to a vegetation category, and the binary mask map is used to distinguish the background area and vegetation area in the RGB image in a binarized manner.

[0041] The fine classification module is used to output a corresponding three-class segmentation map based on the four-band image, the vegetation probability map, the vegetation features and texture features, through a pre-trained fine classification model; wherein, the three-class segmentation map is used to indicate the category of each pixel in the four-band image as background, low vegetation or dense vegetation.

[0042] The fusion module is used to perform fusion processing on the three-class segmentation map based on the binary mask map to obtain the urban green space segmentation result corresponding to the four-band image: wherein the urban green space segmentation result includes: the category label of each pixel is background, low vegetation or dense vegetation.

[0043] Compared with existing technologies, this application has the following advantages: by acquiring four-band satellite remote sensing images and extracting vegetation and texture features, a pre-trained coarse segmentation model is used to achieve preliminary separation of vegetation and background and obtain vegetation probability maps and binary mask maps. Then, by combining the four-band images, vegetation probability maps, and vegetation and texture features, a pre-trained fine classification model is used to output a three-class segmentation map. Finally, the binary mask map is fused, which effectively reduces the computational cost of the single-stage deep learning model and solves the problems of boundary blurring, local structure loss, and missed detection of small classes such as low vegetation caused by weakened edge information in complex urban backgrounds. This significantly improves the overall accuracy and practicality of urban green space segmentation. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart illustrating a deep learning-based urban green space segmentation method provided in this application embodiment;

[0046] Figure 2 A structural example diagram of a coarse segmentation model provided in an embodiment of this application;

[0047] Figure 3 A flowchart illustrating the training method of the coarse segmentation model provided in the embodiments of this application;

[0048] Figure 4 A flowchart illustrating the process of outputting a three-class segmentation map for the fine classification model provided in the embodiments of this application;

[0049] Figure 5 This is a schematic diagram of the edge attention upsampling module provided in an embodiment of this application;

[0050] Figure 6 This is a structural example diagram of the fine classification model provided in the embodiments of this application;

[0051] Figure 7 A flowchart illustrating the training method of the fine classification model provided in the embodiments of this application;

[0052] Figure 8 Example diagram of urban green space segmentation results provided in the embodiments of this application;

[0053] Figure 9 This is a schematic diagram of a deep learning-based urban green space segmentation device provided in an embodiment of this application. Detailed Implementation

[0054] As described earlier, a common urban green space segmentation method in current technology is based on traditional machine learning. This method first calculates various vegetation indices and texture features from remote sensing images, and then uses these features to construct a classification rule model through a decision tree algorithm, thereby extracting and classifying urban green space information. However, traditional machine learning relies on manually designed features, which have limited feature representation capabilities. It cannot adaptively learn the complex, deep differences between low-lying and dense vegetation, thus failing to distinguish between them. Furthermore, it exhibits poor adaptability to different regions and data sources, resulting in insufficient model generalization ability.

[0055] To address the shortcomings of traditional machine learning-based urban green space segmentation methods, current technologies employ deep learning-based approaches. This involves a single encoder that connects a main decoder and an auxiliary decoder in parallel. One main decoder performs standard pixel category prediction, while the auxiliary decoder focuses on generating edge prediction maps. The outputs of both decoders are fused in a so-called "fine-grained segmentation head" to achieve urban green space segmentation. However, this method suffers from high computational overhead and training difficulty, resulting in poor practicality and real-time performance.

[0056] At the boundary between complex urban backgrounds and vegetation types (dense and low vegetation), single-stage models lose edge information during feature upsampling and recovery, resulting in unclear boundaries or local fragmentation. In other words, in complex urban backgrounds, edge information weakening can lead to blurred boundaries and loss of local structures, thus affecting the segmentation accuracy of urban green spaces.

[0057] In urban scenarios, the proportion of low-lying vegetation pixels is significantly smaller than that of the background and dense vegetation. Conventional loss functions and training strategies are insufficient to guarantee the recall rate and stable learning of this small class of low-lying vegetation, which can easily lead to missed detections of this small class and thus affect the segmentation accuracy of urban green spaces.

[0058] Furthermore, current technologies typically rely on spectral channels, which cannot distinguish between low-lying vegetation and dense vegetation that have similar spectra but different structures. This limits the ability to differentiate between vegetation types with similar spectra but different structures, thus affecting the accuracy of urban green space segmentation.

[0059] This application provides a deep learning-based method for urban green space segmentation, comprising: acquiring four-band satellite remote sensing images and obtaining corresponding vegetation features and texture features based on the four-band images; obtaining corresponding vegetation probability maps and binary mask maps based on the RGB images in the four-band images through a pre-trained coarse segmentation model; outputting corresponding three-class segmentation maps based on the four-band images, vegetation probability maps, vegetation features, and texture features through a pre-trained fine classification model; and fusing the three-class segmentation maps based on the binary mask maps to obtain the urban green space segmentation result corresponding to the four-band images. This application embodiment acquires four-band satellite remote sensing images and extracts vegetation and texture features. A pre-trained coarse segmentation model is used to initially separate vegetation from the background and obtain a vegetation probability map and a binary mask map. Then, the four-band images, vegetation probability map, and vegetation and texture features are combined with a pre-trained fine classification model to output a three-class segmentation map. Finally, the binary mask map is fused, which effectively reduces the computational cost of the single-stage deep learning model and solves the problems of boundary blurring, local structure loss, and missed detection of small classes such as low vegetation caused by weakened edge information in complex urban backgrounds. This significantly improves the overall accuracy and practicality of urban green space segmentation.

[0060] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0061] Example 1:

[0062] The following is combined with Figures 1-8 This application provides a detailed description of a deep learning-based urban green space segmentation method.

[0063] like Figure 1 As shown in the figure, the urban green space segmentation method based on deep learning provided in this application includes the following steps:

[0064] S101. Acquire four-band satellite remote sensing images, and based on the four-band images, obtain the corresponding vegetation features and texture features.

[0065] The four-band imagery includes: near-infrared (NIR) band, red (R) band, green (G) band, and blue (B) band. In other words, four-band imagery refers to high-resolution satellite remote sensing images encompassing four specific spectral bands: near-infrared, red, green, and blue. For example, four-band imagery can be acquired using the Gaofen-1 (GF-1) satellite, a high-resolution, multispectral remote sensing satellite for Earth observation.

[0066] Among them, vegetation characteristics are used to characterize the coverage and growth status of vegetation, providing quantitative basis for distinguishing between background and vegetation, and between low vegetation and dense vegetation.

[0067] In one possible implementation, vegetation characteristics include: normalized vegetation index, enhanced vegetation index, or soil-adjusted vegetation index.

[0068] The Normalized Difference Vegetation Index (NDVI) is one of the most commonly used vegetation indices in remote sensing, used to measure the growth status and cover of surface vegetation. Based on the spectral characteristics of vegetation, NDVI quantifies vegetation growth by calculating the difference in reflectance between near-infrared and red light bands, as plant leaves strongly absorb visible light (especially the red band).

[0069] To facilitate understanding, the normalized vegetation index will be introduced below with an example based on formula (1).

[0070] NDVI=(NIR-R) / (NIR+R) (1)

[0071] Wherein, NDVI is the Normalized Difference Vegetation Index, NIR is the Near Infrared Reflectance, and R is the Red Reflectance. The Near Infrared Reflectance is the ground reflectance data recorded in the Near Infrared Channel of the four-band image, and the Red Reflectance is the ground reflectance data recorded in the Red Channel of the four-band image. Both are key spectral information carried by the four-band image.

[0072] Specifically, the NDVI value ranges from [-1, +1]. The closer the NDVI is to +1, the higher the vegetation coverage and the more vigorous the growth; the closer the NDVI is to -1, the more likely it is a background area or a vegetation coefficient area. NDVI helps to distinguish between vegetation and background areas, and also reflects the density of vegetation, providing a basis for further distinguishing between low-lying vegetation and dense vegetation.

[0073] The Enhanced Vegetation Index (EVI) is an improved version of NDVI, designed to reduce the influence of atmospheric and soil background and improve the accuracy of vegetation cover information extraction. By incorporating blue band data, the EVI corrects for the nonlinear saturation effect of atmospheric imagery and vegetation reflectance, making it more sensitive to high-density vegetation areas and reducing the saturation effect.

[0074] The Soil Adjusted Vegetation Index (SAVI) is an improved index designed to address the impact of soil background on vegetation indices. It reduces the interference of soil brightness variations on vegetation monitoring by introducing a soil adjustment factor. SAVI builds upon the Normalized Difference Vegetation Index (NDI) by incorporating a soil adjustment coefficient, which is used to mitigate the influence of soil background on vegetation information extraction.

[0075] Among them, texture features are used to characterize the spatial structural heterogeneity of vegetation, providing a basis for distinguishing between low-lying vegetation and dense vegetation.

[0076] In one possible implementation, texture features include: local binary patterns, gray-level co-occurrence matrices, or histograms of oriented gradients.

[0077] Local Binary Patterns (LBP) is an operator used to describe local texture features of an image. LBP generates binary codes by comparing the gray levels of pixels with their neighbors, thus quantifying local texture features. In other words, LBP can reflect the intensity of texture changes in local areas and is suitable for distinguishing between regular and irregular vegetation structures.

[0078] Specifically, near-infrared band images are extracted from the four-band images, and based on the near-infrared band images in the four-band images, the corresponding local binary patterns (i.e. texture features) are obtained.

[0079] To facilitate understanding, the following example of the local binary mode is used in conjunction with formula (2).

[0080] (2)

[0081] Among them, LBP P，RThis is a local binary mode; P is the number of neighboring pixels, i.e., the number of sampling points around the center pixel, typically 8; R is the neighborhood radius, i.e., the spatial distance from the center pixel to neighboring pixels, typically 1; gp is the gray value of the p-th pixel in the neighborhood, representing the gray information of each sampling point in the neighborhood; gc is the gray value of the center pixel, which is the reference benchmark for the local region, used to compare the gray changes of neighboring pixels; s(x) is the sign function, used to binarize the gray difference between the neighboring pixels and the center pixel. When gp-gc≥0, s(x)=1, otherwise s(x)=0; 2 p The binary weights are used to encode the binary result of each neighboring pixel into a decimal number. Different p corresponds to different bit weights. The final summation result is the LBP code of the center pixel, which uniquely identifies a local texture mode.

[0082] The gray-level co-occurrence matrix (GLCM) describes texture features by statistically analyzing the gray-level binarization of pixel pairs with specific spatial relationships in an image. The GLCM is obtained by statistically analyzing the gray levels of two pixels maintaining a certain distance in an image.

[0083] The Histogram of Oriented Gradient (HOG) characterizes vegetation edges and texture details by calculating the gradient direction distribution of local regions.

[0084] In one possible implementation, after acquiring four-band satellite remote sensing images, radiometric correction, atmospheric correction, orthorectification, panchromatic and multispectral fusion, and image registration are performed on the four-band images to obtain preprocessed four-band images. Correspondingly, based on the preprocessed four-band images, the corresponding vegetation features and texture features are obtained.

[0085] Specifically, radiometric correction is used to eliminate radiometric errors in four-band images caused by factors such as sensor response and solar altitude angle, and to unify the radiometric scale of the images; atmospheric correction is used to remove interference from atmospheric scattering and absorption on the spectral information of four-band images and restore the true reflectivity of ground objects; orthorectification is used to combine satellite trajectory parameters and ground control point data to correct the geometric distortions in four-band images caused by terrain undulations and sensor tilt, so that affected pixels accurately correspond to the actual positions on the ground; panchromatic and multispectral fusion is used to fuse four-band images (multispectral data) with corresponding panchromatic images to improve the spatial resolution of four-band images to a preset accuracy; image registration is used to register the four bands (near-infrared (NIR), red (R), green (G), and blue (B)) within the four-band images after the above processing, to ensure that the four bands are spatially aligned at the pixel level and to avoid feature deviations caused by inter-band misalignment.

[0086] In this embodiment, by performing radiometric correction, atmospheric correction, orthorectification, panchromatic and multispectral fusion, and image registration on the four-band images, the quality of the four-band images is optimized in terms of spectral authenticity, geometric accuracy, and spatial detail, thereby helping to improve the segmentation accuracy of urban green spaces.

[0087] S102. Based on the RGB images in the four-band imagery, the corresponding vegetation probability map and binary mask map are obtained through a pre-trained coarse segmentation model.

[0088] Among them, the RGB image in the four-band image is a combination of red, green and blue bands, which is the most visually intuitive band combination. It can effectively distinguish the green visual features of vegetation from the non-green features of the background (buildings, roads, water bodies, etc.), providing sufficient visual discrimination for the coarse segmentation model while reducing computational complexity.

[0089] The vegetation probability map indicates the probability that each pixel in the RGB image belongs to a vegetation category. Specifically, it's a pixel-level vegetation category confidence distribution map output by a pre-trained coarse segmentation model after processing the RGB image in the four-band imagery. Each pixel in the vegetation probability map has a value ranging from 0 to 1, with the value indicating the probability that the pixel belongs to a vegetation category: the closer the value is to 1, the higher the confidence that the pixel is vegetation; the closer the value is to 0, the higher the confidence that the pixel is background.

[0090] In the embodiments of this application, the vegetation probability map is used as one of the inputs to the pre-trained fine classification model, which helps the fine classification model to more accurately distinguish between low vegetation and dense vegetation. For example, vegetation areas with high confidence are more likely to be dense vegetation, while low confidence areas may be low vegetation or background edges.

[0091] The binary mask is used to distinguish between background and vegetation areas in an RGB image using binarization. Specifically, the binary mask is a pixel-level binary image obtained by binarizing a vegetation probability map. Each pixel in the binary mask has a value of either 0 or 1; a value of 1 indicates that the pixel belongs to a vegetation area, and a value of 0 indicates that the pixel belongs to the background area.

[0092] In this embodiment, the binary mask image serves as a "filter" in the final fusion step, assisting in correcting the category of the background area in the binary mask image to be background, ensuring that the background area is not misclassified as vegetation, thereby improving the reliability of the final urban green space segmentation.

[0093] In one possible implementation, the RGB effects in the four-band image are input into a pre-trained coarse segmentation model, so that the coarse segmentation model can encode the RGB image and generate a first feature map corresponding to the RGB image. Based on the first feature map, the coarse segmentation model calculates the probability of each pixel in the RGB image belonging to a vegetation category using the softmax function, generates a corresponding vegetation probability map, and based on the corresponding vegetation probability map, determines the category of each pixel belonging to a vegetation category using the argmax function, generates a corresponding binary mask map, and outputs the vegetation probability map and the binary mask map.

[0094] The first feature map integrates multi-scale semantic information of vegetation and background in the RGB image (such as the green texture of vegetation and the building outlines in the background), providing a basis for subsequent probability calculations.

[0095] The softmax function is a commonly used activation function in multi-class classification tasks. In this embodiment, the softmax function is used to convert the output of each pixel in the first feature map into a probability value belonging to the vegetation category.

[0096] The argmax function is used to select the index of the maximum value in a set of values. In this embodiment, if the probability of a pixel belonging to the vegetation category is greater than or equal to a preset threshold, the pixel is determined to be a vegetation area and assigned a value of 1; otherwise, it is a background area and assigned a value of 0, thus generating a binary mask image.

[0097] In one possible implementation, the coarse segmentation model can be a DeepLabV3+ structure, a Fully Convolutional Networks structure, a U-Net structure, a SegNet structure, or a Transformer-like structure.

[0098] To make it easier to understand, the following will be combined with... Figure 2 This paper introduces a coarse segmentation model based on the DeepLabV3+ architecture. The Spatial Pyramid Pooling (ASPP) module of the DeepLabV3+ architecture, through multi-dilution rate convolutions (such as dilution rates of 6, 12, and 16), can simultaneously capture global contextual information of large-area vegetation patches in cities (such as extensive arbor forests) and local details of vegetation edges (such as the boundary between low vegetation and buildings). This achieves effective fusion of multi-scale features, helps maintain the integrity of target boundaries, and therefore can effectively handle large-scale contextual information, making it suitable for large-scale target recognition and classification.

[0099] like Figure 2As shown, the input RGB image from the four-band imagery is first processed by a 7×7 convolution operation to extract preliminary features. Then, max pooling is used to reduce the feature map resolution while retaining key information, preparing for subsequent encoding stages. The pooled feature map enters hidden layers 1, 2, and 3. These layers are network layers based on transfer learning using a general large-scale dataset (such as COCO). Leveraging the generalization ability obtained from training a coarse segmentation model on the general large-scale dataset, they quickly acquire basic semantic features of vegetation and background. The feature map output from the hidden layers enters hidden layer 4, where a 1×1 convolution is used to adjust the number of channels. This is followed by 3×3 dilated convolutions (with dilation rates of 6, 12, and 16 respectively) and pooling to achieve multi-scale feature extraction (capturing vegetation and background features from different receptive fields), resulting in the first feature map. The first feature map output from the encoder enters the decoder, where 1×1 and 3×3 convolutions are used to gradually restore the feature map resolution, ultimately outputting a vegetation probability map and a binary mask map.

[0100] In one possible implementation, the coarse segmentation model is trained through transfer learning.

[0101] Transfer learning is a machine learning method that transfers knowledge or models learned from one task to another related task to improve the learning efficiency, performance, or generalization ability of the target task. Specifically, when the target task has limited data, high training costs, or is related to the source task, reusing general features or model structures already learned in the source task helps the target task converge quickly, improves accuracy, and avoids the waste of resources from training the model from scratch.

[0102] To make it easier to understand, the following will be combined with... Figure 3 This section details the training method for the coarse segmentation model.

[0103] like Figure 3 As shown, the coarse segmentation model is trained through the following steps:

[0104] S301. Obtain the preset general large-scale dataset and urban green space dataset.

[0105] In this context, general-purpose large-scale datasets refer to publicly available datasets that cover a wide range of visual scenes and contain a large amount of data, such as COCO (Common Objects in Context) and ImageNet. These datasets contain millions of images, covering thousands of categories such as people, animals, buildings, and vegetation, and can be used to train coarse segmentation models to learn general visual knowledge such as "object recognition, semantic segmentation, and multi-scale feature extraction".

[0106] The urban green space dataset includes: multiple four-band images and urban green space segmentation results corresponding to the multiple four-band images. The urban green space segmentation results include the category label of each pixel as background, low vegetation or dense vegetation.

[0107] Specifically, the urban green space dataset is a dataset constructed for the task of urban green space segmentation. For example, multiple four-band images covering the target urban area are acquired using the Gaofen-1 satellite, and the four-band images are preprocessed; then, the preprocessed four-band images are sliced ​​according to a preset resolution / size, and pixel-level category labels are assigned to the four-band images of each slice to obtain the urban green space dataset. Furthermore, the urban green space dataset can be divided into training and testing datasets according to a preset ratio.

[0108] S302. Binarize the background, low vegetation, and dense vegetation labels in the urban green space dataset into vegetation labels and background labels to obtain a coarsely segmented dataset.

[0109] The coarse segmentation dataset includes: RGB images in each four-band image, and binary classification results corresponding to multiple RGB images. The binary classification results include the category label of each pixel as vegetation or background.

[0110] The labels for the urban green space dataset include: background, low vegetation, and dense vegetation. Therefore, it is necessary to binarize the three types of labels in the urban green space dataset, merge low vegetation and dense vegetation into a unified vegetation label, and retain only the background and vegetation labels.

[0111] In this embodiment, the coarse segmentation dataset reduces the complexity of the first-stage coarse segmentation task: there is no need to distinguish between low vegetation and dense vegetation, and only the focus is on fast and robust separation of vegetation and background. By simplifying the task objective, the coarse segmentation model can learn the common features of vegetation (such as green spectrum and vegetation texture) more efficiently, generate reliable vegetation probability maps and binary masks, and provide spatial priors for the second-stage fine classification.

[0112] S303. The coarse segmentation model is trained by transfer learning using a pre-set general large-scale dataset to obtain a coarse segmentation model after transfer learning. The coarse segmentation model after transfer learning is then fine-tuned using the coarse segmentation dataset to obtain a pre-trained coarse segmentation model.

[0113] The coarse segmentation model is trained by transfer learning using a pre-set general large-scale dataset to obtain the transfer training weights of the coarse segmentation model. The coarse segmentation model after transfer learning has the ability to extract general visual features, but it has not yet been adapted to the urban green space scenario. Therefore, the coarse segmentation model after transfer learning is fine-tuned using the coarse segmentation dataset to obtain a pre-trained coarse segmentation model that is adapted to the urban green space segmentation scenario.

[0114] In one possible implementation, the transfer learning strategy could be a progressive unfreezing strategy, a strategy of freezing some convolutional layers, or a group training strategy, etc.

[0115] Taking the progressive unfreezing strategy as an example: freeze the first three layers of the backbone network in the coarse segmentation model after transfer learning (reusing the general low-level features learned from the pre-set general large-scale dataset), and only unfreeze the model's classification head (responsible for the final binary classification output) and trainable encoder layer. Use the training set of the coarse segmentation dataset for training to optimize the classification head parameters to initially adapt to the binary classification logic of vegetation and background until the model loss function converges. After the classification head training converges, unfreeze the high layers of the backbone network and use the training set of the coarse segmentation dataset for end-to-end fine-tuning. At this time, the model can simultaneously optimize general features and vegetation-specific features (such as the green spectrum features and canopy texture features of vegetation), further improving its adaptability to urban vegetation scenes.

[0116] The above combination Figure 3 The process of obtaining a pre-trained coarse segmentation model has been explained in detail. The following section will continue with... Figure 1 This application introduces a deep learning-based method for segmenting urban green spaces.

[0117] S103. Based on four-band imagery, vegetation probability map, vegetation features, and texture features, a pre-trained fine classification model is used to output the corresponding three-class segmentation map.

[0118] The three-class segmentation map indicates whether each pixel in the four-band image belongs to the background, low vegetation, or dense vegetation category. Specifically, in the three-class segmentation map, a pixel belonging to the background category, such as buildings, roads, water bodies, bare soil, and other non-vegetated features, has a label value of 0; a pixel belonging to the low vegetation category, such as grasslands, lawns, low shrubs, and other vegetation with a uniform canopy structure, has a label value of 1; and a pixel belonging to the dense vegetation category, such as woodlands, dense shrublands, and other vegetation with a complex canopy structure, has a label value of 2.

[0119] To make it easier to understand, the following will be combined with... Figure 4 This paper describes how a pre-trained fine-classification model outputs a three-class segmentation map. Specifically, four-band imagery, vegetation probability maps, vegetation features, and texture features are input into the pre-trained fine-classification model to enable it to perform the following steps:

[0120] S401. Integrate the features of the four-band image, vegetation probability map, vegetation features and texture features to obtain initial feature data, and then perform multi-scale encoding processing on the initial feature data through an encoder to generate a multi-scale feature map.

[0121] Among them, the multi-scale feature map includes multiple scale feature maps of different scales.

[0122] Specifically, the core of feature integration is spatial dimension alignment and channel dimension splicing.

[0123] In one possible implementation, the encoder consists of a 7×7 convolutional layer → a max pooling layer → four hidden layers (including residual blocks). Assuming the initial feature data is 256×256, the features processed by the 7×7 convolutional layer and the max pooling layer will output a 64×64 scale feature map after the first hidden layer, a 32×32 scale feature map after the second hidden layer, a 16×16 scale feature map after the third hidden layer, and an 8×8 scale feature map after the fourth hidden layer.

[0124] Specifically, the encoder reduces the resolution of the initial feature data by downsampling layer by layer, while increasing the number of channels, ultimately generating multiple sets of scale feature maps at different scales. The smaller the scale, the lower the feature map resolution. Low-resolution, high-channel feature maps focus on global semantics, which helps distinguish macro categories such as large areas of dense vegetation and continuous backgrounds (buildings / roads). The larger the scale, the higher the feature map resolution. High-resolution, low-channel feature maps focus on local details, which can capture micro-features such as the edges of low vegetation and small patches of vegetation.

[0125] In the embodiments of this application, the fine classification model integrates multi-source features and extracts multi-scale features from details to the global scale through the encoder. This not only solves the problem that single spectral features are difficult to distinguish between low and dense vegetation, but also lays a comprehensive feature foundation for boundary optimization and category prediction in the subsequent decoding stage.

[0126] S402. The smallest scale feature map is upsampled layer by layer by the decoder to restore the resolution and obtain the corresponding second feature map.

[0127] Among them, the smallest scale feature map is the deepest output of the encoder of the fine classification model (i.e., the scale feature map of the last layer output). It is the feature map with the richest semantic information but the lowest spatial resolution. It needs to be restored to the original image resolution by upsampling layer by layer in order to achieve pixel-level classification.

[0128] Each layer of upsampling operation integrates an edge attention upsampling module.

[0129] For each layer of upsampling operation, the upsampling operation includes: the fine classification model upsampling the input feature map to obtain an upsampled feature map, and then performing edge attention processing on the upsampled feature map to generate an edge attention map that matches the scale of the upsampled feature map; the upsampled feature map and the corresponding edge attention map are fused element-wise using residual connections to obtain a fused feature map, and then the fused feature map and a scale feature map that matches the scale of the fused feature map are fused using skip connections to obtain the feature map output by the upsampling operation, thus completing one upsampling operation; wherein, the second feature map is the feature map output by the last layer of upsampling operation.

[0130] Among them, the edge attention map refers to the weight map that is perfectly matched with the scale of the feature map after input upsampling, which is generated to explicitly model the structural features of vegetation edges and guide the network to focus on the reconstruction of edge region details. Its core function is to quantify the importance of edges at different locations in the image, provide the basis for edge enhancement for subsequent feature fusion, and solve the problem of boundary blurring caused by ignoring edge structure differences in traditional upsampling operations.

[0131] For example, for the first-layer upsampling operation, the lowest-scale feature is upsampled to obtain an upsampled feature map; edge attention is applied to the upsampled feature map to generate an edge attention map that matches the scale of the upsampled feature map; the upsampled feature map and the corresponding edge attention map are fused element-wise using residual connections to obtain a fused feature map; and the fused feature map is then fused with a scale feature map (the second smallest scale feature map) that matches the scale of the fused feature map using skip connections to obtain the feature map output by the first-layer upsampling operation, thus completing one upsampling operation. The feature map output by the first-layer upsampling operation is then used as the input feature map for the second-layer upsampling operation.

[0132] In one possible implementation, the edge attention upsampling module can employ attention mechanisms such as EAUM, SE (Squeeze-and-Excitation), and CBAM.

[0133] In one possible implementation, the gradient magnitude of the upsampled feature map is extracted using a fixed edge detection operator to obtain the corresponding gradient magnitude map. A lightweight convolutional neural network consisting of two convolutional layers and a batch normalization (BatchNorm) and ReLU activation function between the two convolutional layers is used to optimize the features of the corresponding gradient magnitude map. Finally, an edge attention map matching the scale of the upsampled feature map is generated using the sigmoid function.

[0134] Fixed edge detection operators include the Sobel operator, Laplacian operator, and Canny operator. Fixed edge detection operators can reliably capture the horizontal and vertical edge gradients of the upsampled feature map, avoiding the edge feature degradation problem that occurs during training of trainable edge operators.

[0135] In this embodiment, gradient magnitudes are extracted using a fixed edge detection operator (such as Sobel). Leveraging mature prior knowledge of edge detection, the edge contours and intensity information of the upsampled feature map can be captured quickly and cost-effectively, providing a reliable foundation for subsequent edge refinement and avoiding the blindness of edge information extraction without prior guidance. Furthermore, a lightweight convolutional neural network optimizes the gradient magnitude map, adaptively strengthening edge features highly relevant to the urban green space segmentation task (such as the boundary between low and dense vegetation, and the boundary between vegetation and buildings) through learnable convolutional parameters, while ensuring controllable computational overhead. This compensates for the insufficient adaptability of fixed edge detection operators to complex scene edges. Further, the edge attention map generated by the Sigmoid function maps edge feature values ​​to the [0,1] interval, accurately quantifying the edge importance of each pixel position. Simultaneously, the attention map perfectly matches the scale of the upsampled feature map, providing a precise spatial correspondence for subsequent element-by-element fusion operations and ensuring the accuracy of edge enhancement.

[0136] To make it easier to understand, the following will be combined with... Figure 5 The structure of the edge attention upsampling module is shown, and an example is given to illustrate how to obtain the fused feature map.

[0137] like Figure 5 As shown, the input feature map is upsampled through an upsampling branch, namely a 3×3 convolution and an upsampling operation, to obtain an upsampled feature map. Specifically, the input feature map is initially transformed by a 3×3 convolution, and then the resolution of the feature map is amplified by upsampling operations (such as bilinear interpolation and transposed convolution), providing a scale basis that matches the original task for subsequent edge information processing.

[0138] like Figure 5 As shown, the upsampled feature map is convolved in both the horizontal and vertical directions using a fixed Sobel operator with a horizontal convolution kernel Kx and a vertical convolution kernel Ky, yielding gradient responses in both directions. The edge intensity of each pixel is calculated using the gradient magnitude formula to generate the gradient magnitude. The gradient magnitude map is used to a priori capture the edge positions and intensities between vegetation and background, and between vegetation subclasses in the image.

[0139] The horizontal convolution kernel Kx is used to capture the gradient changes of the upsampled feature map in the horizontal direction; the vertical convolution kernel Ky is used to capture the gradient changes of the upsampled feature map in the vertical direction.

[0140] To facilitate understanding, let's take an example of a gradient magnitude diagram based on formula (3):

[0141] (3)

[0142] Where E is the gradient magnitude map; F is the upsampled feature map; * is the convolution operation; Kx is the horizontal edge detection operator, i.e., the horizontal convolution kernel of the Sobel operator; Ky is the vertical edge detection operator, i.e., the vertical convolution kernel of the Sobel operator; ε is a very small positive number used to avoid the instability of calculation when the value inside the square root is 0, and to ensure the robustness of the formula calculation.

[0143] like Figure 5 As shown, the gradient magnitude map generates an edge attention map through "3×3 convolution + BatchNorm + ReLU + 1×1 convolution + Sigmoid". The two convolutional layers, BatchNorm, and ReLU refine the features of the gradient magnitude map in a learnable way, making the edge information more relevant to the task requirements (such as enhancing the features of vegetation boundaries in urban green spaces). Sigmoid activation maps the feature values ​​to the [0,1] interval to generate the edge attention map. The value of each pixel represents the probability that the position belongs to the edge (the closer the value is to 1, the stronger the edge importance), and the scale is completely consistent with the upsampled feature map, preparing for subsequent element-wise fusion.

[0144] To facilitate understanding, let's take an example of an edge attention graph, based on formula (4):

[0145] A=σ(Conv 1×1 (ReLU(BN(Conv 3×3 (E))))) (4)

[0146] Where A is the edge attention map; σ is the Sigmoid function; Conv 1×1 A 1×1 convolutional layer; ReLU is the ReLU activation function; BN is the batch normalization layer; Conv 3×3 It is a 3×3 convolutional layer; E is the gradient magnitude map.

[0147] like Figure 5As shown, the upsampled feature map is residually connected to the edge attention map and then fused using principal elements to obtain a fused feature map. The residual connection ensures that the original semantic information of the upsampled feature map is preserved (avoiding feature loss caused by edge enhancement); the element-wise fusion (usually element-wise multiplication or enhancement according to formula 1+A) enhances the features of regions with high edge importance in the edge attention map. The final output fused feature map contains rich semantic information and accurately enhances edge details, providing better feature input for subsequent fine-grained segmentation of urban green spaces (distinguishing between low vegetation, dense vegetation, and background).

[0148] To facilitate understanding, let's take an example of fused feature maps, based on formula (5):

[0149] Fout = F⊙(1+A) (5)

[0150] Where Fout is the fused feature map, F is the upsampled feature map, and A is the edge attention map.

[0151] S403. Based on the second feature map, the probability of each pixel in the four-band image belonging to the three categories of background, low vegetation, and dense vegetation is calculated by the softmax function, and the corresponding three-class probability map is generated. Based on the corresponding three-class probability map, the category of each pixel is determined by the argmax function, and the three-class segmentation map is generated and output.

[0152] The above combination Figure 4 and Figure 5 An example was given illustrating the upsampling operation in a fine-grained classification model. The following section will combine... Figure 6 The structure of the subdivision model is shown, and an example is given to illustrate how to obtain a three-class segmentation map.

[0153] like Figure 6 As shown, four-band imagery, vegetation probability maps, vegetation features, and texture features are input into a fine-grained classification model. The model first fuses these multi-source inputs at the channel or feature level to obtain initial feature data. For example, four-band imagery provides spectral information, vegetation features supplement vegetation cover information, texture features supplement texture differences, and vegetation probability maps provide prior probabilities of vegetation and background. The integration of multi-source information provides a rich feature foundation for subsequent encoding.

[0154] After the initial features are input into the encoder, they pass through a 7×7 convolutional layer, a max pooling layer, hidden layer 1, hidden layer 2, hidden layer 3, and hidden layer 4, generating multiple scale feature maps of different scales. For example, hidden layer 1 outputs a scale feature map with "256 channels and a resolution of H / 4×W / 4"; hidden layer 2 outputs a scale feature map with "512 channels and a resolution of H / 8×W / 8"; hidden layer 3 outputs a scale feature map with "1024 channels and a resolution of H / 16×W / 16"; and hidden layer 4 outputs a scale feature map with "2048 channels and a resolution of H / 32×W / 32" (the smallest scale feature map).

[0155] The feature map of hidden layer 4 is input to the edge upsampling layer to obtain a fused feature map of "H / 16×W / 16". This fused feature map is then connected to the scale feature map output by hidden layer 3 in the encoder via a skip connection to obtain an upsampled output feature map. This upsampled output feature map is then fed into the edge upsampling layer to obtain a fused feature map of "H / 8×W / 8". This fused feature map is then connected to the scale feature map output by hidden layer 2 in the encoder via a skip connection to obtain an upsampled output feature map. This process is repeated to output the second feature map.

[0156] By using a 3×3 convolutional layer and a 1×1 convolutional layer, a three-class probability map is obtained through the softmax function, and then the three-class segmentation map is obtained by processing the three-class probability map through the argmax function.

[0157] To make it easier to understand, the following will be combined with... Figure 7 This section details how to obtain a pre-trained fine-classification model.

[0158] S701. Obtain the urban green space dataset.

[0159] The urban green space dataset includes: multiple four-band images and urban green space segmentation results corresponding to the multiple four-band images respectively. The urban green space segmentation results include the category label of each pixel as background, low vegetation or dense vegetation.

[0160] S702. Based on the urban green space dataset, vegetation features and texture features corresponding to multiple four-band images are obtained. Based on the RGB images in the multiple four-band images in the urban green space dataset, vegetation probability maps corresponding to multiple four-band images are obtained through a pre-trained coarse segmentation model.

[0161] S703. Based on multiple four-band images, vegetation features and texture features corresponding to the multiple four-band images, vegetation probability maps corresponding to the multiple four-band images, and urban green space segmentation results corresponding to the multiple four-band images, a fine classification training set is obtained.

[0162] S704. Based on the fine-classification training set, the fine-classification model is trained using a weighted multi-task loss function to obtain a pre-trained fine-classification model.

[0163] The weighted multi-task loss function is obtained by weighted summation of the weighted cross-entropy loss function and the vegetation class Dice loss function, as shown in formula (6):

[0164] Loss=λ1×LossCE+λ2×LossDice (6)

[0165] Where Loss is the weighted multi-task loss function, λ1 is the weight of the weighted cross-entropy loss function, LossCE is the weighted cross-entropy loss function, λ2 is the weight of the vegetation class Dice loss function, and LossDice is the Dice loss function.

[0166] In the weighted cross-entropy loss function, the class weights are inversely proportional to the class frequencies. In urban scenes, the pixel frequency of low-lying vegetation is much lower than that of the background and dense vegetation (class imbalance). Therefore, the weight of low-lying vegetation is set to a higher value, while the background has a lower weight. By assigning high weights to the minor class (low-lying vegetation), the model is forced to pay more attention to the classification error of the minor class during training, solving the problem of "missed detection of minor classes due to class imbalance" and improving the recall rate of the minor class.

[0167] To facilitate understanding, the weighted cross-entropy loss function will be introduced below in conjunction with formula (7):

[0168] (7)

[0169] Where LossCE is the weighted cross-entropy loss function; c is the category label index, with values ​​of 0, 1, and 2, corresponding to the three category labels: background, low vegetation, and dense vegetation, respectively; w c For class weights; y c Let y be a category function, where y is the true category of the sample when y is c. c =1 otherwise 0, used to label the true class of the sample; p c To predict the probability that a sample belongs to class c for a fine-grained classification model; f c Let w be the frequency of category c. And let w be the category weight. c The frequency f of the square root of category c c The reciprocal of.

[0170] Among them, the category probability is obtained by collecting a large amount of data. The frequency of low vegetation is low, assumed to be 0.05, then the corresponding category weight is 4.47; the frequency of background is high, assumed to be 0.55, then the corresponding category weight is about 1.35.

[0171] In the vegetation loss function, the weight of the low-lying vegetation category is set to 3, and the weight of the dense vegetation category is set to 0.5. The Dice loss function is more sensitive to segmentation errors in boundary regions and small areas, making it suitable for optimizing targets like vegetation that have complex edges and a high proportion of small categories. Giving low-lying vegetation (a smaller category, more prone to missed detection) a high weight (i.e., 3) can specifically improve its boundary segmentation accuracy and overall recall; the relatively low weight (i.e., 0.5) for dense vegetation is to ensure its segmentation effect while avoiding interference with the optimization of low-lying vegetation.

[0172] To facilitate understanding, the vegetation loss function will be introduced below in conjunction with formula (8):

[0173] (8)

[0174] Where LossDice is the vegetation class loss function, used to optimize the segmentation accuracy of low vegetation and dense vegetation; c is the class label index, with values ​​of 1 and 2, corresponding to the three class labels of low vegetation and dense vegetation, respectively; λ c y represents the category weights used to adjust the importance of different vegetation classes in the loss; the weight for low-lying vegetation is set to 3, and the weight for dense vegetation is set to 0.5. c Let y be a category function, where y is the true class of the sample when c is true. c =1 otherwise 0, used to label the true class of the sample; p c Predict the probability that a sample belongs to class c for a fine-classification model.

[0175] Where, ∑p c y c This represents the intersection of the predicted and ground truth labels, i.e., the sum of the number of pixels that the model predicts as class c and that are actually class c, reflecting the "accuracy" of the segmentation. ∑p c +∑y c This represents the union of the predicted and true labels, which is the sum of the number of pixels predicted by the model as class c and the number of pixels that are actually class c, reflecting the "recall" of the segmentation.

[0176] The above combination Figure 7 This section introduced how to obtain a pre-trained fine-classification model. The following section will continue with... Figure 1 This application introduces a deep learning-based method for segmenting urban green spaces.

[0177] S104. Based on the binary mask image, the three-class segmentation image is fused to obtain the urban green space segmentation result corresponding to the four-band image.

[0178] The urban green space segmentation results include: the category label for each pixel is background, low vegetation, or dense vegetation.

[0179] Specifically, the results of urban green space segmentation are used to support urban ecological monitoring and assessment (quantifying ecological service functions), urban planning and management (providing accurate spatial data), smart city and environmental monitoring (achieving automated dynamic supervision), and policy formulation and evaluation (providing quantitative basis).

[0180] In one possible implementation, based on a binary mask image, the category of the pixel corresponding to the vegetation area in the binary mask image in the three-class segmentation image is retained as the category label of the pixel corresponding to the vegetation area in the binary mask image in the four-band image. The category label of the pixel corresponding to the background area in the binary mask image in the four-band image is uniformly updated to background, thus obtaining the urban green space segmentation result corresponding to the four-band image.

[0181] Specifically, using the prior knowledge of the background and vegetation regions in the binary mask image, the category determination of the three-class segmentation image is constrained. For each pixel in the segmentation image, its value in the binary mask image is determined:

[0182] If the pixel is 0 (background area) in the binary mask image, the category label of the pixel in the final urban green space segmentation result will be directly determined as background; if the pixel is 1 (vegetation area) in the binary mask image, the category of the pixel in the three-class segmentation image (i.e., low vegetation, dense vegetation or background) will be retained.

[0183] In this embodiment, the coarse segmentation model achieves accurate separation between vegetation areas and background areas, ensuring that the determination of background areas is not interfered with by potential misjudgments of the fine classification model; the fine classification model's ability to distinguish between low-lying vegetation and dense vegetation ensures that the fine-grained classification results of vegetation areas are effective.

[0184] To make it easier to understand, the following will be combined with... Figure 8 This paper provides an example of an urban green space segmentation method based on deep learning, which yields an urban green space segmentation result.

[0185] like Figure 8 The original four-band image shown in (a) presents a grayscale visual effect of the urban area, in which the spectral and texture information of various land features such as buildings, roads, and different types of vegetation are mixed, making it difficult for the human eye to directly distinguish the boundaries and categories of low vegetation, dense vegetation and background areas.

[0186] like Figure 8 The urban green space segmentation result shown in (b) is the urban green space segmentation result corresponding to the original four-band image. Figure 8In (b), different shades of gray are used to distinguish between background, low vegetation, and dense vegetation. Black represents the background, light gray represents low vegetation, and dark gray represents dense vegetation. Background includes all non-vegetation features such as buildings, roads, water bodies, and bare soil; low vegetation includes vegetation types with relatively uniform canopy structures such as grasslands, lawns, and low shrubs; dense vegetation includes vegetation types with complex three-dimensional structures and rough textures such as woodlands and dense shrublands.

[0187] Furthermore, the evaluation metrics of the urban green space segmentation method based on deep learning provided in this application embodiment include: intersection-union ratio, average pixel accuracy, average F1 score, and average intersection-union ratio. These metrics comprehensively measure the segmentation performance from multiple aspects such as pixel-level accuracy, category recognition ability, and region boundary fit.

[0188] Intersection over Union (IoU) measures the degree of overlap between the predicted region and the actual region for a single category.

[0189] Mean Pixel Accuracy (MPA) calculates the pixel prediction accuracy for each category separately and then averages it across all categories. This is used to avoid the excessive dominance of large categories (such as background) on the overall accuracy and to more fairly reflect the pixel-level prediction accuracy of each category.

[0190] The Mean F1 Score is the harmonic mean of precision and recall for each class, and then averaged over all classes. It is used to balance the accuracy of prediction for a class with the ability of a class to be fully recalled.

[0191] The mean intersection over union (mIoU) is the average of the intersection over union (IoU) of all categories, which can simultaneously reflect the segmentation accuracy of each category and the discriminative power between categories.

[0192] Table 1 shows the evaluation results of the urban green space segmentation method in this application embodiment, and compares them with the evaluation results of the segmentation effects of other existing segmentation models.

[0193] Table 1

[0194]

[0195] As shown in Table 1, the urban green space segmentation method of this application embodiment has the best segmentation effect.

[0196] This application provides a deep learning-based method for urban green space segmentation, comprising: acquiring four-band satellite remote sensing images and obtaining corresponding vegetation features and texture features based on the four-band images; obtaining corresponding vegetation probability maps and binary mask maps based on the RGB images in the four-band images through a pre-trained coarse segmentation model; outputting corresponding three-class segmentation maps based on the four-band images, vegetation probability maps, vegetation features, and texture features through a pre-trained fine classification model; and fusing the three-class segmentation maps based on the binary mask maps to obtain the urban green space segmentation result corresponding to the four-band images. This application embodiment acquires four-band satellite remote sensing images and extracts vegetation and texture features. A pre-trained coarse segmentation model is used to initially separate vegetation from the background and obtain a vegetation probability map and a binary mask map. Then, the four-band images, vegetation probability map, and vegetation and texture features are combined with a pre-trained fine classification model to output a three-class segmentation map. Finally, the binary mask map is fused, which effectively reduces the computational cost of the single-stage deep learning model and solves the problems of boundary blurring, local structure loss, and missed detection of small classes such as low vegetation caused by weakened edge information in complex urban backgrounds. This significantly improves the overall accuracy and practicality of urban green space segmentation.

[0197] Specifically, this application embodiment achieves urban green space segmentation based on two-stage deep learning. In the first stage, a coarse segmentation model is used to complete the coarse segmentation of vegetation and background areas. The generated vegetation probability map and binary mask map provide a clear spatial constraint range for the fine classification in the second stage. This allows the fine classification model in the second stage to focus on high-confidence vegetation areas without having to perform fine-grained classification on the entire image. This significantly reduces the search space and background interference of the fine classification task, reducing unnecessary computational overhead and avoiding the problem of background information overshadowing vegetation details during full-image fine classification.

[0198] The second-stage fine-classification model uses the vegetation probability map from the first stage as spatial guidance, integrating multi-source features to complete the fine-classification of low-lying and dense vegetation. This combination of spatial prior and multi-source features solves the pain point in current technology that "it is difficult to distinguish spectrally similar but structurally different vegetation by relying solely on spectral features" (e.g., low-lying grassland and sparse woodland have similar spectra but significant differences in texture and density), significantly improving the ability to distinguish vegetation subclasses, and finally outputting a three-class segmentation map, meeting the core requirement of fine-grained segmentation of urban green spaces.

[0199] Furthermore, the coarse segmentation model is trained using transfer learning, which enables it to converge quickly even in small sample scenarios, improving the initialization performance of the coarse segmentation model and reducing the difficulty of training and tuning the coarse segmentation model.

[0200] Furthermore, the fine-grained classification model utilizes multi-source inputs—four-band imagery, vegetation features, texture features, and vegetation probability maps—to achieve complementary features and form a comprehensive vegetation description system. Four-band imagery provides fundamental spectral information, serving as the core basis for vegetation identification; vegetation features quantify vegetation coverage and growth status; texture features capture local structural differences; and the first-stage vegetation probability map provides spatial location priors, preventing non-vegetated area features from interfering with vegetation classification during feature fusion. This multi-source information collaborative modeling completely solves the current technology's problems of insufficient utilization of multi-source information and difficulty in distinguishing vegetation subclasses based solely on spectral data, significantly improving the model's ability to represent differences between low-lying and dense vegetation.

[0201] Furthermore, the edge attention upsampling module in the fine classification model explicitly strengthens the edge region features by fusing the residuals of the upsampled feature map and the edge attention map, which solves the defects of traditional upsampling edge information weakening, resulting in blurred boundaries and local fragmentation, and significantly improves the vegetation boundary fit.

[0202] Example 2:

[0203] The following is combined with Figure 9 This application provides a detailed description of a deep learning-based urban green space segmentation device.

[0204] like Figure 9 As shown in the figure, the urban green space segmentation device based on deep learning provided in this application includes the following modules:

[0205] The acquisition module 901 is used to acquire four-band images from satellite remote sensing and obtain corresponding vegetation features and texture features based on the four-band images; wherein, the four-band images include: near-infrared (NIR) band, red (R) band, green (G) band and blue (B) band images;

[0206] The coarse segmentation module 902 is used to obtain the corresponding vegetation probability map and binary mask map based on the RGB image in the four-band image through a pre-trained coarse segmentation model. The vegetation probability map is used to indicate the probability that each pixel in the RGB image belongs to the vegetation category, and the binary mask map is used to distinguish the background area and vegetation area in the RGB image in a binarized manner.

[0207] The fine classification module 903 is used to output a corresponding three-class segmentation map based on four-band imagery, vegetation probability map, vegetation features and texture features, through a pre-trained fine classification model; wherein, the three-class segmentation map is used to indicate whether each pixel in the four-band imagery is background, low vegetation or dense vegetation.

[0208] The fusion module 904 is used to fuse the three-class segmentation map based on the binary mask map to obtain the urban green space segmentation result corresponding to the four-band image: the urban green space segmentation result includes: the category label of each pixel is background, low vegetation or dense vegetation.

[0209] In one possible implementation, the coarse segmentation module 902 is specifically used to input the RGB image from the four-band image into the pre-trained coarse segmentation model, so that the coarse segmentation model can encode the RGB image to generate a first feature map corresponding to the RGB image; based on the first feature map, the coarse segmentation model calculates the probability of each pixel in the RGB image belonging to a vegetation category through the softmax function, generates the corresponding vegetation probability map, and based on the corresponding vegetation probability map, performs category determination on the probability of each pixel belonging to a vegetation category through the argmax function, generates the corresponding binary mask map, and outputs the vegetation probability map and the binary mask map.

[0210] In one possible implementation, the fine classification module 903 includes a downsampling module, an upsampling module, and a three-classification module.

[0211] The downsampling module is used to input four-band imagery, vegetation probability maps, vegetation features, and texture features into a pre-trained fine-grained classification model. This allows the fine-grained classification model to integrate features from the four-band imagery, vegetation probability maps, vegetation features, and texture features to obtain initial feature data. The encoder then performs multi-scale encoding on the initial feature data to generate multi-scale feature maps. These multi-scale feature maps include multiple scale feature maps of different scales.

[0212] The upsampling module is used to recover the resolution of the smallest scale feature map by performing upsampling operations layer by layer through the decoder to obtain the corresponding second feature map. The edge attention upsampling module is integrated in each upsampling operation.

[0213] For each layer of upsampling operation, the upsampling operation includes: the fine classification model upsampling the input feature map to obtain an upsampled feature map, and then performing edge attention processing on the upsampled feature map to generate an edge attention map that matches the scale of the upsampled feature map; the upsampled feature map and the corresponding edge attention map are fused element-wise using residual connections to obtain a fused feature map, and then the fused feature map is fused with a scale feature map that matches the scale of the fused feature map through skip connections to obtain the feature map output by the upsampling operation, thus completing one upsampling operation; the second feature map is the feature map output by the last layer of upsampling operation;

[0214] The three-classification module is used to calculate the probability of each pixel in the four-band image belonging to the three categories of background, low vegetation, and dense vegetation based on the second feature map and using the softmax function, generating the corresponding three-classification probability map. Based on the corresponding three-classification probability map, the argmax function is used to determine the category of each pixel, generating and outputting the three-classification segmentation map.

[0215] In one possible implementation, the upsampling module is specifically used to extract the gradient magnitude of the upsampled feature map using a fixed edge detection operator to obtain the corresponding gradient magnitude map; a lightweight convolutional neural network consisting of two convolutional layers, BatchNorm and ReLU activation function between the two convolutional layers is used to optimize the features of the corresponding gradient magnitude map, and an edge attention map with a scale matching the upsampled feature map is generated by the sigmoid function.

[0216] In one possible implementation, the fusion module 904 is specifically used to retain the category of the pixel corresponding to the vegetation area in the binary mask in the three-class segmentation image, based on the binary mask image, as the category label of the pixel corresponding to the vegetation area in the binary mask in the four-band image, and uniformly update the category label of the pixel corresponding to the background area in the binary mask in the four-band image to the background, so as to obtain the urban green space segmentation result corresponding to the four-band image.

[0217] In one possible implementation, the device further includes a preprocessing module. The preprocessing module performs radiometric correction, atmospheric correction, orthorectification, panchromatic and multispectral fusion, and image registration on the four-band imagery to obtain preprocessed four-band imagery.

[0218] This application provides a deep learning-based urban green space segmentation device, comprising: an acquisition module 901 for acquiring four-band satellite remote sensing images and obtaining corresponding vegetation features and texture features based on the four-band images; a coarse segmentation module 902 for obtaining corresponding vegetation probability maps and binary mask maps based on RGB images in the four-band images using a pre-trained coarse segmentation model; a fine classification module 903 for outputting corresponding three-class segmentation maps based on the four-band images, vegetation probability maps, vegetation features, and texture features using a pre-trained fine classification model; and a fusion module 904 for fusing the three-class segmentation maps based on the binary mask map to obtain urban green space segmentation results corresponding to the four-band images. This application embodiment acquires four-band satellite remote sensing images and extracts vegetation and texture features. A pre-trained coarse segmentation model is used to initially separate vegetation from the background and obtain a vegetation probability map and a binary mask map. Then, the four-band images, vegetation probability map, and vegetation and texture features are combined with a pre-trained fine classification model to output a three-class segmentation map. Finally, the binary mask map is fused, which effectively reduces the computational cost of the single-stage deep learning model and solves the problems of boundary blurring, local structure loss, and missed detection of small classes such as low vegetation caused by weakened edge information in complex urban backgrounds. This significantly improves the overall accuracy and practicality of urban green space segmentation.

[0219] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0220] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A deep learning-based method for segmenting urban green spaces, characterized in that, include: Four-band satellite remote sensing images are acquired, and corresponding vegetation features and texture features are obtained based on the four-band images; wherein, the four-band images include: near-infrared (NIR) band, red (R) band, green (G) band, and blue (B) band images; Based on the RGB images in the four-band imagery, a pre-trained coarse segmentation model is used to obtain the corresponding vegetation probability map and binary mask image. The vegetation probability map is used to indicate the probability that each pixel in the RGB image belongs to a vegetation category, and the binary mask image is used to distinguish the background area and vegetation area in the RGB image in a binarized manner. Based on the four-band imagery, the vegetation probability map, the vegetation features, and the texture features, a pre-trained fine classification model outputs a corresponding three-class segmentation map; wherein, the three-class segmentation map is used to indicate whether each pixel in the four-band imagery is classified as background, low vegetation, or dense vegetation. Based on the binary mask image, the three-class segmentation image is fused to obtain the urban green space segmentation result corresponding to the four-band image: wherein the urban green space segmentation result includes: the category label of each pixel is background, low vegetation or dense vegetation.

2. The method according to claim 1, characterized in that, The RGB images from the four-band imagery are used to obtain corresponding vegetation probability maps and binary mask maps through a pre-trained coarse segmentation model, including: The RGB images from the four-band imagery are input into a pre-trained coarse segmentation model, which encodes the RGB images to generate a first feature map corresponding to the RGB images. Based on the first feature map, the coarse segmentation model calculates the probability of each pixel in the RGB images belonging to a vegetation category using the softmax function, generating a corresponding vegetation probability map. Based on the corresponding vegetation probability map, the argmax function is used to determine the category of each pixel's probability of belonging to a vegetation category, generating a corresponding binary mask map, and outputting the vegetation probability map and the binary mask map.

3. The method according to claim 1 or 2, characterized in that, The pre-trained coarse segmentation model was obtained through transfer learning. The pre-trained coarse segmentation model is obtained through transfer learning and includes: Obtain a preset general large-scale dataset and an urban green space dataset; wherein, the urban green space dataset includes: multiple four-band images and urban green space segmentation results corresponding to the multiple four-band images respectively, and the urban green space segmentation results include the category label of each pixel as background, low vegetation or dense vegetation; The background, low vegetation, and dense vegetation labels in the urban green space dataset are binarized into vegetation labels and background labels to obtain a coarse segmentation dataset. The coarse segmentation dataset includes: RGB images in each of the four-band images, and binary classification results corresponding to multiple RGB images respectively. The binary classification results include the category label of each pixel as vegetation or background. The coarse segmentation model is trained by transfer learning using the preset general large-scale dataset to obtain a coarse segmentation model after transfer learning. The coarse segmentation model after transfer learning is then fine-tuned using the coarse segmentation dataset to obtain a pre-trained coarse segmentation model.

4. The method according to claim 1, characterized in that, Based on the four-band imagery, the vegetation probability map, the vegetation features, and the texture features, a pre-trained fine-classification model outputs a corresponding three-class segmentation map, including: The four-band image, the vegetation probability map, the vegetation features, and the texture features are input into a pre-trained fine classification model, so that the fine classification model can integrate the features of the four-band image, the vegetation probability map, the vegetation features, and the texture features to obtain initial feature data. The initial feature data is then processed by an encoder to perform multi-scale encoding to generate a multi-scale feature map. The multi-scale feature map includes multiple scale feature maps of different scales. The fine classification model uses a decoder to perform upsampling operations layer by layer on the smallest scale feature map to restore the resolution and obtain the corresponding second feature map. Each upsampling operation integrates an edge attention upsampling module. Specifically, for each upsampling operation, the upsampling operation includes: the fine classification model upsampling the input feature map to obtain an upsampled feature map, and then performing edge attention processing on the upsampled feature map to generate an edge attention map that matches the scale of the upsampled feature map; the upsampled feature map and the corresponding edge attention map are fused element-wise using residual connections to obtain a fused feature map, and then the fused feature map and a scale feature map that matches the scale of the fused feature map are fused using skip connections to obtain the feature map output by the upsampling operation, thus completing one upsampling operation; wherein, the second feature map is the feature map output by the last upsampling operation; The fine classification model is based on the second feature map. It uses the softmax function to calculate the probability that each pixel in the four-band image belongs to the three categories of background, low vegetation, and dense vegetation, and generates a corresponding three-class probability map. Based on the corresponding three-class probability map, the argmax function is used to determine the category of each pixel, and a three-class segmentation map is generated and output.

5. The method according to claim 4, characterized in that, The step of performing edge attention processing on the upsampled feature map to generate an edge attention map that matches the scale of the upsampled feature map includes: By using a fixed edge detection operator, the gradient magnitude of the upsampled feature map is extracted to obtain the corresponding gradient magnitude map; A lightweight convolutional neural network consisting of two convolutional layers and a batch normalization (BatchNorm) and ReLU activation function between the two convolutional layers is used to optimize the corresponding gradient magnitude map, and an edge attention map matching the scale of the upsampled feature map is generated by the sigmoid function.

6. The method according to claim 4, characterized in that, The pre-trained fine-classification model was trained in the following way: Obtain an urban green space dataset; wherein, the urban green space dataset includes: multiple four-band images and urban green space segmentation results corresponding to the multiple four-band images respectively, and the urban green space segmentation results include the category label of each pixel as background, low vegetation or dense vegetation; Based on the urban green space dataset, vegetation features and texture features corresponding to the multiple four-band images are obtained respectively. Based on the RGB images in the multiple four-band images in the urban green space dataset, vegetation probability maps corresponding to the multiple four-band images are obtained through a pre-trained coarse segmentation model. Based on the multiple four-band images, the vegetation features and texture features corresponding to the multiple four-band images, the vegetation probability maps corresponding to the multiple four-band images, and the urban green space segmentation results corresponding to the multiple four-band images, a fine classification training set is obtained. Based on the fine-classification training set, the fine-classification model is trained using a weighted multi-task loss function to obtain a pre-trained fine-classification model. The weighted multi-task loss function is obtained by weighted summation of a weighted cross-entropy loss function and a vegetation class Dice loss function. In the weighted cross-entropy loss function, the class weights are inversely proportional to the class frequencies. In the vegetation class Dice loss function, the weight of the low-growing vegetation class is set to 3, and the weight of the dense vegetation class is set to 0.

5.

7. The method according to claim 1, characterized in that, The process of fusing the three-class segmentation map based on the binary mask image to obtain the urban green space segmentation result corresponding to the four-band image includes: Based on the binary mask image, the categories of pixels in the three-class segmentation image corresponding to the vegetation areas in the binary mask image are retained as category labels for pixels in the four-band image corresponding to the vegetation areas in the binary mask image. The category labels of pixels in the four-band image corresponding to the background areas in the binary mask image are uniformly updated to background, thus obtaining the urban green space segmentation result corresponding to the four-band image.

8. The method according to claim 1, characterized in that, The vegetation characteristics include: Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index (EVI), or Soil Adjusted Vegetation Index (SAVI). The texture features include: Local Binary Pattern (LBP), Gray-Level Co-occurrence Matrix (GLCM), or Histogram of Oriented Gradients (HOG).

9. The method according to claim 1, characterized in that, After acquiring the four-band satellite remote sensing imagery, the method further includes: Radiometric correction, atmospheric correction, orthorectification, panchromatic and multispectral fusion, and image registration are performed on the four-band images to obtain preprocessed four-band images.

10. A deep learning-based urban green space segmentation device, characterized in that, include: The acquisition module is used to acquire four-band images from satellite remote sensing, and obtain corresponding vegetation features and texture features based on the four-band images; wherein, the four-band images include: near-infrared (NIR) band, red (R) band, green (G) band, and blue (B) band images; The coarse segmentation module is used to obtain the corresponding vegetation probability map and binary mask map based on the RGB image in the four-band image using a pre-trained coarse segmentation model; wherein, the vegetation probability map is used to indicate the probability that each pixel in the RGB image belongs to a vegetation category, and the binary mask map is used to distinguish the background area and vegetation area in the RGB image in a binarized manner. The fine classification module is used to output a corresponding three-class segmentation map based on the four-band image, the vegetation probability map, the vegetation features and texture features, through a pre-trained fine classification model; wherein, the three-class segmentation map is used to indicate the category of each pixel in the four-band image as background, low vegetation or dense vegetation. The fusion module is used to perform fusion processing on the three-class segmentation map based on the binary mask map to obtain the urban green space segmentation result corresponding to the four-band image: wherein the urban green space segmentation result includes: the category label of each pixel is background, low vegetation or dense vegetation.

Images