Urban vegetation mapping at high resolution based on multiscale vegetation index guided approach

Abstract

The application provides a kind of urban vegetation high-resolution mapping method based on cross-scale vegetation index guide. Main steps include: 1) two scenes cover the same area, obtain time similar Sentinel-2 L2A image and high-resolution RGB image are resampled, geometric registration, radiation correction and other pretreatment operations;2) a new cross-scale vegetation index is proposed, the coverage of urban vegetation is extracted initially, and urban vegetation and non-urban vegetation samples are obtained automatically by threshold segmentation;3) the semantic segmentation model DeepLab v3+ is trained using the automatically generated samples, and the trained model can directly predict the high-resolution mapping result of urban vegetation. The technical advantage of the method is that without any manual sample labeling, high-precision urban vegetation high-resolution mapping results can be obtained automatically, which can provide a low-cost and effective solution for large-scale urban vegetation high-resolution mapping.

Description

Technical Field

[0001] This invention relates to the field of remote sensing measurement technology, and in particular to a high-resolution mapping method for urban vegetation based on cross-scale vegetation indices. Background Technology

[0002] With rapid urbanization, urban populations are increasing dramatically. Urban ecosystems are facing unprecedented challenges, including a range of environmental problems such as climate change, biodiversity loss, and various forms of pollution. A growing number of people believe that urban vegetation (UV) plays a beneficial role in mitigating these environmental pressures, such as carbon sequestration, noise reduction, mitigating the urban heat island effect, and improving the physical and mental health of urban residents. Therefore, accurate and timely information on urban vegetation distribution is crucial for urban planning and management, as well as urban environmental research.

[0003] Remote sensing technology, with its advantages of multi-scale, repeated observation, and wide coverage, is widely used for vegetation mapping in various scenarios. Compared to natural vegetation, urban vegetation appears as smaller, more fragmented patches in remote sensing imagery, and the background environment is more complex, making urban vegetation extraction more challenging. Multispectral images (MSI) with medium spatial resolution (e.g., Landsat and Sentinel-2) are widely used for mapping and monitoring vegetation. Due to their rich spectral information, MSI can provide key spectral features of vegetation, which are crucial for urban vegetation extraction. The normalized difference vegetation index (NDVI) is an effective indicator for extracting and monitoring vegetation cover because it utilizes the most prominent vegetation features: high reflectivity in the near-infrared (NIR) band and strong absorption at red wavelengths. Although MSI is suitable for large-scale vegetation mapping, its low spatial resolution prevents it from capturing sufficient spatial detail. Accurate information on urban vegetation cover and its changes is crucial for urban management; therefore, it is necessary to develop high-resolution urban vegetation mapping methods.

[0004] In recent years, high-resolution RGB imagery has become the most commonly used data source for detailed urban vegetation mapping due to its lower cost compared to multispectral sensors. Vegetation mapping methods using RGB imagery can be divided into spectral index-based vegetation mapping and classifier-based vegetation mapping. Currently, the most widely used vegetation indices that only use the RGB band include NGRDI [see Tucker C J. Red and photographic infrared linear combinations for monitoring vegetation[J]. Remote sensing of Environment, 1979, 8(2): 127-150.], GLI [see Louhaichi M, Borman MM, Johnson D E. Spatially located platform and aerial photography for documentation of grazing impacts on wheat[J]. Geocarto International, 2001, 16(1): 65-70.], RGBVI [see Bendig J, Yu K, Aasen H, et al. Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring inbarley[J]. International Journal of Applied Earth Observation and Geoinformation, 2015, 39: 79-87.], and MGRVI [see Bendig J, Yu K, Aasen H, et al. Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring inbarley[J]. International Journal of Applied Earth Observation and Geoinformation, 2015, 39: 79-87.], and MGRVI [see Bendig J, Yu K, Aasen H, et al. Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring inbarley[J]. International Journal of Applied Earth Observation and Geoinformation, 2015, 39: 79-87.], and MGRVI [see Bendig J, Yu K, Aasen H, et al. Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring inbarley]. al.Combining UAV-based plant height from crop surface models, visible, and nearinfrared vegetation indices for biomass monitoring in barley[J].International Journal of Applied Earth Observation and Geoinformation, 2015,39: 79-87.Indices such as VARI [see Gitelson AA, Kaufman YJ, Stark R, et al. Novel algorithms for remote estimation of vegetation fraction[J]. Remote sensing of Environment, 2002, 80(1): 76-87.] are typically designed based on the characteristic that vegetation has high reflectance in the green band and low reflectance in the red and blue bands. However, studies have shown that the red, green, and blue bands contain too little spectral information for accurate vegetation mapping [see R. Neyns and F. Canters, “Mapping of Urban Vegetation with High-Resolution Remote Sensing: A Review,” Remote Sens., vol. 14, no. 4, 2022.]. Some non-vegetated land cover types (such as water bodies) have similar spectral characteristics to vegetation: high reflectance in the green band and low reflectance in the red and blue bands, making it difficult for the above indices to accurately distinguish vegetation from other land cover. Some studies have combined lidar data and RGB imagery to facilitate urban vegetation extraction [see N. Audebert, B. Le Saux, and S. Lefèvre, “Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks,” ISPRS J. Photogramm. RemoteSens., vol. 140, pp. 20–32, 2018]. Baker et al. combined citizen science data (vector polygon data) with RGB imagery to map the spatial distribution of urban domestic gardens [see Baker F, Smith CL, Cavan G. A combined approach to classifying land surface cover of urban domestic gardens using citizen science data and high resolution image analysis[J]. RemoteSensing, 2018, 10(4): 537.]However, these auxiliary data are costly and difficult to obtain. Due to the unique spectral characteristics of vegetation compared to all other land surface types, both the red and near-infrared bands are indispensable for accurate vegetation detection: light radiation in the red band is absorbed by chlorophyll, while light radiation in the near-infrared band is reflected by leaf cell structures [see Tucker C J. Red and photographic infrared linear combinations for monitoring vegetation[J]. Remote sensing of Environment, 1979, 8(2): 127-150.]. Therefore, utilizing the near-infrared band of freely available multispectral data (such as Sentinel-2) seems to be a reasonable and applicable solution for improving the accuracy of vegetation mapping.

[0005] Because existing RGB vegetation indices are insufficient for distinguishing vegetation against complex backgrounds, some researchers have introduced advanced classifiers to improve the accuracy of vegetation mapping. Typically, researchers first manually design feature sets comprising spectral, textural, geometric, or contextual features, and then feed these into supervised machine learning classifiers for classification. In recent years, deep learning technology has made significant progress in land cover mapping. The advantage of deep learning lies in its ability to automatically extract multi-level features from low to high levels, rather than relying on manually designed features. Since the advent of fully convolutional neural networks (FCNs), image semantic segmentation techniques have made great strides. Many scholars have used FCN-like models (such as Deeplab, UNet, and SegNet) for urban vegetation extraction and achieved good mapping accuracy. However, these deep learning models are also difficult to apply in practice due to their reliance on large numbers of labeled samples, especially for urban vegetation mapping of large areas.

[0006] In summary, both spectral index methods and image classification methods face their own challenges in urban vegetation mapping, namely, difficulty in accurately distinguishing vegetation from complex backgrounds or heavy reliance on a large number of labeled samples. Therefore, there is an urgent need to develop new methods to address these issues and achieve the goal of large-scale, accurate urban vegetation mapping. Summary of the Invention

[0007] This invention addresses the shortcomings of existing technologies by providing a high-resolution urban vegetation mapping method guided by cross-scale vegetation indices. It overcomes the problems of existing high-resolution urban vegetation mapping, such as the difficulty in accurately distinguishing vegetation from complex backgrounds and the high dependence on a large number of labeled samples. The invention proposes a high-precision automatic high-resolution urban vegetation mapping method that requires no sample labeling.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A high-resolution urban vegetation mapping method guided by cross-scale vegetation indices includes the following steps:

[0010] S1: For the same area, acquire Sentinel-2 L2A images and high-resolution RGB images at similar times, and resample the Sentinel-2 L2A images to make their spatial resolution consistent with that of the high-resolution RGB images.

[0011] S2: For the two images in step S1, select a number of pixels as control points and perform spatial registration operation between the Sentinel-2 L2A image and the high-resolution RGB image to make the two images coincide in spatial position.

[0012] S3: Using the surface reflectance of the Sentinel-2 L2A image as a benchmark, perform relative radiometric correction on the red band of the high-resolution RGB image to adjust the reflectance of the original RGB image's red band. Correction to This reduces the difference in surface reflectance between the two images;

[0013] S4: Based on the two images processed in steps S1-S3, calculate the cross-scale vegetation index at each location in the region. ,get CSVI Imagery; with cross-scale vegetation indices at various locations. The calculation formula is:

[0014]

[0015] In the formula, express CSVI The first in the image i Line 1 j Cross-scale vegetation indices of the column; , These represent the high-resolution RGB image number 1 and 2 respectively. i Line 1 j The surface reflectance of the green and blue bands of the array; This indicates the Sentinel-2 L2A image number 1 i Line 1 j Surface reflectance in the near-infrared band; This indicates the high-resolution RGB image after relative radiometric correction. i Line 1 j Surface reflectance in the red band; numerical value n It is used to expand the range of vegetation indices across scales;

[0016] S5: Set threshold , Threshold segmentation CSVI Image generation of pseudo samples of urban and non-urban vegetation S The specific method is as follows:

[0017]

[0018] In the formula, express CSVI The first in the image i Line 1 j The sample labels for each pixel are defined as follows: a label value of 0 indicates that the pixel is non-urban vegetation, a label value of 1 indicates that the pixel is urban vegetation, and a label value of 2 indicates that the pixel is of an uncertain type and should be ignored.

[0019] S6: The pseudo-samples generated in step S5 S The DeepLab v3+ semantic segmentation model was used to train the model. After training, the model was used to further predict and obtain high-resolution mapping results of urban vegetation.

[0020] To optimize the above technical solution, the specific measures also include:

[0021] Furthermore, in step S1, the resampling operation employs the nearest neighbor method.

[0022] Furthermore, in step S2, the spatial registration method is completed using the Georeferencing tool built into ArcGIS software, employing a second-order polynomial transformation method.

[0023] Furthermore, in step S2, the number of selected control points is greater than 20.

[0024] Further, in step S5, the threshold The threshold is set to 0.3. Set it to 0.2.

[0025] A computer-readable storage medium storing a computer program that causes a computer to perform the high-resolution mapping method for urban vegetation as described in any of the preceding claims.

[0026] An electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the high-resolution mapping method for urban vegetation as described in any of the preceding claims.

[0027] The beneficial effects of this invention are:

[0028] 1) A cross-scale vegetation index, CSVI, was creatively proposed. This index can achieve higher accuracy in extracting urban vegetation than any other RGB vegetation index. Its effect is particularly evident in the suppression of background land cover (such as water bodies).

[0029] 2) Based on the proposed CSVI, this application also proposes an index-guided automatic semantic segmentation framework that enables high-precision interpretation of urban vegetation without any manual sample annotation. Based on the method of this application, high-resolution urban vegetation can be automatically and accurately extracted without any sample annotation, providing an important methodological reference for large-scale high-resolution mapping applications of urban vegetation. Attached Figure Description

[0030] Figure 1 This is a flowchart of the high-resolution automatic mapping method for urban vegetation guided by cross-scale vegetation indices, as described in this application.

[0031] Figure 2 This application embodiment shows the location of the study area used for method verification and its corresponding RGB image, Sentinel-2 image, and real label sample.

[0032] Figure 3 This is a comparison chart of the cross-scale vegetation index CSVI proposed in this application with other various RGB vegetation indices.

[0033] Figure 4 This is a comparison of the mapping accuracy of the cross-scale vegetation index CSVI proposed in this application with other various RGB vegetation indices at different thresholds.

[0034] Figure 5 This is a comparison chart of the proposed method with the CSVI threshold segmentation results and the fully supervised semantic segmentation results. Detailed Implementation

[0035] The invention will now be described in further detail with reference to the accompanying drawings.

[0036] Figure 1This invention presents a flowchart of a high-resolution automatic urban vegetation mapping method guided by a cross-scale vegetation index. It creatively combines spectral index and semantic segmentation methods, proposing a spectral index-guided automatic semantic segmentation framework for automatically achieving high-precision urban vegetation mapping from high-resolution RGB imagery. First, a novel cross-scale vegetation index (CSVI) is proposed, which combines the near-infrared band of RGB imagery and the red band of Sentinel-2 imagery to provide an initial spatial distribution map of urban vegetation. Second, threshold segmentation is performed on the CSVI image to automatically generate urban and non-urban samples, which are then used to train the semantic segmentation model. Therefore, the entire workflow eliminates the need for manual sample labeling, offering significant potential for large-scale, high-resolution urban vegetation mapping. The method includes the following steps:

[0037] Step 1: For two Sentinel-2 L2A and high-resolution RGB images covering the same area and acquired around the same time, the Sentinel-2 L2A image is first resampled to match the spatial resolution of the high-resolution RGB image. The Sentinel-2 L2A image used in this invention was downloaded from the European Space Agency's official website https: / / scihub.copernicus.eu / dhus / # / home. This product is a surface reflectance product that has undergone official radiometric calibration and atmospheric correction. The near-infrared spatial resolution required for this invention is 10 meters. The RGB image used consists of three bands: red, green, and blue, with a spatial resolution of 0.5 meters.

[0038] Step 2: Select several control points and perform spatial registration between the Sentinel-2 L2A image and the high-resolution RGB image to make the two images coincide in spatial position. This operation is completed by the Georeferencing tool in ArcGIS software.

[0039] Step 3: Using the surface reflectance of the Sentinel-2 L2A image as a benchmark, perform relative radiometric correction on the red band of the high-resolution RGB image to adjust the reflectance of the original RGB image's red band. Correction to This minimizes the difference in surface reflectance between the two images.

[0040] Step 4, calculate the cross-scale vegetation index (CSVI), the expression of which is:

[0041]

[0042] in, , These represent the surface reflectance of the green and blue bands of the high-resolution RGB image, respectively. This represents the surface reflectance in the near-infrared band of Sentinel-2 L2A imagery. This represents the surface reflectance of the red band in a high-resolution RGB image after relative radiometric correction. The effect is manifested in two aspects. First, removing the influence of shadows. In urban scenes, tall buildings easily generate shadows, hindering the extraction of urban vegetation. To remove the influence of shadows, a normalized index is constructed to remove them, utilizing the high reflectivity of shadows in the blue light band and the low reflectivity in the green light band [see H. Fang et al., "Detection of Building Shadow in Remote Sensing Imagery of Urban Areas With Fine Spatial Resolution Based on Saturation and Near-Infrared Information," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 2695-2706, Aug. 2019.]. Second, vegetation typically has high reflectivity in the green band and low reflectivity in the red and blue bands; therefore, a normalized index is used. This can further enhance the ability to identify urban vegetation.

[0043] Step 5, set the threshold , Pseudo-samples of urban and non-urban vegetation are automatically generated from CSVI images by threshold segmentation. The calculation method for pseudo-sample S can be expressed as follows:

[0044]

[0045] in, i and j They respectively represent the positions located at the th j Line 1 j Column of pixels, , The values ​​are set to 0.3 and 0.2 respectively. A label value of 1 represents urban vegetation, 0 represents non-urban vegetation, and 2 represents an uncertain category, which is ignored during model training.

[0046] Step 6: Train the DeepLab v3+ semantic segmentation model using automatically generated samples. The trained model can further predict high-resolution mapping results of urban vegetation. For a detailed introduction to the DeepLab v3+ model, see LCChen, et al., “Encoder-decoder with atrous separable convolution for semantic image segmentation,” Lect. Notes Comput. Sci. (including Subser. Lect. NotesArtif. Intell. Lect. Notes Bioinformatics), vol. 11211 LNCS, pp. 833–851, 2018. Specific examples are described below.

[0047] The study area selected in this application is located in Qixia District and Jiangning District of Nanjing City, Jiangsu Province. The location and coverage of the study area are detailed in [link to relevant documentation]. Figure 2 The surface labels are generated through visual interpretation of high-resolution imagery and manual annotation via field surveys, and verified by multiple professionals to ensure accuracy. The method of this invention mainly comprises two parts: the first part obtains an initial urban vegetation spatial distribution map using the proposed CSVI; the second part automatically generates high-confidence positive and negative samples based on this initial distribution map through threshold segmentation, which are then used to train a semantic segmentation model to obtain the final high-resolution urban vegetation mapping result. Figure 3 , Figure 4 It is evident that, compared to the other five RGB vegetation indices, the CSVI index proposed in this invention achieves significantly improved mapping accuracy, especially in water body suppression: CSVI can effectively distinguish urban vegetation from water bodies, while other indices struggle to differentiate them. The main reason is that water bodies and urban vegetation exhibit very similar spectral characteristics in the visible light band: high reflectance in the green band and low reflectance in the red and blue bands. Therefore, vegetation indices based solely on the three RGB bands are insufficient to differentiate between the two types of land cover. CSVI, however, is derived from the normalized difference between the near-infrared band of Sentinel-2 imagery and the red band of RGB imagery, thus easily distinguishing water bodies from urban vegetation. However, since CSVI combines a 10-meter resolution Sentinel-2 near-infrared band with a 0.5-meter resolution red band, although the resolution remains 0.5 meters, it inevitably creates a scale inconsistency problem at the edges of land cover features, manifesting as a "jagged effect," such as... Figure 3 The white box in the middle (h) indicates this. Therefore, CSVI alone is insufficient for achieving high-precision mapping of urban vegetation.

[0048] Figure 5 The comparison chart shows the proposed method with CSVI threshold segmentation results and fully supervised semantic segmentation results. It can be seen that: 1) CSVI threshold segmentation results exhibit a significant "jagged edge effect," but the proposed method (CSVI + semantic segmentation) effectively eliminates this problem, such as... Figure 5 As shown in the white box. Therefore, it can be confirmed that the CSVI-guided semantic segmentation method can effectively suppress the "sawtooth effect" compared to the original CSVI index, thereby further improving the mapping accuracy of urban vegetation. 2) Compared with fully supervised methods, the method proposed in this invention can achieve similar urban vegetation mapping accuracy.

[0049] The cross-scale vegetation index CSVI and its guided automatic semantic segmentation technology used in this invention are algorithmic innovations. The core idea of ​​the algorithm is to use the effective medium-resolution near-infrared band for vegetation identification to assist high-resolution RGB images lacking this band in improving the ability to identify urban vegetation.

[0050] It should be noted that the terms such as "upper", "lower", "left", "right", "front", and "back" used in the invention are only for clarity of description and are not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

[0051] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A high-resolution urban vegetation mapping method guided by cross-scale vegetation indices, characterized in that, Includes the following steps: S1: For the same area, acquire Sentinel-2 L2A images and high-resolution RGB images at similar times, and resample the Sentinel-2 L2A images to make their spatial resolution consistent with that of the high-resolution RGB images. S2: For the two images in step S1, select a number of pixels as control points and perform spatial registration operation between the Sentinel-2 L2A image and the high-resolution RGB image to make the two images coincide in spatial position. S3: Using the surface reflectance of the Sentinel-2 L2A image as a benchmark, perform relative radiometric correction on the red band of the high-resolution RGB image to adjust the reflectance of the original RGB image's red band. RGB Corrected to red RGB This reduces the difference in surface reflectance between the two images; S4: Based on the two images processed in steps S1-S3, calculate the cross-scale vegetation index (CSVI) for each location in the region. (i,j) CSVI images were obtained; the cross-scale vegetation index (CSVI) at each location was also obtained. (i,j) The calculation formula is: In the formula, CSVI (i,j) Represents the cross-scale vegetation index in the i-th row and j-th column of a CSVI image; green RGB(i,j) blue RGB(i,j) Represent the surface reflectance of the green and blue bands in the i-th row and j-th column of the high-resolution RGB image, respectively; NIR S2(i,j) Red′ represents the surface reflectance in the near-infrared band of the i-th row and j-th column of the Sentinel-2L2A image; RGB(i,j) This represents the surface reflectance of the red band in the i-th row and j-th column of the high-resolution RGB image after relative radiometric correction; the value n is used to expand the range of the cross-scale vegetation index. S5: Set threshold T fg T bg Pseudo-samples S of urban and non-urban vegetation are generated by thresholding CSVI images. The specific method is as follows: In the formula, S (i,j) This represents the sample label of the pixel in the i-th row and j-th column of the CSVI image. When the label value is 0, it means that the pixel is non-urban vegetation. When the label value is 1, it means that the pixel is urban vegetation. When the label value is 2, it means that the pixel is of uncertain type and is ignored. S6: Use the pseudo-samples S generated in step S5 to train the semantic segmentation model DeepLab v3+. After the model is trained, it is used to further predict and obtain high-resolution mapping results of urban vegetation.

2. The urban vegetation high-resolution mapping method based on cross-scale vegetation indices guided by claim 1, characterized in that, In step S1, the resampling operation uses the nearest neighbor method.

3. The high-resolution urban vegetation mapping method based on cross-scale vegetation indices guided by claim 1, characterized in that, In step S2, the spatial registration method is completed using the Georeferencing tool built into ArcGIS software, employing a second-order polynomial transformation method.

4. The urban vegetation high-resolution mapping method based on cross-scale vegetation indices guided by claim 1, characterized in that, In step S2, the number of selected control points is greater than 20.

5. The high-resolution urban vegetation mapping method based on cross-scale vegetation indices guided by claim 1, characterized in that, In step S5, the threshold T fg The threshold T is set to 0.

3. bg Set it to 0.

2.

6. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to execute the high-resolution mapping method for urban vegetation as described in any one of claims 1-5.

7. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the high-resolution mapping method for urban vegetation as described in any one of claims 1-5.

Images