Building height extraction method based on satellite-borne laser radar and high-resolution remote sensing image

Abstract

The invention discloses a building height extraction method based on a satellite-borne laser radar and a high-resolution remote sensing image. The building height extraction method comprises the following steps: acquiring satellite-borne laser radar ATLAS / ICESat-2L2A global positioning photon product data of an urban area; carrying out preprocessing operation on the obtained data to obtain high-quality spaceborne laser radar photons in the region, and further extracting ICESat-2 photon data with higher quality; the method comprises the following steps: acquiring high-resolution remote sensing image data of an urban area, and performing cutting and data labeling on a high-resolution remote sensing image; constructing and training a deep learning model for building roof extraction and a deep learning model for building side facade extraction; the model predicts a building roof extraction result and a building side facade extraction result; correcting the building roof based on the building side vertical surface to obtain a building base extraction result; and carrying out vectorization processing on a building base extraction result, carrying out vector intersection on the result and ICESat-2 photons, identifying building points based on a spatial analysis method, and carrying out building height precision evaluation. According to the invention, the height of the building can be extracted, and a scientific basis is provided for urban planning and ecological environment research.

Description

Technical Field

[0001] The present invention relates to the technical field of building height extraction, and particularly to a method for extracting building height based on spaceborne lidar and high-resolution remote sensing images. Background Art

[0002] Building height information is the core data support in fields such as urban refined management, ecological environment assessment, and disaster emergency response. In the urban planning scenario, building height information provides a quantitative basis for spatial layout optimization, building density regulation, and sunlight and ventilation simulation; in ecological environment research, it is a key input for urban heat island effect analysis and the construction of atmospheric pollutant diffusion models; in the field of disaster emergency, it directly affects the accuracy of fire rescue plan formulation, high-rise building escape simulation, and flood inundation analysis. However, high-precision building height data is very scarce, and traditional manual measurement methods are costly and inefficient, unable to meet the dynamic monitoring needs of large-scale urban areas. In recent years, the progress of remote sensing technology has provided new possibilities for building height extraction. By efficiently extracting and analyzing remote sensing data of urban areas, large-scale building height extraction can be achieved. Current methods mainly include estimating building height using multi-source remote sensing image fusion and estimating building height using spaceborne lidar.

[0003] The literature "Wu, W.-B., Ma, J., Banzhaf, E., Meadows, M.E., Yu, Z.-W., Guo, F.-X., Sengupta, D., Cai, X.-X., & Zhao, B. (2023). A first Chinese building height estimate at 10m resolution (CNBH-10m) using multi-source earth observations and machine learning. Remote Sensing of Environment, 291, 113578. https: / / doi.org / 10.1016 / j.rse.2023.113578" made the first 10-meter resolution Chinese building height map based on multi-source remote sensing images, including data such as Sentinel-1, Sentinel-2, and PALSAR. The optical data characteristics of Sentinel-1 and Sentinel-2 played an important role in it. However, for the estimation of the height of a single building, the accuracy may not meet the application requirements that need higher accuracy.

[0004] The literature "Lao, J., Wang, C., Zhu, X., Xi, X., Nie, S., Wang, J., Cheng, F., & Zhou, G. (2021). Retrieving building height in urban areas using ICESat-2 photon-counting LiDAR data. International Journal of Applied Earth Observation and Geoinformation, 104, 102596. https: / / doi.org / 10.1016 / j.jag.2021.102596" extracted building heights in Beijing, China based on the spaceborne lidar ATLAS / ICESat-2 L2A global positioning photon product data. However, this method has certain limitations in practical applications: First, its algorithm mainly relies on the feature separation of photon point clouds and lacks adaptability to complex urban terrains, resulting in poor applicability in different regions. Second, due to the complex process of point cloud data processing and long computing time, it is difficult to meet the extraction requirements of large areas and high efficiency.

[0005] The patent "Method for automatically extracting building height based on satellite images", with the publication number CN114120140A, the method includes: obtaining satellite images, preprocessing the obtained original images, processing the preprocessed and corrected images into coordinate-bearing image maps, geometrically correcting, equalizing light and color, mosaicking and cropping the images with coordinates that have been processed, and finally obtaining an orthophoto image result map; extracting building shadows based on the object-oriented classification method based on rule information, the algorithm classification method of decision trees, the normalized difference vegetation index (NDVI) analysis method, the normalized difference water index (NDWI) analysis method, and the shadow spectral characteristics; calculating the building shadow length by calculating the pixel average value; constructing a building height inversion model based on the calculated shadow length, combined with the azimuth angle, altitude angle of the sun and the satellite, and the location of the building, and calculating the building height. In this method, a building height inversion model is constructed by combining the azimuth angle, altitude angle of the sun and the satellite, and the location of the building. However, this model does not fully consider that satellite images may be obtained at different times, resulting in time-varying differences in the azimuth angle and altitude angle of the sun and the satellite, which may affect the accuracy of height inversion.

[0006] Patent "A Method for Automatically Extracting Building Heights Based on Stereo Satellite Images". Its publication number is CN106871864A. The method includes: obtaining the original satellite stereo image pair, SRTM data, and DOM data of the target building; preprocessing the original satellite stereo image pair, then performing relative orientation and absolute orientation in sequence to generate a coregistered image for extracting DSM data; extracting the initial DSM data of the target building, referring to the road layer, landform layer of the DLG data, and the DOM data of the target building to obtain qualified checkpoints; obtaining DEM data with the elevation information of the target building filtered out; integrating the data; performing stereo inspection and correction on the vertex elevation value and foundation elevation value of the target building; combining the house layer of the target building in the DOM and DLG, and subtracting the building vertex elevation value from the building site foundation elevation value to obtain the building elevation information. This method has certain limitations in practical applications because many cities face significant difficulties in obtaining digital surface model (DSM) data. Summary of the Invention

[0007] Object of the Invention: The present invention provides a method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images, which can extract building heights and provide a scientific basis for urban planning and ecological environment research.

[0008] Technical Solution: A method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images according to the present invention includes the following steps:

[0009] Step 1: Obtain spaceborne lidar ATLAS / ICESat-2 L2A global positioning photon product data of the urban area;

[0010] Step 2: Perform preprocessing operations on the obtained data to obtain high-quality spaceborne lidar photons in the area, and further extract higher-quality ICESat-2 photon data;

[0011] Step 3: Obtain high-resolution remote sensing image data of the urban area, and perform cropping and data annotation on the high-resolution remote sensing images;

[0012] Step 4: Construct and train a deep learning model for extracting building roofs and a deep learning model for extracting building side facades;

[0013] Step 5: The model predicts the building roof extraction result and the building side facade extraction result;

[0014] Step 6: Correct the building roof based on the building side facade to obtain the building base extraction result;

[0015] Step 7: Vectorize the building base extraction results and perform vector intersection with ICESat-2 photons. Identify building points based on spatial analysis methods and evaluate the building height accuracy.

[0016] Further, in Step 1, combining the high-definition image and the study area range, download the L2A global geolocated photon data of the spaceborne lidar ATLAS / ICESat-2 passing through the urban area from the NASA Ice Data Center. The data is published in the.hdf5 format. The information at the photon level is stored in heights, and the relevant information such as geographical location is stored in geolocation.

[0017] Further, in Step 2, perform preprocessing operations on the obtained data to obtain high-quality spaceborne lidar photons in the region, which specifically includes the following steps:

[0018] Step 21: Read lon_ph, lat_ph, h_ph, and dist_ph_along in heights to screen the ICESat-2 photons within the urban area and obtain their heights and the distances from the segment start within the segment.

[0019] Step 22: Read signal_conf_ph in heights to screen out the ICESat-2 photons with a confidence level greater than 3.

[0020] Step 23: Read segment_id, ph_index_beg, segment_dist_x, and segment_ph_cn in geolocation to calculate the along-track length of each ICESat-2 photon.

[0021] Further, in Step 2, assign segment_id to each photon using the number of photons segment_ph_cn in each segment and the first photon number ph_index_beg in each segment; calculate the along-track length of each photon using the segment id of each photon, the length of each segment segment_dist_x, and the distance from the segment start within the segment of each photon; construct a KD-tree spatial index with the along-track length as the abscissa and the height as the ordinate for the ICESat-2 photons; perform clustering analysis through the DBSCAN algorithm to further extract higher-quality ICESat-2 photon data. The DBSCAN algorithm defines clusters through two parameters: eps (radius) and min_samples (minimum number of samples).

[0022] Further, in step 3, high-resolution remote sensing image data of the urban area is obtained from Google Satellite High-Definition Map according to the longitude and latitude range of the city. Labelme is used to crop and annotate the high-resolution remote sensing images. The images are cropped into 384px * 384px, and the building roofs and side facades are respectively annotated.

[0023] Further, in step 4, the specific steps for constructing and training the deep learning model for building roof extraction and the deep learning model for building side facade extraction are as follows:

[0024] Step 41: Adopt the open-source DeeplabV3+ deep learning model and design a loss function suitable for large-scale roof and side facade extraction; the specific loss function is as follows:

[0025] L(p i ,p i ')=α*L dice (p i ,p i ')+β*L bce (p i ,p i ')

[0026] Among them, pi is the predicted value of the i-th sample, p i ' is the true value of the i-th sample, L dice (·) is the dice loss function, L bce (·) is the cross-entropy loss function, and α and β are the weight coefficients of the loss function;

[0027] Step 42: Use the cropped and annotated roof dataset and side facade dataset to train the model respectively. The dataset is divided into a training set, a validation set, and a test set, accounting for 70%, 20%, and 10% of the total data respectively. Pictures without building content are mixed in each category to reduce the class imbalance problem and improve the robustness of the model.

[0028] Further, in step 5, the specific steps for the model to predict the building roof extraction result and the building side facade extraction result are as follows:

[0029] Step 51: Determine the small patch images of the size of patchsize that need to be cut from the large remote sensing image during inference according to the patchsize during training;

[0030] Step 52: Divide the width of the large remote sensing image by the patchsize to obtain a remainder. Subtract this remainder from the patchsize, and the resulting difference is the number of pixels to be filled in the horizontal (width) direction. At this time, fill the difference in pixels on the rightmost side of the original image; divide the height of the original image by the patchsize to obtain a remainder, subtract the remainder just calculated from the patchsize, and the resulting difference is the number of pixels to be filled in the vertical (height) direction. At this time, fill the difference in pixels at the bottommost side of the original image.

[0031] Step 53: Fill pixels around again. Assume that our patchsize is 1024 and the step size is 512. We need to fill 256 pixels (step size / 2 = 512 / 2 = 256) on each of the four sides of top, bottom, left, and right, so that we can cut the original image into integer parts with a size of 1024 * 1024. The moving step size is 256.

[0032] Step 54: Put these patches into the result image obtained after model inference. Take the central 512 * 512 area of each patch in order for stitching, and we can completely obtain the size after the first filling, that is, the size of the second-step image. Then crop the newly added part on the right side of the first step and the newly added part at the bottom to obtain the result image of the size of the original remote sensing large image.

[0033] Further, in step 6, the specific steps for correcting the building roof based on the building facade to obtain the building base extraction result are as follows:

[0034] Step 61: Load the black-and-white binary images of the building roof and facade, and use the morphological erosion algorithm to separate the connected facade contours. The expression of the morphological erosion algorithm is:

[0035]

[0036] where A is the image to be eroded, B is the erosion structuring element, and each element takes a value of 0 or 1.

[0037] Step 62: Filter out the building roofs and facades with too small areas, and extract the contours of the roofs and facades.

[0038] Step 63: Traverse each facade and match the corresponding roof for each facade. First, detect whether there is a roof contour overlapping with the facade. Prioritize matching the overlapping facade and roof contours. If there is more than one roof overlapping with the facade, compare the overlapping areas and prioritize matching the roof with the larger overlapping area. For the facades that do not overlap with the roof contours, set a search radius and find the roof contour with the largest area within the search radius.

[0039] Step 64: For the paired roof and side elevation that match, calculate the distance and direction that the roof should move; perform rectangular fitting on the successfully matched side elevation and roof, calculate the center of the roof contour, find the two sides of the rectangular fitting of the side elevation that are farthest from the center of the roof contour, calculate the vectors from the center of the roof contour to the midpoints of the two sides of the farthest rectangular fitting of the side elevation, and move the roof contour according to the vectors.

[0040] Step 65: Create a blank base map and draw the moved roof to obtain the building base.

[0041] Furthermore, in step 7, the vectorization process of the building base extraction result and the vector intersection with ICESat-2 photons specifically include the following steps:

[0042] Step 71: Use the GDAL library to read the projection information of the original remote sensing large image and assign it to the binary black and white image of the building base.

[0043] Step 72: Import the binary black and white image of the building base into ArcGIS Pro, and use the raster reclassification function to only retain the contour of the building base.

[0044] Step 73: Use the raster to polygon function in ArcGIS Pro to convert the building base map into a vector polygon and save it as a shp file.

[0045] Step 74: Use python code to batch process the intersection operation between ICESat-2 photons and the building base shp.

[0046] Furthermore, in step 7, identify building points based on the spatial analysis method. Take the photon with the highest height within the building base shp after the intersection of ICESat-2 photons and the building base shp as the building point; extract the ICESat-2 photons within 50 meters of the building point, calculate the difference between the maximum value and the minimum value of these points. If it is greater than 4m, proceed to the next step. If it is less than 4m, proceed to the next building point. Generate a window with a height of 2m and traverse the photons within 50m at a step of 1m. If there are no photons in at least one window, consider the minimum value photon as the ground point; calculate the building height by subtracting the average height of the photons within 1m above the ground point from the average height of the photons within 1m below the building point; perform accuracy evaluation on the extracted building height, and draw the scatter plot and error histogram of the building height.

[0047] Beneficial effects: Compared with the prior art, the present invention has the following remarkable advantages: (1) The spaceborne lidar ICESat-2 photon data and Google high-resolution remote sensing images adopted by the present invention are both publicly available data resources, without relying on commercial remote sensing platforms or expensive ground measurement means, having a good foundation for popularization and application, with clear and traceable data sources, being conducive to the repeated verification of results and the construction of standardization, and ensuring the reliability and scalability of the technology in scientific research and practical applications; (2) Using deep learning algorithms to extract the roofs and side facades of buildings, and correcting the positions of building roofs through a correction algorithm, reducing the offset of building roofs in non-orthorectified remote sensing images, making the positions of buildings more accurate; (3) Adopting the method of combining spaceborne lidar and high-resolution remote sensing images to extract the spaceborne lidar photons falling within the building range, realizing the precise screening and spatial constraint of photon data, and being more time-saving and not affected by terrain compared with traditional algorithms; (4) The building heights extracted by the present invention have high accuracy and low cost, and can provide data support for urban planning and environmental research of relevant departments. Brief Description of the Drawings

[0048] Figure 1 It is a schematic flow chart of the method of the present invention.

[0049] Figure 2 It is a scatter plot of the building heights extracted by the present invention.

[0050] Figure 3 It is an error histogram of the building heights extracted by the present invention. Detailed Embodiment

[0051] As Figure 1 shown, a method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images includes the following steps:

[0052] Step 1, obtain the spaceborne lidar ATLAS / ICESat-2 L2A global positioning photon product data of the urban area.

[0053] Combined with the high-definition image and the research area range, download the L2A global geolocation photon data of the spaceborne lidar ATLAS / ICESat-2 passing through the urban area in the NASA Cryosphere Data Center. The data is published in the.hdf5 format, and the information at the photon level is stored in heights, and the relevant information such as geographical location is stored in geolocation.

[0054] The present invention first selects two urban areas of Nanjing and Hong Kong.

[0055] Step 2, perform cropping and quality screening preprocessing operations on the obtained data, which specifically include the following steps:

[0056] Step 21: Read lon_ph, lat_ph, h_ph, and dist_ph_along in heights to screen ICESat-2 photons within the city range and obtain their heights and distances from the start of the segment;

[0057] Step 22: Read signal_conf_ph in heights to screen out ICESat-2 photons with a confidence level greater than 3;

[0058] Step 23: Read segment_id, ph_index_beg, segment_dist_x, and segment_ph_cn in geolocation to calculate the along-track length of each ICESat-2 photon.

[0059] Construct a KD-tree spatial index for ICESat-2 photons with the along-track length as the abscissa and the height as the ordinate.

[0060] Perform clustering analysis through the DBSCAN algorithm to further extract higher-quality ICESat-2 photon data. The DBSCAN algorithm defines clusters through two parameters: eps (radius) and min_samples (minimum number of samples).

[0061] Input the above five features and use a random forest model for the classification and extraction of planted forests and natural forests. Divide the dataset into a training set, a validation set, and a test set, which account for 70%, 20%, and 10% of the total data respectively. Use the random forest model for classification training, set 150 decision trees to improve the robustness of the classification, and set a random seed to ensure the repeatability of the results.

[0062] Step 3: Obtain high-resolution remote sensing image data of the urban area from Google Satellite High-Definition Maps according to the longitude and latitude range of the city.

[0063] Use labelme to crop and annotate the high-resolution remote sensing image. Crop the image to 384px * 384px and annotate the building rooftops and side facades respectively.

[0064] Step 4: The specific steps for constructing and training a deep learning model for building rooftop extraction and a deep learning model for building side facade extraction are as follows:

[0065] Step 41: Adopt the open-source DeeplabV3+ deep learning model and design a loss function suitable for large-scale rooftop and side facade extraction. The loss function suitable for large-scale rooftop and side facade extraction is as follows:

[0066] L(p i ,p i') = α * L dice (p i ,p i ') + β * L bce (p i ,p i ')

[0067] Among them, pi is the predicted value of the i-th sample, p i ' is the true value of the i-th sample, L dice (·) is the dice loss function, L bce (·) is the cross-entropy loss function, and α and β are the weight coefficients of the loss function.

[0068] Step 42: Use the cropped and labeled roof dataset and side elevation dataset to train the model respectively. Divide the dataset into a training set, a validation set, and a test set, accounting for 70%, 20%, and 10% of the total data respectively, and mix pictures without building content into each category to reduce the class imbalance problem and improve the robustness of the model.

[0069] Step 5: Use dilation prediction to predict the results of the large remote sensing image. The specific steps are as follows:

[0070] Step 51: Determine the small block images of the large remote sensing image that need to be cut into the size of patchsize during inference according to the patchsize during training;

[0071] Step 52: Divide the width of the large remote sensing image by patchsize to get a remainder, subtract this remainder from patchsize, and the resulting difference is the number of pixels to be filled in the horizontal (width) direction. At this time, fill the difference number of pixels on the rightmost side of the original image. Similarly, divide the height of the original image by patchsize to get a remainder, subtract the remainder just calculated from patchsize, and the resulting difference is the number of pixels to be filled in the vertical (height) direction. At this time, fill the difference number of pixels at the bottommost side of the original image.

[0072] Step 53: Fill pixels around again. Suppose our patchsize is 1024 and the stride is 512. Then we need to fill 256 pixels (stride / 2 = 512 / 2 = 256) on each of the four sides of the top, bottom, left, and right. In this way, the original image can be cut into integer pieces with a size of 1024 * 1024, and the moving stride is 256.

[0073] Step 54: Place these patches into the result image obtained after model inference. Take the central 512*512 region of each patch in sequence for stitching, and the size after the first filling can be obtained completely, which is the size of the second-step image. Then, crop the newly added part on the right in the first step and the newly added part below to obtain the result image with the size of the original remote sensing large image.

[0074] Step 6: The specific steps for rectifying the building roof based on the building facade to obtain the building base extraction result are as follows:

[0075] Step 61: Load the black-and-white binary images of the building roof and facade, and use the morphological erosion algorithm to separate the connected facade contours. The expression of the morphological erosion algorithm is:

[0076]

[0077] Among them, A is the image to be eroded, B is the structural element for erosion, and each element takes a value of 0 or 1.

[0078] Step 62: Filter out the building roofs and facades with too small areas, and extract the contours of the roofs and facades;

[0079] Step 63: Traverse each facade and match the corresponding roof for each facade; first, detect whether there is a roof contour overlapping with the facade, and give priority to matching the overlapping facade and roof contours. If there is more than one roof overlapping with the facade, compare the overlapping areas and give priority to matching the roof with the larger overlapping area. Secondly, for the facades that do not overlap with the roof contours, set a search radius to find the roof contour with the largest area within the search radius.

[0080] Step 64: For the paired roofs and facades, calculate the distance and direction that the roof should move;

[0081] Step 65: Create a blank base map and draw the moved roof to obtain the building base.

[0082] Step 7: The detailed steps for vectorizing the building base extraction result and performing vector intersection with ICESat-2 photons are as follows:

[0083] Step 71: Use the GDAL library to read the projection information of the original remote sensing large image and assign it to the black-and-white binary image of the building base;

[0084] Step 72: Import the black-and-white binary image of the building base into ArcGIS Pro, and use the raster reclassification function to only retain the contour of the building base;

[0085] Step 73: Use the raster to polygon function in ArcGIS Pro to convert the building base map into vector polygons and save them as shp files;

[0086] Identify building points based on spatial analysis methods. Use Python code to batch process the intersection operation between ICESat-2 photons and the building base shp. Take the photon with the highest height within the building base shp after the intersection of ICESat-2 photons and the building base shp as the building point.

[0087] Extract ground points near the building points. Extract ICESat-2 photons within 50 meters of the building points; calculate the difference between the maximum and minimum values of these points. If it is greater than 4m, proceed to the next step. If it is less than 4m, move on to the next building point; generate a window with a height of 2m and traverse the photons within 50m at a step of 1m. If there are no photons in at least one window, consider the photon with the minimum value as the ground point.

[0088] Calculate the building height by subtracting the average height of photons within 1m above the ground point from the average height of photons within 1m below the building point.

[0089] Conduct accuracy assessment on the extracted building heights, and draw a scatter plot and an error histogram of the building heights.

Claims

1. A method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images, characterized in that, It includes the following steps: Step 1: Obtain the spaceborne lidar global positioning photon product data of the urban area; Step 2: Perform preprocessing operations on the obtained data to obtain high-quality spaceborne lidar photons within the region, and further extract higher-quality ICESat-2 photon data; Step 3: Obtain the high-resolution remote sensing image data of the urban area, and perform cropping and data annotation on the high-resolution remote sensing image; Step 4: Construct and train a deep learning model for building roof extraction and a deep learning model for building side elevation extraction; Step 5: The model predicts the building roof extraction result and the building side elevation extraction result; Step 6: Correct the building roof based on the building side elevation to obtain the building base extraction result; Step 7: Vectorize the building base extraction result and perform vector intersection with ICESat-2 photons. Based on the spatial analysis method, identify building points and evaluate the building height accuracy.

2. The method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images according to claim 1, characterized in that In Step 1, in combination with the high-definition image and the research area range, download the L2A global geolocation photon data of the spaceborne lidar ATLAS / ICESat-2 passing through the urban area from the NASA Cryosphere Data Center. The data is published in the.hdf5 format. The photon-level information is stored in heights, and the location-related information is stored in geolocation.

3. The method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images according to claim 1, wherein In Step 2, the preprocessing operations performed on the obtained data to obtain high-quality spaceborne lidar photons within the region specifically include the following steps: Step 21: Read lon_ph, lat_ph, h_ph, dist_ph_along in heights to screen out ICESat-2 photons within the urban area and obtain their heights and distances from the beginning of the segment within the segment; Step 22: Read signal_conf_ph in heights to screen out ICESat-2 photons with a confidence level greater than 3; Step 23: Read segment_id, ph_index_beg, segment_dist_x, segment_ph_cn in geolocation to calculate the along-track length of each ICESat-2 photon.

4. The method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images according to claim 1, characterized in that, In Step 2, use the number of photons segment_ph_cn in each segment and the first photon number ph_index_beg in each segment to assign segment_id to each photon; use the segment id of each photon, the length of each segment segment_dist_x, and the distance from the beginning of the segment within the segment dist_ph_along of each photon to calculate the along-track length of each photon; construct a KT-tree spatial index with the along-track length as the abscissa and the height as the ordinate for ICESat-2 photons; perform clustering analysis through the DBSCAN algorithm to further extract higher-quality ICESat-2 photon data. The DBSCAN algorithm defines clusters through two parameters: eps (radius) and min_samples (minimum number of samples).

5. The method for extracting building height based on spaceborne lidar and high-resolution remote sensing images according to claim 1, characterized in that, In step 3, high-resolution remote sensing image data of the urban area is obtained from Google Satellite High-Definition Map according to the latitude and longitude range of the city. Labelme is used to crop and annotate the high-resolution remote sensing image. The image is cropped into 384px * 384px, and the building rooftops and side facades are annotated respectively.

6. The method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images according to claim 1, wherein In step 4, constructing and training the deep learning models for building rooftop extraction and building side facade extraction specifically includes the following steps: Step 41: Adopt the open-source DeeplabV3+ deep learning model and design a loss function suitable for large-scale rooftop and side facade extraction. The loss function is specifically as follows: L(p i ,p i ') = α * L dice (p i ,p i ') + β * L bce (p i ,p i ') where, pi is the predicted value of the i-th sample, and p i ' is the true value of the i-th sample, and L dice (·) is the dice loss function, and L bce (·) is the cross-entropy loss function, and α and β are the weight coefficients of the loss function; Step 42: Use the cropped and annotated rooftop dataset and side facade dataset to train the model respectively. The dataset is divided into a training set, a validation set, and a test set, accounting for 70%, 20%, and 10% of the total data respectively, and pictures without building content are mixed in each category.

7. The method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images according to claim 1, characterized in that, In step 5, the model predicts the building rooftop extraction result and the building side facade extraction result specifically includes the following steps: Step 51: According to the patchsize during training, determine that the large remote sensing image needs to be cut into small images of patchsize size during inference. Step 52: Divide the width of the large remote sensing image by patchsize to get a remainder. Subtract this remainder from patchsize, and the resulting difference is the number of pixels to be filled in the horizontal (width) direction. At this time, fill the difference number of pixels on the rightmost side of the original image. Divide the height of the original image by patchsize to get a remainder. Subtract the remainder just calculated from patchsize, and the resulting difference is the number of pixels to be filled in the vertical (height) direction. At this time, fill the difference number of pixels at the bottommost side of the original image. Step 53: Fill pixels around again. Assume our patchsize is 1024 and the step size is 512. We need to fill 256 pixels (step size / 2 = 512 / 2 = 256) on each of the four sides of the top, bottom, left, and right, so that the original image can be cut into integer pieces of 1024 * 1024 size, and the moving step size is 256. Step 54: Put these patches into the model for inference, and take the central 512 * 512 area of each in order for stitching. Then we can completely get the size after the first filling, that is, the size of the second step image. Then crop the newly added part on the right side of the first step and the newly added part at the bottom to get the result image of the original large remote sensing image size.

8. The method for extracting building height based on spaceborne lidar and high-resolution remote sensing images according to claim 1, characterized in that In step 6, correcting the building rooftop based on the building side facade to obtain the building base extraction result specifically includes the following steps: Step 61: Load the black and white binary images of the building rooftop and side facade, and separate the connected side facade contours through the morphological erosion algorithm. The expression of the morphological erosion algorithm is: Among them, A is the image to be eroded, B is the erosion structure element, and each element takes a value of 0 or 1; Step 62: Filter out the building rooftops and side facades with too small areas, and extract the contours of the rooftops and side facades. Step 63: Traverse each side elevation and match a corresponding roof to it. First, detect if there is a roof contour overlapping with the side elevation. Prioritize matching the overlapping side elevation and roof contour. If there is more than one roof overlapping with the side elevation, compare the overlapping areas and prioritize matching the roof with the larger overlapping area. For a side elevation that does not overlap with a roof contour, set a search radius and find the roof contour with the largest area within the search radius. Step 64: For the paired roof and side elevation, calculate the distance and direction the roof should move. Perform rectangular fitting on the successfully matched side elevation and roof, calculate the center of the roof contour, find the two sides of the side elevation fitting rectangle that are farthest from the center of the roof contour, calculate the vector from the center of the roof contour to the midpoints of the two farthest sides of the side elevation fitting rectangle, and move the roof contour according to the vector. Step 65: Create a blank base map and draw the moved roof to obtain the building base.

9. The building height extraction method based on spaceborne lidar and high-resolution remote sensing images according to claim 1, wherein In Step 7, the vectorization process of the building base extraction result and the vector intersection with ICESat-2 photons specifically include the following steps: Step 71: Use the GDAL library to read the projection information of the original remote sensing large image and assign it to the binary black and white image of the building base. Step 72: Import the binary black and white image of the building base into ArcGIS Pro and use the raster reclassification function to only retain the contour of the building base. Step 73: Use the raster to polygon function in ArcGIS Pro to convert the building base map into a vector polygon and save it as a shp file. Step 74: Use python code to batch process the intersection operation between ICESat-2 photons and the building base shp.

10. The method for extracting building heights based on spaceborne lidar and high-resolution remote sensing images according to claim 1, characterized in that, In Step 7, based on the spatial analysis method to identify building points, take the photon with the highest height within the building base shp after the intersection of ICESat-2 photons and the building base shp as the building point. Extract the ICESat-2 photons within 50 meters of the building point, calculate the difference between the maximum value and the minimum value of these points. If it is greater than 4m, proceed to the next step; if it is less than 4m, proceed to the next building point. Generate a window with a height of 2m and traverse the photons within 50m at a step of 1m. If there are no photons in at least one window, regard the photon with the minimum value as the ground point. Calculate the building height by subtracting the average height of the photons within 1m above the ground point from the average height of the photons within 1m below the building point. Conduct accuracy evaluation on the extracted building height, and draw a scatter plot and an error histogram of the building height.

Images