Urban area height estimation method based on high resolution images and view transformation

Abstract

This application discloses a method for estimating the height of urban areas based on high-resolution images and viewpoint transformation, belonging to the field of remote sensing image processing. The method includes: generating tilted viewpoint height data at a reference time based on reference height data and a reference remote sensing image of the training area at a reference time; generating tilted viewpoint height data at a target time based on the reference height data and a target remote sensing image of the training area at a target time; combining the tilted viewpoint height data and remote sensing images corresponding to different times into corresponding sets of training samples to form a height estimation training sample set for the training area; and training the height estimation model based on the height estimation training sample set to obtain a trained height estimation model. This application can generate stable height representations even under non-nadir imagery, and supplements the model with reference height when there is no height data at the target time, thereby improving the generalization and reliability of the model under complex imaging conditions.

Description

Technical Field

[0001] This application belongs to the field of remote sensing image processing, specifically relating to a method for estimating the height of urban areas based on high-resolution images and viewpoint transformation, as well as a training method, apparatus, storage medium, device, and computer program product for the height estimation model. Background Technology

[0002] With the increasing demand for applications such as urban 3D modeling, surface change monitoring, and disaster assessment, obtaining spatial height information of urban areas based on remote sensing images has gradually become one of the important basic data for urban digital construction. High-resolution remote sensing images can provide rich texture and structural information of ground features, making it possible to infer the height distribution of buildings and the ground surface from images. Therefore, how to use remote sensing images to estimate the height of urban areas has become an important research direction in the field of remote sensing image processing.

[0003] In existing technologies, the height estimation of urban areas usually relies on a dataset consisting of orthophotos and corresponding height data. This relies on the characteristics of orthophotos, such as the shooting angle being close to the ground, the image geometric distortion being small, the occlusion of ground objects being weak, and the clear representation of building rooftops. Deep learning models are used to learn the mapping relationship between image texture and height, thereby achieving the prediction of urban area height under relatively ideal image conditions.

[0004] However, in real-world applications, sensors on satellites or aircraft platforms often struggle to maintain a perfectly vertical orientation when taking images. The resulting remote sensing images are generally off-nadir, with tilted viewpoints, significant geometric distortions, and complex occlusion of ground features. The spatial representation of buildings in these images differs significantly from that in orthophotos, making it difficult for height estimation models trained on orthophotos to maintain stable generalization capabilities under real-world image conditions. Summary of the Invention

[0005] This application aims to provide a method, apparatus, storage medium, device, and computer program product for estimating urban area height based on high-resolution images and viewpoint transformation, as well as a method for training height estimation models, which at least solves the problem of insufficient generalization ability of height estimation models under non-nadir remote sensing image conditions.

[0006] In a first aspect, embodiments of this application disclose a method for training a height estimation model, comprising: Based on the reference height data and reference remote sensing image of the training area at the reference time, tilt angle height data of the training area at the reference time is generated. Then, based on the reference height data and the target remote sensing image of the training area at the target time, tilt angle height data of the training area at the target time is generated. The reference height data is used to characterize the spatial height of each location in the training area. The tilt angle height data at different times have different shooting angles. Each pixel position of the tilt angle height data has a corresponding spatial height value at the corresponding shooting angle. The tilt angle height data corresponding to different times are combined with remote sensing images to form a set of training samples, and all the training samples are combined to form the height estimation training sample set of the training area. The height estimation model is trained based on the height estimation training sample set to obtain the trained height estimation model; the height estimation model is used to predict the spatial height estimate value corresponding to each pixel position in the remote sensing image at the input target time.

[0007] Secondly, embodiments of this application also disclose a method for estimating the height of urban areas based on high-resolution images and viewpoint transformation, including: The remote sensing image of the region to be estimated is input into the height estimation model trained according to the height estimation model training method described in the first aspect, so as to obtain the spatial height estimation value corresponding to each pixel position in the remote sensing image of the region to be estimated, so as to form a spatial height estimation map of the region to be estimated.

[0008] Thirdly, embodiments of this application also disclose a training apparatus for a height estimation model, comprising: The perspective transformation module is used to generate tilted perspective height data of the training area at the reference time based on the reference height data and the reference remote sensing image of the training area at the reference time, and to generate tilted perspective height data of the training area at the target time based on the reference height data and the target remote sensing image of the training area at the target time; the reference height data is used to characterize the spatial height of each position in the training area; the tilted perspective height data at different times have different shooting perspectives; each pixel position of the tilted perspective height data has a corresponding spatial height value at the corresponding shooting perspective. The training set building module is used to combine the tilt angle height data corresponding to different times with the remote sensing images respectively as a set of training samples, and to combine all the training samples together to form the height estimation training sample set of the training area. The model training module is used to train the height estimation model based on the height estimation training sample set to obtain the trained height estimation model; the height estimation model is used to predict the spatial height estimate value corresponding to each pixel position in the remote sensing image at the input target time.

[0009] Fourthly, embodiments of this application also disclose an urban area height estimation device based on high-resolution image viewpoint transformation, comprising: The image recognition module is used to input the remote sensing image of the region to be estimated into the height estimation model trained according to the height estimation model training method described in the first aspect, so as to obtain the spatial height estimation value corresponding to each pixel position in the remote sensing image of the region to be estimated, so as to form a spatial height estimation map of the region to be estimated.

[0010] Fifthly, embodiments of this application also disclose a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the first or second aspect.

[0011] In a sixth aspect, embodiments of this application also disclose an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, performs the steps as described in the first or second aspect.

[0012] In a seventh aspect, embodiments of this application also disclose a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps described in the first or second aspect.

[0013] In summary, in this embodiment, firstly, reference height data for the reference time is generated using both reference height data and reference remote sensing images available at the reference time. Then, using this reference height data as the height source, and combining it with the target remote sensing image available only at the target time, tilt angle height data for the target time is generated. This ensures that both the reference and target remote sensing images obtain height representations corresponding to their imaging methods, providing complete height input for subsequent training. Based on this, the tilt angle height data obtained at the reference and target times are combined with the corresponding remote sensing images to form a height estimation training sample set for training. This ensures that the training samples simultaneously possess image and height information across time dimensions and maintain continuity in spatial structure presentation, thus providing a stable data foundation for the model to establish cross-time image height correspondences. Subsequently, the height estimation model is trained using the aforementioned training sample set. During the learning process of image texture and height relationships, the model continuously encounters multi-time height representations extended from the reference height data, ultimately forming a unified height representation method for remote sensing images acquired at different times. This allows the model to output structurally reasonable height estimation results even when real height data is lacking at the target time. Therefore, the method based on the embodiments of this application possesses the ability to adapt to height representation deviations caused by imaging tilt when facing remote sensing images with non-nadir characteristics. Furthermore, it can generate stable height representations based on height information from a reference time, even when true height data is lacking at the target time, thereby improving the generalization performance of the height estimation model under non-nadir remote sensing image conditions. In addition, in the embodiments of this application, when a non-nadir remote sensing image is input into the trained height estimation model, the model generates an elevation image matching the imaging method of the image based on the correspondence between the image and the tilted viewpoint height data formed during the training phase. This ensures that the height estimation results remain stable in terms of spatial structure and height representation, which is beneficial for continuously utilizing multi-temporal remote sensing images to obtain consistent height information in applications such as urban 3D modeling and change monitoring. Attached Figure Description

[0014] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating the steps of a training method for a height estimation model provided in an embodiment of this application. Figure 2 This is a schematic diagram illustrating the principle of generating tilt angle height data according to an embodiment of this application; Figure 3 This is a flowchart illustrating the steps of another height estimation model training method provided in this application embodiment; Figure 4 This is a flowchart illustrating the steps of a method for estimating urban area height based on high-resolution images and viewpoint transformation, as provided in an embodiment of this application. Figure 5 This is a flowchart illustrating the steps of another method for estimating urban area height based on high-resolution images and viewpoint transformation provided in this application embodiment; Figure 6 This is a comparison chart of height estimation results under an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a training device for a height estimation model provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an urban area height estimation device based on high-resolution images and viewpoint transformation provided in an embodiment of this application; Figure 9 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0016] In existing technologies, urban area height estimation typically relies on a large-scale training set consisting of orthophotos and corresponding height data. However, in practical applications, the large-scale acquired remote sensing images are mostly oblique images with significant off-nadir characteristics; while height data is usually orthophoto projection. Because oblique images differ significantly from orthophoto projection height data in terms of ground feature geometry, building facade representation, shadow distribution, and height projection relationships, existing height estimation models trained on orthophotos lack adaptability when processing oblique images, often resulting in incomplete building outlines, missing facade heights, and unstable local height predictions, making it difficult to meet the practical needs of multi-temporal height estimation.

[0017] In this application, to address the insufficient adaptability between strabismus images and existing models, this application starts from the construction logic of the training data source. Focusing on the imaging characteristics and height expression patterns of strabismus images, it unifies and structures multi-temporal strabismus images and their height data, ensuring the alignability of the viewpoint representations of strabismus images and height data at different imaging times. By constructing a training sample set with a consistent representational basis, the height estimation model can stably learn the mapping relationship between texture, facade structure, and height data after viewpoint changes in strabismus images during the training phase, thus possessing reliable height estimation capabilities under multi-temporal conditions.

[0018] Based on the above overall design logic, such as Figure 1 The diagram illustrates a training method for a height estimation model provided in this application, which specifically includes the following steps: Step 101: Generate tilt angle height data of the training area at the reference time based on the reference height data and reference remote sensing image of the training area at the reference time, and generate tilt angle height data of the training area at the target time based on the reference height data and target remote sensing image of the training area at the target time.

[0019] Among them, the reference height data is used to characterize the spatial height of each location in the training area; the tilt angle height data at different times have different shooting angles; each pixel position of the tilt angle height data has a corresponding spatial height value under the corresponding shooting angle.

[0020] In some embodiments of this application, to ensure that remote sensing images acquired at different times possess a height source usable for training a height estimation model, it is necessary to combine reference height data at a reference time with a reference remote sensing image to generate tilt angle height data at the reference time. Then, based on the same reference height data, this data is combined with a target remote sensing image at the target time to generate tilt angle height data at the target time. Specifically, by utilizing the reference height data to provide spatial height values ​​and combining it with the imaging geometry of the remote sensing images at each time, the generated tilt angle height data can provide a spatial height representation for each pixel position under the corresponding shooting angle. Tilt angle height data is a type of height representation that combines height information with the imaging angle. Its height value originates from the reference height data and is matched with the spatial projection relationship of the remote sensing image during the generation process. By generating tilt angle height data at the reference time and the target time, remote sensing images at different times can all possess height information, providing a continuous and unified height input for subsequently constructing cross-time training samples.

[0021] like Figure 2As shown in a specific example, the training area's reference time has a normalized digital surface model (nDSM) as reference height data, and a high-resolution satellite fusion image stored in GeoTIFF raster format, containing red, green, and blue spectral bands, as a reference remote sensing image. The target time, however, only has a single target remote sensing image, also stored in GeoTIFF raster format. Combined with... Figure 2 The xyz coordinate system (where xy represents the ground plane and z represents the vertical direction) shows that the reference height data was acquired at a near-orthogonal angle. Its height representation is shown in the lower part of 2-b, with colors transitioning from dark to light corresponding to increasing height, providing the vertical height of the building's top. However, because the imaging angle is perpendicular to the z-axis, the building's sidewall structure is not shown in the image. In contrast, the remote sensing images at both the reference and target times were acquired at an oblique angle, as indicated by the blue dashed line in the upper part of 2-a. The imaging angle is not vertical along the z-axis but enters the scene at a certain angle. Therefore, in the lower part of 2-a, the building's sidewall region, marked by a blue box, can be seen in the acquired remote sensing image. These sidewall pixels correspond to the building's horizontal extension in the xy plane and its height variation in the z-direction in xyz space, structural features that only appear in images under oblique angle imaging conditions. When generating tilted-view height data, the vertical height value provided by the reference height data is used as a basis. Combined with the imaging geometry of the remote sensing image at the corresponding time, the reference height data is reprojected in the xyz coordinate system to a tilted view consistent with the image at that time, ensuring that the generated height data maintains spatial consistency with the tilted-view image. The upper part of 2-c uses blue dashed lines to illustrate the tilted view corresponding to the generated height data, while the lower part of 2-c shows the height distribution after spatial transformation. At the location of the building's side wall, the height distribution can be seen... Figure 2The blue box area in section -a represents the sidewall height, with color variations reflecting the height change from low to high. At the reference time, the spatial projection relationship between the reference height data and the reference remote sensing image can be matched, so that the generated reference time tilt angle height data presents a spatial structure in the building sidewall area consistent with the tilt angle image. At the target time, since directly acquired height data is lacking, the reference height data can still be used as the height source, and the target tilt angle height data can be generated by combining it with the imaging geometry of the target remote sensing image, so that it presents tilt angle structure features similar to the lower part of 2-c in the sidewall area. Through this generation method based on the reference height data and the tilt angle imaging geometry of remote sensing images at each time, the tilt angle height data at both the reference and target times can present spatial structure features consistent with the imaging direction of their respective images, and express the corresponding height changes in the image. Thus, remote sensing images at different times all have height expressions that match their tilt angle imaging methods, providing a complete and unified source of height information for subsequent construction of cross-time training samples.

[0022] Step 102: Combine the tilt angle height data corresponding to different times with the remote sensing images to form a set of training samples, and combine all the training samples together to form a height estimation training sample set for the training area.

[0023] In some embodiments of this application, to construct a dataset suitable for training a height estimation model, it is necessary to combine tilted view height data acquired at different times with corresponding remote sensing images, so that each moment forms a training sample that simultaneously contains image texture information and spatial height representation. Specifically, by pairing the tilted view height data at each moment with its corresponding remote sensing image, a consistent correlation structure is formed between the spatial structure presentation and height information of the image, thereby constituting a height estimation training sample set covering the training area. By forming this training sample set, the model can be exposed to the image-height correspondence across time dimensions during the training phase, providing the model with a continuous and structurally consistent data source when learning the mapping rules between image texture and height, thus providing a stable data foundation for subsequent model training.

[0024] In a specific example, remote sensing images at each time step can be paired with their corresponding tilt angle height data. The remote sensing images are stored in GeoTIFF raster format, containing high-resolution satellite fusion imagery in red, green, and blue spectral bands. The tilt angle height data is generated jointly by nDSM and the imaging geometry of the images at each time step. After pairing, the images and height data can be divided into fixed-size image patches according to the model input requirements, so that each image patch contains both image texture and height information, thereby constructing a height estimation training sample set covering the training area. The sample set formed in this way can provide height representations at multiple time steps in the temporal dimension, providing diverse and structurally consistent data sources for the subsequent training of the height estimation model.

[0025] Step 103: Train the height estimation model based on the height estimation training sample set to obtain a trained height estimation model.

[0026] The height estimation model is used to predict the spatial height estimate of each pixel position in the remote sensing image at the target time.

[0027] In some embodiments of this application, to enable the height estimation model to learn the mapping relationship between image texture and spatial height, it is necessary to train the model using a height estimation training sample set. This allows the model to gradually build the predictive ability from remote sensing images to height representations. Specifically, image-height pairing data from the training sample set is input into the model, allowing it to continuously adjust its internal parameters through multiple iterations to adapt to the height distribution structure under different times and imaging conditions. A height estimation model is a type of learning model used to infer spatial height from remote sensing images, and its output is the height estimate corresponding to each pixel location. By training based on the training sample set, the model can possess a stable height inference capability when faced with remote sensing images at a target time, providing a directly usable model foundation for subsequent height estimation processes.

[0028] In a specific example, image-height pairing data from the height estimation training sample set can be input into the model in batches. The image data comes from high-resolution satellite fusion imagery in GeoTIFF raster format, and the height data comes from the tilted view height representation generated from nDSM and imaging geometry. During training, the model performs forward inference and backward updates on each batch of samples, gradually converging the internal parameters to a state that stably represents the image-height relationship. Through this training process, the model can output height estimates for each pixel location after receiving remote sensing images at the target time, thus forming a trained and completed model that can be used for subsequent height estimation tasks.

[0029] In summary, in this embodiment, firstly, reference height data for the reference time is generated using both reference height data and reference remote sensing images available at the reference time. Then, using this reference height data as the height source, and combining it with the target remote sensing image available only at the target time, tilt angle height data for the target time is generated. This ensures that both the reference and target remote sensing images obtain height representations corresponding to their imaging methods, providing complete height input for subsequent training. Based on this, the tilt angle height data obtained at the reference and target times are combined with the corresponding remote sensing images to form a height estimation training sample set for training. This ensures that the training samples simultaneously possess image and height information across time dimensions and maintain continuity in spatial structure presentation, thus providing a stable data foundation for the model to establish cross-time image height correspondences. Subsequently, the height estimation model is trained using the aforementioned training sample set. During the learning process of image texture and height relationships, the model continuously encounters multi-time height representations extended from the reference height data, ultimately forming a unified height representation method for remote sensing images acquired at different times. This allows the model to output structurally reasonable height estimation results even when real height data is lacking at the target time. Therefore, the method based on the embodiments of this application possesses the ability to adapt to height representation deviations caused by imaging tilt when facing remote sensing images with non-nadir characteristics. Furthermore, it can generate stable height representations based on height information from a reference time, even when true height data is lacking at the target time, thereby improving the generalization performance of the height estimation model under non-nadir remote sensing image conditions. In addition, in the embodiments of this application, when a non-nadir remote sensing image is input into the trained height estimation model, the model generates an elevation image matching the imaging method of the image based on the correspondence between the image and the tilted viewpoint height data formed during the training phase. This ensures that the height estimation results remain stable in terms of spatial structure and height representation, which is beneficial for continuously utilizing multi-temporal remote sensing images to obtain consistent height information in applications such as urban 3D modeling and change monitoring.

[0030] Figure 3 This application provides another method for training a height estimation model, which specifically includes the following steps: Step 201: Generate tilt angle height data of the training area at the reference time based on the reference height data and reference remote sensing image of the training area at the reference time, and generate tilt angle height data of the training area at the target time based on the reference height data and target remote sensing image of the training area at the target time.

[0031] Among them, the reference height data is used to characterize the spatial height of each location in the training area; the tilt angle height data at different times have different shooting angles; each pixel position of the tilt angle height data has a corresponding spatial height value under the corresponding shooting angle.

[0032] The method shown in this step has been explained in step 101 and will not be repeated here.

[0033] Optionally, when the reference remote sensing image is a satellite image, in order to generate tilt angle height data of the training area at the reference time based on the reference height data of the training area at the reference time and the reference remote sensing image, the angle transformation of the reference height data can be performed on the reference height data by establishing the object-image spatial mapping relationship between the reference height data and the reference remote sensing image, so as to obtain the tilt angle height data of the training area at the reference time.

[0034] In some embodiments of this application, to ensure that the height representation at the reference time is consistent with the imaging geometry of the reference remote sensing image, the reference height data may be optionally transformed based on the object-image spatial mapping relationship between the reference height data and the reference remote sensing image, so that the spatial projection of the reference height data is consistent with the imaging method of the satellite image. Specifically, by using the regularization parameters and the Rational Polynomial Coefficient (RPC) provided by the satellite image to establish the mapping relationship from the object space to the image space, the reference height data can be reprojected according to the imaging geometry of the satellite image during the projection process, thereby generating tilted angle height data of the training area at the reference time. Through this angle transformation method, the reference height data can be made consistent with the satellite image in spatial structure, providing a unified imaging angle basis for the subsequent construction of image-height training samples.

[0035] In a specific example, a forward rational function model representing the mapping from object space to image space can be used to perform a viewpoint transformation on the reference nDSM, where the forward rational function model can be expressed as: , ,in, For calculating the vertical coordinates of the image The third-order combination of spatial quantities, derived from ground coordinates The product terms constitute the spatial components of the object point projected onto the longitudinal direction of the image during the imaging process. This is a third-order combination of depth scaling factors normalized to the aforementioned spatial quantities, used to characterize the depth variation of an object point along the line of sight. Similarly, For calculating the horizontal coordinates of the image The third-order combination of spatial quantities is used to characterize the spatial component of the object point projected onto the image side in the lateral direction during the imaging process. The third-order combination of depth scaling factors, which normalizes the aforementioned spatial quantities, is used to characterize the depth variation of an object point along the line of sight. During viewpoint transformation, the three-dimensional coordinates of each pixel on the reference nDSM can be substituted into the aforementioned forward rational function model to calculate its planar coordinates in the satellite non-nadir imagery, ensuring that the reference altitude data maintains consistency with the imaging geometry of the satellite imagery during projection. The tilted viewpoint altitude data generated in this way can form a unified viewpoint representation with the satellite imagery at the reference time in terms of spatial structure, providing a consistent imaging basis for subsequent image-altitude training sample construction.

[0036] Similar to the processing method of reference remote sensing images, when the target remote sensing image is a satellite image, in order to generate tilt angle height data of the training area at the target time based on the reference height data and the target remote sensing image of the training area at the target time, the angle transformation of the reference height data can be performed by the object-image spatial mapping relationship established between the reference height data and the target remote sensing image to obtain the tilt angle height data at the target time.

[0037] In some embodiments of this application, to ensure that the height representation at the target time is geometrically consistent with the target remote sensing image, the reference height data can optionally undergo a perspective transformation based on the object-image spatial mapping relationship between the reference height data and the target remote sensing image. This makes the spatial projection of the reference height data consistent with the imaging method of the satellite image at the target time. Since this perspective transformation process is consistent in principle with the processing method at the reference time, and also establishes a mapping relationship from object space to image space based on the imaging parameters provided by the satellite image, when the target image is a satellite image and perspective consistency is required, the same processing method as at the reference time can be used to generate tilted perspective height data of the training area at the target time. In this way, the height representation at the target time can be geometrically consistent with the target image, providing a unified perspective basis for subsequent construction of image-height training samples.

[0038] In a specific example, the perspective transformation process at the reference time can be directly adopted. The three-dimensional coordinates of each pixel on the reference nDSM are input into the imaging model of the target satellite image, and its planar coordinates in the target image are calculated to obtain the tilt angle height data at the target time. Since the previous steps have explained in detail the specific principles and processing methods of perspective transformation based on satellite image imaging parameters, the relevant formulas and parameter structures will not be repeated here. It will only be stated that the same perspective transformation process can be used when the target image is a satellite image, so that the height expression at the target time is consistent with the target image in imaging geometry.

[0039] Step 202: By comparing the features of the remote sensing images at the reference time and the target time, each target remote sensing image is divided into multiple regions. The target regions in each divided target remote sensing image are marked and / or removed according to the preset region marking rules, so as to update each target remote sensing image separately.

[0040] In some embodiments of this application, in order to distinguish the types of height changes in urban areas at different times in the training data and to avoid abnormal height data caused by sudden changes from entering the training process, it is necessary to compare the remote sensing images at the reference time and the target time, and divide the remote sensing image at the target time into different types of regions such as vegetation growth areas, building demolition areas, and height abrupt change areas. Subsequently, according to these preset region labeling rules, slowly changing and unchanged areas are labeled and retained for subsequent training, while height abrupt change areas or occluded areas are labeled and / or removed to avoid these areas disrupting the image-height mapping relationship during the training phase. Through this region division and updating based on the characteristics of urban scene changes, the target remote sensing image can be distinguished between stable and abnormal changes before training, making the height representation of subsequent training samples more reliable and reducing the interference of abrupt data introduced by factors such as new construction, demolition projects, or image occlusion on the model learning process.

[0041] In a specific example, the remote sensing image at a reference time can be used as a baseline to determine the changes in the remote sensing image at the target time. Analyzing the remote sensing images at both times reveals that some buildings show a slow increase in height at the target time, which is normal construction progress; other areas show a sudden decrease in height at the target time, possibly corresponding to demolition or large-scale occlusion; and some areas show abnormal height jumps at the target time, possibly caused by differences in facade projection due to changes in imaging angle. Based on preset area labeling rules, slowly changing areas can be marked as retainable areas for subsequent model training; areas with abrupt height changes and occlusion areas are marked as unusable areas and deleted from the target remote sensing image. After this processing, the remote sensing image at the target time retains only areas with stable changes, providing a more reliable data source for constructing training samples on urban scene change patterns.

[0042] It should be noted that whether this step is performed depends on whether there are identifiable height changes in the training area at the target time. For example, in some urban areas, building structures remain stable at different times, without any differences in construction, demolition, or occlusion caused by changes in imaging angle. In this case, the remote sensing image at the target time is generally consistent with the remote sensing image at the reference time, without any obvious change patterns. In this case, there is no need to perform regional division or labeling on the remote sensing image at the target time, and the process can proceed directly to the subsequent steps.

[0043] Optionally, step 202 includes the following sub-steps: Sub-step 2021 compares the remote sensing image at the reference time with the remote sensing image at the target time, identifies the regions that have changed between the two times based on the features of the two images, and divides the target remote sensing image into three categories of regions.

[0044] The three types of areas include stable built-up areas, areas with slight changes and vegetation growth, and areas with significant changes.

[0045] In some embodiments of this application, in order to delineate different types of height change areas in remote sensing images at a target time, manual interpretation can be used to identify the main change types causing changes in the height of ground features within the coverage area, including three categories: unchanged built-up areas, unchanged vegetation areas, and areas altered by human activities. Alternatively, a supervised classification method can be used, constructing multi-channel feature maps such as spectral and textural features from images of different time phases as input to the classification model. By selecting representative pixels for the above three types of change in the images and configuring model parameters, training can be completed to achieve automatic extraction of different change types.

[0046] In a specific example, the images at the reference and target times can first be manually interpreted to identify unchanged built-up areas, unchanged vegetation areas, and areas altered by human activities such as construction, demolition, tree planting, or logging. These areas can then be used as auxiliary information for the type of height change. Alternatively, multi-channel feature maps with spectral and textural characteristics can be constructed, and random forest methods can be used for classification to automatically extract areas of different change types. The first category is stable built-up areas, typically corresponding to unchanged built-up areas or unchanged vegetation areas; the second category is areas of slight change and vegetation growth, typically corresponding to vegetation growth or minor construction changes; and the third category is areas of significant change, typically corresponding to new construction, demolition, or large-scale occlusion changes. In this way, three categories of areas can be formed in the remote sensing image at the target time, providing a clear basis for subsequent labeling and removal processes.

[0047] Sub-step 2022 involves marking at least one of the stable built-up areas with slight changes and the vegetation growth areas, and / or removing areas with significant changes, in order to update the remote sensing image at the target time.

[0048] In some embodiments of this application, to ensure that the remote sensing image at the target time possesses a stable region structure suitable for training in terms of change patterns, the remote sensing image at the target time needs to be updated based on the aforementioned change type delineation results. Specifically, at least one of the slightly changed and vegetation-grown areas and the unchanged stable built-up areas is marked, enabling these areas to be identified as retainable areas during the subsequent construction of training samples. Simultaneously, areas of significant change are removed, preventing abrupt height changes caused by construction, demolition, tree planting, or logging from entering the training samples, thereby avoiding interference from abnormal changes on the image-height mapping relationship. Through this change type-based marking and removal method, the remote sensing image at the target time can distinguish between stable and abnormal changes before training, providing a more reliable height representation basis for the subsequent construction of training samples.

[0049] In a specific example, the three categories of regions identified above can be processed separately. For instance, stable built-up areas, which show almost no height change, typically correspond to unchanged built-up areas or unchanged vegetation areas and can be directly retained. Areas with slight changes and vegetation growth typically correspond to local height changes caused by natural vegetation growth or minor construction, and these areas can be marked so that they can be identified as retainable areas during the subsequent construction of training samples. Areas with significant changes typically correspond to abrupt height changes caused by the construction or demolition of buildings or by planting, pruning, or felling trees, and these areas can be removed from the remote sensing images at the target time to prevent them from entering the training samples. The height data at the target time processed in this way is more stable in terms of change patterns, providing a reliable height representation for the subsequent construction of training samples.

[0050] Step 203: Combine the tilt angle height data corresponding to different times with the remote sensing images to form a set of training samples, and combine all the training samples together to form a height estimation training sample set for the training area.

[0051] The method shown in this step has been explained in step 102 and will not be repeated here.

[0052] Optionally, step 203 includes the following sub-steps: Sub-step 2031 involves performing interval linear mapping on the height data of each tilt angle according to a preset height value interval, so as to obtain the normalized tilt angle height data corresponding to each tilt angle height data.

[0053] In some embodiments of this application, to ensure that the height data from different time dimensions at tilted viewpoints are consistent with the image in terms of numerical scale, it is necessary to perform interval linear mapping on the height data from each tilted viewpoint according to a preset height value range, so that the original height values ​​are expressed within a uniform scale range. Specifically, by linearly mapping each height value according to a preset minimum and maximum height, the height data is converted into a normalized height expression while maintaining the spatial structure. Through this interval mapping method, height data from different times can have a consistent numerical range during the training phase, thereby avoiding training instability caused by differences in height scale and providing a unified height expression basis for the subsequent construction of image-height training samples.

[0054] In a specific example, the original height range of the tilted view height data can be set to 0–80 meters, and the height value of each pixel can be linearly mapped according to this range to an 8-bit (i.e., 0–255) numerical range. This linear mapping method ensures that the tilted view height data at different times maintains a consistent numerical scale, providing a unified height representation for the subsequent construction of image-height training samples, and enabling stable training of the subsequent height estimation model.

[0055] It is important to emphasize that the aforementioned interval linear mapping is only used to establish a unified numerical scale between height maps in different time dimensions, thereby maintaining consistency in height representation during the training phase. As for the model training process itself, it can be performed directly using the mapped 0–255 range, or it can continue using the original height range (e.g., 0–80 meters). There is no substantial difference in the training logic between the two methods; the only difference lies in whether a corresponding mapping and restoration mechanism is set up internally or externally within the model. If the 0–255 range is used for training, the model can restore the predicted results to the original height range through inverse linear mapping during the output phase. If the original height range is used for training, a unified expression of height values ​​can be achieved before or after training through a fixed linear mapping relationship. Regardless of the method used, the consistency of the model's height representation scale can be guaranteed, enabling the training process to stably learn height change patterns and output height estimates consistent with the true height range during the inference phase.

[0056] Sub-step 2032 involves combining the one-to-one corresponding remote sensing images with the normalized tilt angle height data to obtain a set of training samples, and then determining all the training samples as the height estimation training sample set.

[0057] In some embodiments of this application, to construct image-height pairing data suitable for training height recognition models, it is necessary to combine one-to-one corresponding remote sensing images with normalized tilt angle height data. This ensures that each covered area simultaneously contains image texture information and height representation at a uniform scale within the same spatial range. Specifically, within the same spatial range, remote sensing images are used as image feature inputs, and normalized tilt angle height data obtained through interval linear mapping is used as height labels or height feature inputs. This maintains spatial and numerical scale consistency in the training samples at the layer level. This combination method ensures that the height estimation training sample set remains uniform in both numerical scale and spatial structure, providing a stable source of image-height pairing data for subsequent model training.

[0058] In a specific example, the remote sensing image at the reference time can be used as a two-dimensional or multi-channel image dataset, and the corresponding normalized tilt angle height data can be used as height map data within the same spatial range. These two datasets can be combined one-to-one to form a training sample set. Similarly, the remote sensing image at the target time can be combined with its corresponding normalized tilt angle height data to form another training sample set. By performing layer-level combination of remote sensing images and normalized tilt angle height data from multiple coverage areas or multiple time phases, a training sample set consisting of multiple image-height pairing data sets can be obtained. This set is then designated as the height estimation training sample set, providing a complete and scale-uniform training data source for the subsequent training of the height estimation model.

[0059] Step 204: Data augmentation is performed on the height estimation training sample set based on at least one set of training samples in the height estimation training sample set to update the height estimation training sample set.

[0060] In some embodiments of this application, to enrich the spatial structure and image texture of the training data, data augmentation is performed on the training sample set based on at least one set of training samples from the height estimation training sample set, thereby further expanding the diversity and stability of the training data. Specifically, by performing enhancement operations such as image tiling, random perturbation, and viewpoint variation on selected training samples, the image texture and height representation can present more variation patterns while maintaining spatial correspondence, thus improving the model's adaptability under different imaging conditions. In urban remote sensing scenarios, building facades, road structures, and shadow distribution may differ at different times and imaging angles. Data augmentation allows the model to be exposed to a wider range of image-height combinations during the training phase, providing a more comprehensive sample source for subsequent model training.

[0061] In a specific example, a training sample consisting of GeoTIFF imagery and tilted perspective height data can be selected from the height estimation training sample set, and the enhancement operations described in the handover materials can be performed on this sample. For example, the imagery and height data can be segmented using a sliding window approach, dividing the complete image into non-overlapping or overlapping segments of a fixed size of 256×256 pixels or 512×512 pixels, enabling the model to learn the height structure of local areas; the imagery and height data can also be simultaneously subjected to random horizontal or vertical flipping to ensure that the building facade texture maintains a consistent height expression in different directions; the imagery can also be randomly rotated and scaled to simulate urban scenes under different imaging angles and spatial scales. Through these enhancement methods, the training sample set forms a richer combination in terms of image texture, spatial structure, and imaging conditions, providing diverse and structurally consistent training data for subsequent model training.

[0062] It should be noted that this step is optional, and its execution depends primarily on the processing capabilities of the height estimation model and the size and coverage of the original training data. When the model has strong generalization ability and the number of training samples, scene types, and imaging conditions can cover the target application requirements, the original height estimation training sample set can be used directly for training without additional data augmentation. When the model capacity is limited or the original training sample size is small and the scene distribution is uneven, this step can be used to augment the training sample set to improve the effective size and diversity of the training data without significantly increasing the acquisition cost.

[0063] Step 205: Train the height estimation model based on the height estimation training sample set to obtain a trained height estimation model.

[0064] The height estimation model is used to predict the spatial height estimate of each pixel position in the remote sensing image at the target time.

[0065] The method shown in this step has been explained in step 103 and will not be repeated here.

[0066] Optionally, step 205 includes the following sub-steps: Sub-step 2051: Input each group of training samples in the training sample set into the height estimation model to obtain the spatial height estimate corresponding to each group of training samples based on the output of the height estimation model, and obtain the prediction regression deviation feature of the height estimation model under the training sample set based on the discrimination module of the height estimation model.

[0067] In some embodiments of this application, to obtain the height prediction results and structural deviation features of the height estimation model on the training sample set, each group of training samples in the training sample set needs to be input into the height estimation model. During the forward inference process, the model outputs a spatial height estimate corresponding to that training sample, and the model's discrimination module obtains the predictive regression deviation features representing the structural representation of that training sample. Specifically, by inputting image-height pairing data into the height estimation model, the model generates a height estimation map in the height prediction branch that is consistent with the spatial range of the input image. Simultaneously, the discrimination module discriminates the spatial structural consistency between the image and the height estimation map, thereby obtaining the height prediction value and structural deviation features used for subsequent construction of the joint loss.

[0068] In a specific example, the height estimation model employs a Conditional Generative Adversarial Network (cGAN) architecture. The generator uses a U-Net encoder-decoder convolutional network structure with both downsampling and upsampling factors set to 2. During forward inference, the generator extracts multi-scale features from a fixed-size input image patch (e.g., a 256×256×3 color image patch) and progressively restores the spatial resolution in the decoding path, outputting a simulated height map (e.g., a 256×256×1 simulated nDSM) consistent with the spatial extent of the image patch, forming the height prediction value for the training data. The discrimination module uses a patch-based Generative Adversarial Network (PatchGAN) structure that discriminates at the image patch scale. It takes the image patch and the height map output by the generator as input, discriminates the spatial structure consistency at the local image patch scale, and outputs a structural deviation feature matrix composed of multiple local regions, used to measure the deviation of the generated height map in local structural representation. PatchGAN's local discrimination mechanism effectively captures deviations in height maps regarding edge structure, texture continuity, and local height variation patterns, thus providing structural constraints for height estimation tasks. Through the aforementioned forward inference process, the predicted height and structural deviation features for each training data point in the training sample set can be obtained, providing foundational data for constructing a joint loss based on these features.

[0069] Sub-step 2052: Establish a joint loss for the height estimation model based on all spatial height estimates and predicted regression deviation features, and train the height estimation model based on the joint loss.

[0070] In some embodiments of this application, to ensure that the height estimation model simultaneously constrains the numerical accuracy of height prediction and the consistency of spatial structure during the training phase, a joint loss for the height estimation model needs to be established based on all spatial height estimates and prediction regression deviation features, and the height estimation model is trained according to the joint loss. Specifically, the difference between the spatial height estimates and the actual height is constructed as the height regression loss, and the prediction regression deviation features are constructed as the structural discriminant loss, so that the height estimation model considers both the regression error of the height values ​​and the degree of deviation in the spatial structure representation during the training process. By backpropagating the joint loss, the parameters of the height estimation model can be continuously updated in multiple iterations, enabling the model to gradually improve its ability to express the image-height relationship.

[0071] In a specific example, the difference between the spatial height estimate output by the generator and the true height map can be constructed as a regression loss based on pixel-level height difference. For example, an L1 loss form based on the difference between the height estimate and the true height value can be used to measure the deviation of the height estimate from the overall height trend. Simultaneously, the predicted regression deviation features output by the discrimination module can be constructed as an adversarial discriminant loss to measure the deviation of the generated height map from the local spatial structure. Here, the spatial height estimate corresponds to the height prediction result of the height estimation model, and the predicted regression deviation features correspond to the structural deviation expression obtained by the discrimination module in discriminating image-height pairing data at the local image patch scale. In the specific construction of the joint loss, the regression loss based on the spatial height estimate and the adversarial discriminant loss based on the predicted regression deviation features can be weighted and summed according to preset weights to form the joint loss function used to train the height estimation model. During training, the Adam optimizer can be used to backpropagate the joint loss, allowing the parameters of the generator and discrimination module to be continuously updated in multiple iterations, thereby simultaneously reducing the regression error between the height estimate and the true height, and reducing the degree of structural deviation represented by the predicted regression deviation features. By using this joint loss training method based on spatial height estimation and predicted regression deviation characteristics, the height estimation model can stably learn the image-height mapping law during the training phase.

[0072] Sub-step 2053: If the joint loss satisfies the preset convergence condition, or if the training epochs of the height estimation model reach the preset epoch threshold, the trained height estimation model is obtained.

[0073] In some embodiments of this application, to obtain a trained height estimation model, the training process of the height estimation model needs to be terminated and the trained height estimation model output when the joint loss meets a preset convergence condition or the number of training epochs of the height estimation model reaches a preset epoch threshold. Specifically, by continuously monitoring the changing trend of the joint loss during training, when the joint loss remains stable or decreases below a preset threshold for several consecutive epochs, it can be determined that the height estimation model has reached a convergence state; if the number of training epochs reaches a preset maximum epoch threshold, the training process can also be terminated and the height estimation model in the current state can be output. Through this training termination method based on convergence conditions or epoch thresholds, it can be ensured that the height estimation model obtains stable parameter expressions during the training phase.

[0074] In a specific example, the joint loss can be monitored round by round during training. The target is set when the joint loss decreases by less than a preset 10% threshold over 20 consecutive iterations. -4 When the residuals are found, the height estimation model can be considered to have converged, and the training process can be terminated. Alternatively, if the number of training epochs reaches the preset maximum threshold of 200 epochs, the training process can also be terminated and the height estimation model in the current training state can be output. After training is terminated, the network parameters of the generator can be used as the final parameters of the height estimation model, which can be used to estimate the height of the input image during the inference phase, thereby forming the trained height estimation model.

[0075] In summary, in this embodiment, firstly, reference height data for the reference time is generated using both reference height data and reference remote sensing images available at the reference time. Then, using this reference height data as the height source, and combining it with the target remote sensing image available only at the target time, tilt angle height data for the target time is generated. This ensures that both the reference and target remote sensing images obtain height representations corresponding to their imaging methods, providing complete height input for subsequent training. Based on this, the tilt angle height data obtained at the reference and target times are combined with the corresponding remote sensing images to form a height estimation training sample set for training. This ensures that the training samples simultaneously possess image and height information across time dimensions and maintain continuity in spatial structure presentation, thus providing a stable data foundation for the model to establish cross-time image height correspondences. Subsequently, the height estimation model is trained using the aforementioned training sample set. During the learning process of image texture and height relationships, the model continuously encounters multi-time height representations extended from the reference height data, ultimately forming a unified height representation method for remote sensing images acquired at different times. This allows the model to output structurally reasonable height estimation results even when real height data is lacking at the target time. Therefore, the method based on the embodiments of this application possesses the ability to adapt to height representation deviations caused by imaging tilt when facing remote sensing images with non-nadir characteristics. Furthermore, it can generate stable height representations based on height information from a reference time, even when true height data is lacking at the target time, thereby improving the generalization performance of the height estimation model under non-nadir remote sensing image conditions. In addition, in the embodiments of this application, when a non-nadir remote sensing image is input into the trained height estimation model, the model generates an elevation image matching the imaging method of the image based on the correspondence between the image and the tilted viewpoint height data formed during the training phase. This ensures that the height estimation results remain stable in terms of spatial structure and height representation, which is beneficial for continuously utilizing multi-temporal remote sensing images to obtain consistent height information in applications such as urban 3D modeling and change monitoring.

[0076] Figure 4 This application provides a method for estimating the height of urban areas based on high-resolution images and viewpoint transformation, specifically including the following steps: Step 301: Input the remote sensing image of the region to be estimated into the height estimation model trained by the height estimation model training method in the above embodiment, so as to obtain the spatial height estimation value corresponding to each pixel position in the remote sensing image of the region to be estimated, so as to form a spatial height estimation map of the region to be estimated.

[0077] In some embodiments of this application, in order to obtain a spatial height representation of the region to be estimated, a remote sensing image of the region to be estimated is input into a height estimation model trained by a height estimation model training method. During forward inference, the model outputs the spatial height estimate corresponding to each pixel position in the remote sensing image. Specifically, by using the image of the region to be estimated as input to the model, the height estimation model infers the texture structure, spatial variation patterns, and local features in the image based on the image-height mapping rules learned during the training phase, thereby generating a spatial height estimation map covering the entire region to be estimated.

[0078] In a specific example, the remote sensing image of the region whose height to be estimated can be cropped or resampled according to the input size of the height estimation model. For instance, the original image can be divided into several image blocks of uniform size, and these image blocks can be sequentially input into the trained height estimation model. During the forward inference process, the height estimation model can analyze the texture features and spatial variation patterns of each image block and output a height estimation result consistent with the spatial extent of that image block. Subsequently, the height estimation results of all image blocks can be stitched together or reconstructed according to their spatial positions in the original image, thereby obtaining a spatial height estimation map covering the entire region whose height to be estimated, providing basic data for subsequent tasks such as height analysis, change detection, or 3D reconstruction.

[0079] In summary, in this embodiment, firstly, reference height data for the reference time is generated using both reference height data and reference remote sensing images available at the reference time. Then, using this reference height data as the height source, and combining it with the target remote sensing image available only at the target time, tilt angle height data for the target time is generated. This ensures that both the reference and target remote sensing images obtain height representations corresponding to their imaging methods, providing complete height input for subsequent training. Based on this, the tilt angle height data obtained at the reference and target times are combined with the corresponding remote sensing images to form a height estimation training sample set for training. This ensures that the training samples simultaneously possess image and height information across time dimensions and maintain continuity in spatial structure presentation, thus providing a stable data foundation for the model to establish cross-time image height correspondences. Subsequently, the height estimation model is trained using the aforementioned training sample set. During the learning process of image texture and height relationships, the model continuously encounters multi-time height representations extended from the reference height data, ultimately forming a unified height representation method for remote sensing images acquired at different times. This allows the model to output structurally reasonable height estimation results even when real height data is lacking at the target time. Therefore, the method based on the embodiments of this application possesses the ability to adapt to height representation deviations caused by imaging tilt when facing remote sensing images with non-nadir characteristics. Furthermore, it can generate stable height representations based on height information from a reference time, even when true height data is lacking at the target time, thereby improving the generalization performance of the height estimation model under non-nadir remote sensing image conditions. In addition, in the embodiments of this application, when a non-nadir remote sensing image is input into the trained height estimation model, the model generates an elevation image matching the imaging method of the image based on the correspondence between the image and the tilted viewpoint height data formed during the training phase. This ensures that the height estimation results remain stable in terms of spatial structure and height representation, which is beneficial for continuously utilizing multi-temporal remote sensing images to obtain consistent height information in applications such as urban 3D modeling and change monitoring.

[0080] Figure 5 This application provides another method for estimating urban area height based on high-resolution images and viewpoint transformation, specifically including the following steps: Step 401: Based on the division of the entire height region to be estimated, the remote sensing full map corresponding to the entire height region to be estimated is divided into remote sensing images corresponding to multiple height regions to be estimated.

[0081] In some embodiments of this application, since the remote sensing map corresponding to the entire height region to be estimated typically has a very large spatial size, directly inputting the entire remote sensing map into the height estimation model would result in insufficient processor video memory or system memory to meet the computational resources required for model inference, thus making it impossible to complete height estimation processing at the whole map scale. Therefore, based on the division method of the entire height region to be estimated, the remote sensing map can be divided into multiple remote sensing images corresponding to each height region to be estimated, ensuring that the size of each region-level remote sensing image meets the input requirements of the height estimation model. Through this segmentation method, the originally unprocessable ultra-large remote sensing map can be converted into multiple region-level image patches that can be directly input into the height estimation model, providing processable input data for the subsequent height estimation process.

[0082] In a specific example, the size of the remote sensing image corresponding to the height estimation region can reach tens of thousands of pixels, such as 20,000 × 20,000 pixels. Since height estimation models typically require fixed-size image patches (e.g., 256 × 256 or 512 × 512 pixels) as input during the inference phase, and the processor's video memory cannot load the entire remote sensing image in a single inference iteration, the height estimation region can be divided according to a preset spatial grid, for example, into several spatial sub-regions of 512 × 512 pixels. The remote sensing image can then be segmented according to the spatial location of these sub-regions, resulting in multiple region-level remote sensing images corresponding to each sub-region. By segmenting the remote sensing image in this way, ultra-large images that cannot be directly input into the height estimation model can be converted into multiple appropriately sized region-level image patches, providing directly processable input data for subsequent region-level height estimation.

[0083] Step 402: Input the remote sensing image of the region to be estimated into the height estimation model trained by the height estimation model training method in the above embodiment, so as to obtain the spatial height estimation value corresponding to each pixel position in the remote sensing image of the region to be estimated, so as to form a spatial height estimation map of the region to be estimated.

[0084] The method shown in this step has been explained in step 301 and will not be repeated here.

[0085] Step 403: The spatial height estimation maps corresponding to each region to be estimated are stitched together according to the division method of the entire region to be estimated, so as to obtain a complete spatial height estimation map of the entire region to be estimated.

[0086] In some embodiments of this application, in order to obtain a complete spatial height estimation map of the entire height region to be estimated, the spatial height estimation maps corresponding to each region to be estimated can be stitched together according to the division method of the entire height region to be estimated, so that the height estimation results of each region are spatially restored to match the original remote sensing data. Figure 1 The overall layout is consistent. Specifically, by maintaining the spatial position of each region whose height is to be estimated during the stitching process, the regional spatial height estimation maps are arranged and combined according to their spatial range within the entire region of the height to be estimated, thus forming a complete spatial height estimation map covering the entire region of the height to be estimated. Through this stitching method, the regional height estimation results can be restored to a global-scale height expression, providing complete height data for subsequent height analysis or 3D reconstruction.

[0087] In a specific example, the entire height region to be estimated has been divided into several spatial sub-regions of 512×512 pixels according to a preset spatial grid, and spatial height estimation maps corresponding to each spatial sub-region have been obtained in the previous embodiment. Subsequently, based on the spatial position of each spatial sub-region in the entire height region to be estimated, these regional spatial height estimation maps can be stitched together in row and column order. For example, multiple regional height maps in the same row can be stitched together horizontally first, and then the stitching results of multiple rows can be combined vertically to form a complete spatial height estimation map covering the entire height region to be estimated. Through this stitching process based on spatial division, the regional height estimation results can be spatially restored to a complete global height representation, providing basic data for subsequent height visualization, change detection, or 3D modeling.

[0088] As a verification of this application, such as Figure 6As shown, this application selects high spatial resolution remote sensing imagery of a typical urban area as experimental data to evaluate the effectiveness of the height estimation method proposed in this application. The image data used in the experiment is a fused imagery of the high-resolution satellite imagery WorldView-2 (WorldView-2 High-Resolution Satellite Imagery), that is, a high-resolution satellite imagery after multispectral and panchromatic fusion, with a spatial resolution of approximately sub-meter level, including red, green, and blue spectral bands. At the same time, airborne laser detection and ranging (LiDAR) point cloud data of the area at a certain imaging time is acquired, and the corresponding reference nDSM is generated through point cloud processing. For WorldView-2 imagery at another imaging time, a multi-temporal image set is constructed, and the imagery at that time is used as the input imagery for height estimation (6-a, where the content of the illustration has been appropriately blurred to avoid exposing specific regional information). To verify the effectiveness of the method proposed in this application, a height map at that moment was generated using a stereo matching method (6-b) and a height map at that moment was generated using a conventional deep learning-based single-scene height estimation method (6-c), and the height estimation results obtained by the method proposed in this application (6-d) were compared.

[0089] In the height map generation and model training stages, the non-nadir nDSM generation at each imaging time point is based on 10 regularization parameters and 80 RPC coefficients contained in the imaging model file of WorldView-2 imagery to establish the imaging geometric relationship between image coordinates and ground coordinates. Multi-temporal images and their corresponding non-nadir nDSMs are all segmented into 256×256×3 pixel image blocks to construct a multi-temporal image-height training sample set. In building the height estimation model, a deep learning architecture based on conditional generative adversarial networks (cGAN) is adopted. The downsampling and upsampling factors are both set to 2, the random inactivation rate is set to 0.2, the batch size is set to 1, the number of iterations is set to 200, and the learning rate is set to 0.0001. Model training is implemented using a GPU-accelerated dataflow deep learning framework (TensorFlow GPU - Accelerated Dataflow Deep Learning Framework, version 1.15.0). For large-area images that require height estimation, they are also divided into image blocks of 256×256×3 pixels and input into the trained height estimation model.

[0090] Experimental results show that the proposed method can obtain relatively accurate height estimation results within the target area (6-d). As can be seen from the figures, the height predictions of ground features such as buildings are more complete, building outlines are clear, and the height representation is consistent with the viewing angle of the image. The facade structure of high-rise buildings is also effectively represented. Compared with the results obtained by the stereo matching method (6-b), the proposed method performs better in terms of ground feature detail and spatial resolution; compared with the results obtained by the conventional single-scene height estimation method (6-c), the proposed method significantly improves the problems of unclear ground feature outlines and unstable local height prediction. The above comparative results verify the effectiveness of the proposed method in multi-temporal height estimation tasks.

[0091] It should be noted that the remote sensing imagery, map information, and related geospatial data used in the embodiments of this application all come from legal and compliant data provision channels and are used for the construction and verification of the technical solution after obtaining the necessary authorization. In the process of using data, this application complies with local laws, regulations, and applicable regulatory requirements regarding surveying and mapping results, satellite remote sensing data, and geographic information products, ensuring that the data sources used are authentic and legal, and does not involve any unauthorized data collection, purchase, download, or dissemination. The imagery and elevation data cited in the embodiments are only used to illustrate the technical solution of this application and do not constitute a public disclosure of the actual geographic information of any specific area.

[0092] In the processing, storage, and display of remote sensing imagery, map information, and elevation data, this application complies with local laws, regulations, and regulatory requirements regarding data security, personal information protection, cybersecurity, and geographic information security management. The data processing in this embodiment is solely for technical verification and does not involve the collection, analysis, or disclosure of personal information, sensitive geographical locations, or restricted surveying and mapping results. This application assumes no responsibility for any unauthorized data processing, dissemination, or use. Users, when actually applying the technical solutions of this application, should ensure that the data sources they use are legal, fully authorized, and processed in compliance with regulations, and should comply with all requirements of local laws and regulations.

[0093] It is important to emphasize that the images, maps, and elevation information shown in the embodiments of this application are only for illustrating the implementation of the technical solution, and their illustrated content does not constitute a disclosure of actual geographic information of any specific area. To avoid exposing identifiable area information or causing unnecessary geographic information security risks, this application has appropriately blurred some of the illustrated content. When actually applying the technical solution of this application, users should ensure that the images, maps, and related data used are all from legally authorized channels, and comply with applicable local laws, regulations, and regulatory requirements during data collection, processing, storage, and display, and bear the corresponding compliance responsibilities themselves.

[0094] In summary, in this embodiment, firstly, reference height data for the reference time is generated using both reference height data and reference remote sensing images available at the reference time. Then, using this reference height data as the height source, and combining it with the target remote sensing image available only at the target time, tilt angle height data for the target time is generated. This ensures that both the reference and target remote sensing images obtain height representations corresponding to their imaging methods, providing complete height input for subsequent training. Based on this, the tilt angle height data obtained at the reference and target times are combined with the corresponding remote sensing images to form a height estimation training sample set for training. This ensures that the training samples simultaneously possess image and height information across time dimensions and maintain continuity in spatial structure presentation, thus providing a stable data foundation for the model to establish cross-time image height correspondences. Subsequently, the height estimation model is trained using the aforementioned training sample set. During the learning process of image texture and height relationships, the model continuously encounters multi-time height representations extended from the reference height data, ultimately forming a unified height representation method for remote sensing images acquired at different times. This allows the model to output structurally reasonable height estimation results even when real height data is lacking at the target time. Therefore, the method based on the embodiments of this application possesses the ability to adapt to height representation deviations caused by imaging tilt when facing remote sensing images with non-nadir characteristics. Furthermore, it can generate stable height representations based on height information from a reference time, even when true height data is lacking at the target time, thereby improving the generalization performance of the height estimation model under non-nadir remote sensing image conditions. In addition, in the embodiments of this application, when a non-nadir remote sensing image is input into the trained height estimation model, the model generates an elevation image matching the imaging method of the image based on the correspondence between the image and the tilted viewpoint height data formed during the training phase. This ensures that the height estimation results remain stable in terms of spatial structure and height representation, which is beneficial for continuously utilizing multi-temporal remote sensing images to obtain consistent height information in applications such as urban 3D modeling and change monitoring.

[0095] like Figure 7 As shown in the embodiments of this application, a training device 50 for a height estimation model is also disclosed, comprising: The viewpoint alignment module 501 is used to generate tilt viewpoint height data of the training area at the reference time based on the reference height data of the training area at the reference time and the reference remote sensing image of the training area at the target time, and to generate tilt viewpoint height data of the training area at the target time based on the reference height data and the target remote sensing image of the training area at the target time; the reference height data is used to characterize the spatial height of each position in the training area; the tilt viewpoint height data at different times have different shooting angles; each pixel position of the tilt viewpoint height data has a corresponding spatial height value at the corresponding shooting angle. The training set building module 502 is used to combine the tilt angle height data corresponding to different times with the remote sensing images as a set of training samples, and to combine all the training samples together to form a height estimation training sample set for the training area. The model training module 503 is used to train the height estimation model based on the height estimation training sample set to obtain a trained height estimation model. The height estimation model is used to predict the spatial height estimate value corresponding to each pixel position in the remote sensing image at the input target time.

[0096] Optionally, the view alignment module 501 includes: The reference transformation submodule is used to perform perspective transformation on the reference height data by means of the object-image spatial mapping relationship established between the reference height data and the reference remote sensing image when the reference remote sensing image is a satellite image, so as to obtain the tilt perspective height data of the training area at the reference time.

[0097] Optionally, the view alignment module 501 includes: The target transformation submodule is used to perform perspective transformation on the reference height data when the target remote sensing image is a satellite image, by using the object-image spatial mapping relationship established between the reference height data and the target remote sensing image, so as to obtain the tilt perspective height data at the target time.

[0098] Optionally, the training apparatus 50 for the height estimation model also includes: The classification and update module is used to divide each target remote sensing image into multiple regions by comparing the features of the remote sensing images at the reference time and the target time, and to mark and / or remove the target regions in each target remote sensing image according to the preset region marking rules, so as to update each target remote sensing image separately.

[0099] Optional, the category update module includes: The classification submodule is used to compare the remote sensing image at the reference time with the remote sensing image at the target time, identify the regions that have changed between the two times based on the features of the two images, and divide the target remote sensing image into three categories of regions: stable built-up areas, areas with slight changes and vegetation growth, and areas with significant changes. The update submodule is used to mark at least one of the stable built-up areas with slight changes and the vegetation growth areas, and / or remove areas with significant changes, so as to update the remote sensing image at the target time.

[0100] Optionally, the training set building module 502 includes: The normalization submodule is used to perform interval linear mapping on the height data of each tilt angle according to the preset height value interval, so as to obtain the normalized tilt angle height data corresponding to each tilt angle height data. The training set module is used to combine one-to-one corresponding remote sensing images with normalized tilt angle height data to obtain a set of training samples, and to determine all training samples as the height estimation training sample set.

[0101] Optionally, the training apparatus 50 for the height estimation model also includes: The training set augmentation module is used to augment the height estimation training sample set with at least one set of training samples from the height estimation training sample set to update the height estimation training sample set.

[0102] Optionally, the model training module 503 includes: The training data submodule is used to input each group of training samples in the training sample set into the height estimation model, so as to obtain the spatial height estimate value corresponding to each group of training samples according to the output of the height estimation model, and obtain the prediction regression deviation feature of the height estimation model under the training sample set according to the discrimination module of the height estimation model. The training submodule is used to establish a joint loss for the height estimation model based on all spatial height estimates and predicted regression deviation features, and to train the height estimation model based on the joint loss. The iteration termination submodule is used to obtain a trained height estimation model when the joint loss meets the preset convergence condition or the training epochs of the height estimation model reach the preset epoch threshold.

[0103] like Figure 8 As shown in the embodiments, this application also discloses an urban area height estimation device 60 based on high-resolution image viewpoint transformation, comprising: The image recognition module 601 is used to input the remote sensing image of the region to be estimated into the height estimation model trained by the height estimation model training method in the above embodiment, so as to obtain the spatial height estimation value corresponding to each pixel position in the remote sensing image of the region to be estimated, so as to form a spatial height estimation map of the region to be estimated.

[0104] Optionally, the urban area height estimation device 60 based on high-resolution image viewpoint transformation further includes: The segmentation module is used to segment the remote sensing full map corresponding to the height to be estimated into remote sensing images corresponding to multiple height regions, based on the division of the entire height region to be estimated. The stitching module is used to stitch together the spatial height estimation maps corresponding to each region whose height is to be estimated according to the division method of the entire region whose height is to be estimated, so as to obtain a complete spatial height estimation map of the entire region whose height is to be estimated.

[0105] In summary, in this embodiment, firstly, reference height data for the reference time is generated using both reference height data and reference remote sensing images available at the reference time. Then, using this reference height data as the height source, and combining it with the target remote sensing image available only at the target time, tilt angle height data for the target time is generated. This ensures that both the reference and target remote sensing images obtain height representations corresponding to their imaging methods, providing complete height input for subsequent training. Based on this, the tilt angle height data obtained at the reference and target times are combined with the corresponding remote sensing images to form a height estimation training sample set for training. This ensures that the training samples simultaneously possess image and height information across time dimensions and maintain continuity in spatial structure presentation, thus providing a stable data foundation for the model to establish cross-time image height correspondences. Subsequently, the height estimation model is trained using the aforementioned training sample set. During the learning process of image texture and height relationships, the model continuously encounters multi-time height representations extended from the reference height data, ultimately forming a unified height representation method for remote sensing images acquired at different times. This allows the model to output structurally reasonable height estimation results even when real height data is lacking at the target time. Therefore, the method based on the embodiments of this application possesses the ability to adapt to height representation deviations caused by imaging tilt when facing remote sensing images with non-nadir characteristics. Furthermore, it can generate stable height representations based on height information from a reference time, even when true height data is lacking at the target time, thereby improving the generalization performance of the height estimation model under non-nadir remote sensing image conditions. In addition, in the embodiments of this application, when a non-nadir remote sensing image is input into the trained height estimation model, the model generates an elevation image matching the imaging method of the image based on the correspondence between the image and the tilted viewpoint height data formed during the training phase. This ensures that the height estimation results remain stable in terms of spatial structure and height representation, which is beneficial for continuously utilizing multi-temporal remote sensing images to obtain consistent height information in applications such as urban 3D modeling and change monitoring.

[0106] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described height estimation model training method or the urban area height estimation method embodiment based on high-resolution images and viewpoint transformation, achieving the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0107] Figure 9 This is a block diagram of an electronic device 700 provided in an embodiment of this application. For example, the electronic device 700 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0108] Reference Figure 9 The electronic device 700 may include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0109] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the training method for the height estimation model or the urban area height estimation method based on high-resolution images and viewpoint transformation described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.

[0110] Memory 704 is used to store various types of data to support the operation of electronic device 700. Examples of this data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, multimedia, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0111] Power supply component 706 provides power to various components of electronic device 700. Power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.

[0112] Multimedia component 708 includes a screen that provides an output interface between electronic device 700 and user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When electronic device 700 is in an operating mode, such as shooting mode or multimedia mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0113] Audio component 710 is used to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) used to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0114] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0115] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of electronic device 700. For example, sensor assembly 714 may detect the on / off state of electronic device 700, the relative positioning of components such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0116] Communication component 716 facilitates wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 7G), or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0117] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement the training method of the height estimation model provided in the embodiments of this application or the urban area height estimation method based on high-resolution images and viewpoint transformation.

[0118] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of an electronic device 700 to complete the training method for the height estimation model or the urban area height estimation method based on high-resolution images and viewpoint transformation. For example, the non-transitory storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0119] In an exemplary embodiment, the electronic device 700 may also be provided as a server, including a processing component 702, which further includes one or more processors 720, and memory resources represented by memory 704 for storing instructions executable by the processing component 702, such as application programs. The application programs stored in memory 704 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 702 is configured to execute instructions to perform the training method for the height estimation model provided in the embodiments of this application or the urban area height estimation method based on high-resolution images and viewpoint transformation.

[0120] Electronic device 700 may also include a power supply component 706 configured to perform power management of electronic device 700, a wired or wireless network interface 718 configured to connect electronic device 700 to a network, and an input / output (I / O) interface 712. Electronic device 700 may operate on an operating system stored in memory 704, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0121] This application also provides a computer program product, including a computer program, a method for training a height estimation model implemented when the computer program is executed by a processor, or a method for estimating the height of urban areas based on high-resolution images and viewpoint transformation.

[0122] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims below.

[0123] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the claimed technical solutions.

[0124] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0125] It will be readily apparent to those skilled in the art that any combination of the above embodiments is feasible. Therefore, any combination of the above embodiments is an implementation scheme of this application. However, due to space limitations, this specification will not describe them in detail here.

[0126] It should be noted that, unless otherwise expressly stated, the training method for the height estimation model or the urban area height estimation method based on high-resolution images and viewpoint transformation provided in the embodiments of this application are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. Based on the above description, the required structure for constructing a system having the solutions of this application is obvious. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0127] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0128] Similarly, it should be understood that, for the purpose of simplification and aiding understanding of one or more aspects of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the application. However, this disclosure should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claimed solution. Rather, as reflected in the claimed solutions, the application aspects consist of fewer features than all those in the single embodiment disclosed above. Therefore, the claimed solutions following the detailed description are thus expressly incorporated into this detailed description, wherein each claimed solution is itself a separate embodiment of the application.

[0129] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in the application and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this application may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0130] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claimed technical solutions, any one of the claimed embodiments can be used in any combination.

[0131] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the training method for the height estimation model according to the embodiments of this application or the urban area height estimation method based on high-resolution images and viewpoint transformation. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0132] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the training method of the height estimation model of the embodiment of this application or the urban area height estimation method based on high-resolution images and viewpoint transformation.

[0133] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0134] It should be noted that the above embodiments are illustrative of this application and not limiting of it, and those skilled in the art can devise alternative embodiments without departing from the scope of the claimed technical solutions. In the claimed technical solutions, any reference signs placed between parentheses should not be construed as limiting the claimed technical solutions. The word "comprising" does not exclude the presence of elements or steps not listed in the claimed technical solutions. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the claimed technical solutions listing several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0135] It should be noted that, for the sake of simplicity, the method embodiments of this application are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of this application.

[0136] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for embodiments of systems or devices, since they are basically similar to the method embodiments, the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0137] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A training method for a height estimation model, characterized in that, include: Based on the reference height data and reference remote sensing image of the training area at the reference time, the tilt angle height data of the training area at the reference time is generated, and based on the reference height data and the target remote sensing image of the training area at the target time, the tilt angle height data of the training area at the target time is generated. The reference height data is used to characterize the spatial height of each location in the training area; The tilt angle height data at different times have different shooting angles; Each pixel position of the tilt angle height data has a corresponding spatial height value under the corresponding shooting angle. The tilt angle height data corresponding to different times are combined with remote sensing images to form a set of training samples, and all the training samples are combined to form the height estimation training sample set of the training area. The height estimation model is trained based on the height estimation training sample set to obtain the trained height estimation model; the height estimation model is used to predict the spatial height estimate value corresponding to each pixel position in the remote sensing image at the input target time.

2. The training method for the height estimation model as described in claim 1, characterized in that, The training method for the height estimation model also includes: By comparing the features of the remote sensing images at the reference time and the target time, each target remote sensing image is divided into multiple regions. The target regions in each divided target remote sensing image are marked and / or removed according to the preset region marking rules, so as to update each target remote sensing image separately.

3. The training method for the height estimation model as described in claim 1, characterized in that, The step of combining the tilt angle height data corresponding to different times with remote sensing images as corresponding training samples, and then combining all the training samples to form the height estimation training sample set for the training area, includes: Each of the tilt angle height data is linearly mapped according to a preset height value range to obtain normalized tilt angle height data corresponding to each of the tilt angle height data. The normalized tilt angle height data corresponding to the reference time and the normalized tilt angle height data corresponding to the target time are combined to obtain a set of training samples, and all of the training samples are determined as the height estimation training sample set.

4. The training method for the height estimation model as described in claim 1, characterized in that, The step of training the height estimation model based on the height estimation training sample set to obtain the trained height estimation model includes: Each group of training samples in the training sample set is input into the height estimation model to obtain a spatial height estimate corresponding to each group of training samples based on the output of the height estimation model, and the prediction regression deviation feature of the height estimation model under the training sample set is obtained based on the discrimination module of the height estimation model. A joint loss for the height estimation model is established based on all the spatial height estimates and the predicted regression deviation features, and the height estimation model is trained based on the joint loss. The trained height estimation model is obtained when the joint loss satisfies a preset convergence condition or when the training epochs of the height estimation model reach a preset epoch threshold.

5. A method for estimating the height of urban areas based on high-resolution images and viewpoint transformation, characterized in that, include: The remote sensing image of the region to be estimated is input into the height estimation model trained by the height estimation model according to any one of claims 1 to 4, so as to obtain the spatial height estimation value corresponding to each pixel position in the remote sensing image of the region to be estimated, so as to form a spatial height estimation map of the region to be estimated.

6. The urban area height estimation method based on high-resolution images and viewpoint transformation as described in claim 5, characterized in that, The urban area height estimation method based on high-resolution images and viewpoint transformation also includes: Based on the division of the entire area of ​​height to be estimated, the remote sensing full map corresponding to the entire area of ​​height to be estimated is divided into remote sensing images corresponding to multiple areas of height to be estimated respectively; The spatial height estimation maps corresponding to each of the regions to be estimated are stitched together according to the division method of the entire region to be estimated, so as to obtain a complete spatial height estimation map of the entire region to be estimated.

7. A training device for a height estimation model, characterized in that, include: The perspective transformation module is used to generate tilt perspective height data of the training area at the reference time based on the reference height data and the reference remote sensing image of the training area at the reference time, and to generate tilt perspective height data of the training area at the target time based on the reference height data and the target remote sensing image of the training area at the target time. The reference height data is used to characterize the spatial height of each location in the training area; The tilt angle height data at different times have different shooting angles; Each pixel position of the tilt angle height data has a corresponding spatial height value under the corresponding shooting angle. The training set building module is used to combine the tilt angle height data corresponding to different times with the remote sensing images respectively as a set of training samples, and to combine all the training samples together to form the height estimation training sample set of the training area. The model training module is used to train the height estimation model based on the height estimation training sample set to obtain the trained height estimation model; the height estimation model is used to predict the spatial height estimate value corresponding to each pixel position in the remote sensing image at the input target time.

8. A device for estimating the height of urban areas based on high-resolution image perspective transformation, characterized in that, include: An image recognition module is used to input a remote sensing image of the region to be estimated into a height estimation model trained by the height estimation model training method according to any one of claims 1 to 4, so as to obtain the spatial height estimation value corresponding to each pixel position in the remote sensing image of the region to be estimated, so as to form a spatial height estimation map of the region to be estimated.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 6.

10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 6.

Images