A building height estimation method, device, terminal equipment and storage medium

Abstract

This invention discloses a method, apparatus, terminal device, and storage medium for estimating building height, belonging to the field of building height estimation. The method includes: acquiring optical images and ground truth data of sample buildings; training a preset building teacher model based on the building optical images and ground truth data of sample buildings to obtain a target building teacher model, and acquiring the multi-task prediction results of the teacher buildings; acquiring SAR images and radar system parameters of sample buildings; training a preset building student model based on the SAR images, radar system parameters, multi-task prediction results of the teacher buildings, and ground truth data of sample buildings to obtain a target building student model; acquiring SAR images and radar system parameters of the building to be detected, and inputting them into the target building student model to obtain a refined height map of the building to be detected, thereby improving the accuracy of building height estimation.

Description

Technical Field

[0001] This invention relates to the field of building height estimation, and more particularly to a building height estimation method, apparatus, terminal equipment, and storage medium. Background Technology

[0002] Synthetic Aperture Radar (SAR), as an active remote sensing technology, possesses all-weather, all-day Earth observation capabilities and has significant application value in the field of urban building height estimation. Existing SAR image-based building height estimation techniques mainly fall into the following categories: The first category is SAR technology based on multi-view observation, which mainly includes methods such as Interferometric Synthetic Aperture Radar (InSAR) and Tomographic Synthetic Aperture Radar (TomoSAR). However, although this type of method has advantages in high-precision elevation reconstruction, its practical application is significantly limited. First, the method heavily relies on multi-temporal and multi-angle data, requiring multiple flight passes to acquire image sequences that meet baseline requirements, resulting in long data acquisition cycles and high costs. Second, the data processing workflow is complex, with extremely high requirements for image registration accuracy and phase quality. In urban areas, due to the complex geometry of buildings, phase decoherence is prone to occur, leading to elevation inversion failure. In addition, this method is difficult to adapt to rapidly changing urban environments and cannot meet the high-timeliness requirements of applications such as emergency monitoring.

[0003] The second category is traditional feature analysis methods based on single SAR images. However, while these methods avoid dependence on multi-temporal data, their drawbacks are also significant. First, manually designed feature extraction rules are poorly robust in complex urban environments, heavily influenced by speckle noise and geometric deformation in SAR images, leading to unstable feature extraction. Second, the methods are sensitive to building shape, orientation, and surrounding environment, exhibiting significantly reduced generalization ability in densely built-up, irregularly laid-out areas. Furthermore, the height estimation process relies on simplified geometric assumptions, failing to accurately reflect the interactions of multiple scattering mechanisms in real-world scenes, resulting in substantial height estimation bias.

[0004] The third category comprises single-image SAR height estimation methods based on deep learning. Although deep learning models such as the SAR2Height framework have made initial progress in single-image SAR height estimation, the inherent speckle noise and complex geometric distortions of SAR images still severely limit the estimation accuracy, resulting in a significant performance gap compared to optical image methods. Furthermore, existing methods typically employ an end-to-end regression architecture, which struggles to effectively decouple building contour extraction and height inversion tasks, leading to a lack of detail discrimination in height maps and blurred height differences between buildings. Summary of the Invention

[0005] This invention provides a method, apparatus, terminal device, and storage medium for estimating building height, which can solve the above-mentioned problems in the prior art and improve the accuracy of building height estimation.

[0006] This invention provides a method for estimating building height, comprising: Obtain optical images and ground truth data of the sample buildings; The optical image of the building is used as the input value, and the ground truth data of the sample building is used as the label to train the preset building teacher model to obtain the target building teacher model, and the multi-task prediction result of the teacher building is obtained based on the target building teacher model. Acquire sample building SAR images and sample radar system parameters for the sample buildings; Using the sample building SAR image and sample radar system parameters as input values, and the teacher building multi-task prediction results and the sample building ground truth data as labels, the preset building student model is trained to obtain the target building student model. Acquire SAR images of the building to be detected and parameters of the radar system to be detected; The SAR image of the building to be detected and the parameters of the radar system to be detected are input into the student model of the target building to obtain the multi-task prediction result of the student building. Based on the multi-task prediction results of the student building, a detailed height map of the building to be tested is obtained.

[0007] In the above scheme, a target building teacher model is obtained by training a preset building teacher model with sample building optical images and sample building ground truth data as labels. The teacher building multi-task prediction results are obtained based on the target building teacher model. The teacher building multi-task prediction results and sample building ground truth data are then used as labels to train a preset building student model in combination with sample building SAR images and sample radar system parameters. This allows the preset building student model to transfer knowledge of building outline and height distribution from sample optical images, retaining richer structural details. Thus, with only the input of the SAR image of the building to be detected and the parameters of the radar system to be detected, a high-precision refined height map can be obtained, significantly improving the accuracy of building height estimation in a single SAR image.

[0008] Furthermore, acquiring the optical image and ground truth data of the sample building includes: Obtain the ground truth data of the sample building height, the ground truth data of the sample building binarized segmentation, and the sample height hierarchy table of the sample building; Based on the sample height hierarchy classification table, the true value data of building height is mapped hierarchically to obtain the sample hierarchy number; The sample level number, the sample building binarized segmentation ground value, and the sample building height ground value are used as the sample building ground value data.

[0009] In the above scheme, by obtaining the sample height level division table and performing level mapping on the height ground value data, continuous height values ​​are discretized into sample level numbers, which together with the sample building binarized segmentation ground value and the sample building height ground value data constitute the sample building ground value data, providing a clear label structure for subsequent multi-task supervised training.

[0010] Further, obtaining the sample building height ground truth data, sample building binarized segmentation ground truth, and height hierarchy table of the sample building includes: Obtain the ground truth data of the sample building height and the ground truth data of the sample building binarized segmentation; Obtain the maximum and minimum values ​​of the sample buildings; The sample height level division table is obtained based on the maximum value of the sample buildings, the minimum value of the sample buildings, and the preset total number of sample levels.

[0011] In the above scheme, a height level division table is generated by using the maximum and minimum values ​​of sample buildings and the preset total number of levels. This makes the level division have the geometric interval characteristics of logarithmic space, which can adapt to buildings of different height ranges and improve the rationality of level mapping.

[0012] Further, the step of using the optical image of the building as input value and the ground truth data of the sample building as label to train a preset architectural teacher model to obtain a target architectural teacher model, and obtaining multi-task prediction results of the teacher building based on the target architectural teacher model, includes: The optical image of the building is input into a preset building teacher model to obtain the teacher training level number, the teacher building binarized segmentation training value, and the teacher building height training probability. The teacher building height training data is obtained based on the teacher building height training probability. The first teacher building loss function is obtained based on the training values ​​of the binarized segmentation of the teacher building and the true values ​​of the binarized segmentation of the sample building. The second teacher building loss function is obtained based on the teacher training level number, the sample level number, and the preset height level weight; A third teacher building loss function is obtained based on the training data of the teacher building height and the ground value data of the sample building height. Construct a target teacher building loss function based on the first teacher building loss function, the second teacher building loss function, and the third teacher building loss function; The target teacher model is obtained by training the preset teacher model based on the target teacher building loss function, and the teacher building multi-task prediction results are obtained based on the target teacher building loss function.

[0013] In the above scheme, when training the preset architectural teacher model, three loss functions are designed for binarization segmentation, hierarchical numbering, and height regression, respectively. Based on the preset height hierarchical weights, the influence of categories is balanced, so that the target architectural teacher model can accurately output the multi-task prediction results of teachers' buildings, providing high-quality soft labels for the subsequent target architectural student model.

[0014] Further, the step of using the sample building SAR image and sample radar system parameters as input values, and using the teacher building multi-task prediction results and the sample building ground truth data as labels to train the preset building student model to obtain the target building student model includes: The sample building SAR image and sample radar system parameters are input into the preset building student model to obtain the student training level number, the student building binarized segmentation training value and the student building height training data. The first student building loss function is obtained based on the training values ​​of the binarized segmentation of the student buildings and the true values ​​of the binarized segmentation of the sample buildings; The second student building loss function is obtained based on the student training level number, the sample level number, and the preset height level weight; A third student building loss function is obtained based on the student building height training data, the sample building height ground value data, and the teacher building height training probability. Construct a target student building loss function based on the first student building loss function, the second student building loss function, and the third student building loss function; The target student building model is obtained by training the preset student building model based on the target student building loss function, and the multi-task prediction results of student buildings are obtained based on the target student building model.

[0015] In the above scheme, when training the target building student model, the training probability of the teacher building height output by the target building teacher model and the sample building ground value data are used as supervision. This allows the target building student model to directly learn the ground value and also to prioritize imitating the high-confidence regression knowledge of the target building teacher model, thereby improving the height estimation accuracy and generalization ability of the target building student model on SAR images.

[0016] Furthermore, the multi-task prediction result for student buildings includes a target building outline mask, a target level classification map, and candidate height values ​​for each target level. The step of obtaining a refined height map of the building to be detected based on the multi-task prediction result for student buildings includes: Based on the target level attribution map and the height candidate values ​​for each target level, an initial assembly height map is obtained; Based on the target building outline mask and the initial height map, the refined height map is obtained.

[0017] In the above scheme, an initial height map is obtained by combining the target level assignment map with the height candidate values ​​of each target level. Then, it is refined by combining the target building outline mask to obtain a refined height map. This process effectively removes noise in non-building areas, sharpens building outlines, and improves the boundary accuracy of the final height map.

[0018] Further, the step of obtaining the initial height map based on the target level attribution map and the height candidate values ​​for each target level includes: Obtain the pixel coordinates and level index of each pixel in the target level attribution map; Based on the hierarchical number index, the corresponding index height channel is selected from the candidate height values ​​of each level of the target; An initial height value is selected from the index height channel based on the pixel coordinates; The initial stitching height map is obtained based on the initial height value corresponding to all the pixels.

[0019] In the above scheme, the initial height value of each pixel is extracted from the corresponding height channel using the hierarchical number index of each pixel, and all pixel values ​​are collected to form an initial stitched height map, which realizes the rapid and accurate fusion of hierarchical segmentation results and multi-channel regression results.

[0020] Another embodiment of the present invention provides a building height estimation device, comprising: The first basic data acquisition module acquires optical images and ground truth data of the sample buildings. The teacher model training module is used to take the optical image of the building as the input value and the ground truth data of the sample building as the label to train the preset building teacher model to obtain the target building teacher model, and obtain the teacher building multi-task prediction result based on the target building teacher model. The second basic data acquisition module is used to acquire sample building SAR images and sample radar system parameters of sample buildings; The student model training module is used to train the preset building student model by taking the sample building SAR image and sample radar system parameters as input values, and the teacher building multi-task prediction results and sample building ground truth data as labels, to obtain the target building student model. The data acquisition module is used to acquire the SAR image of the building to be detected and the parameters of the radar system to be detected. The student building multi-task prediction result generation module is used to input the SAR image of the building to be detected and the parameters of the radar system to be detected into the student model of the target building to obtain the student building multi-task prediction result. The refined height map acquisition module is used to obtain the refined height map of the building to be detected based on the multi-task prediction results of the student building.

[0021] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of a building height estimation method as described in the present invention.

[0022] Another embodiment of the present invention also provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform steps such as the building height estimation method of the present invention. Attached Figure Description

[0023] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a building height estimation method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a building height estimation device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the building data processing module structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the segmentation-regression prediction process module structure provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the segmentation-regression knowledge distillation module structure provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0027] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0030] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0031] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0032] See Figure 1 To address the aforementioned problems in the prior art and improve the accuracy of building height estimation, an embodiment of the present invention provides a building height estimation method, apparatus, terminal device, and storage medium, comprising: Step S1: Obtain the optical image and ground truth data of the sample building; Step S2: Use the optical image of the building as the input value and the ground truth data of the sample building as the label to train the preset building teacher model to obtain the target building teacher model, and obtain the teacher building multi-task prediction result based on the target building teacher model; Step S3: Obtain sample building SAR images and sample radar system parameters for the sample buildings; Step S4: Using the sample building SAR image and sample radar system parameters as input values, and the teacher building multi-task prediction results and the sample building ground truth data as labels, train the preset building student model to obtain the target building student model. Step S5: Obtain the SAR image of the building to be detected and the parameters of the radar system to be detected; Step S6: Input the SAR image of the building to be detected and the parameters of the radar system to be detected into the student model of the target building to obtain the multi-task prediction result of the student building; Step S7: Based on the multi-task prediction results of the student building, obtain the refined height map of the building to be detected.

[0033] In the above scheme, a target building teacher model is obtained by training a preset building teacher model with sample building optical images and sample building ground truth data as labels. The teacher building multi-task prediction results are obtained based on the target building teacher model. The teacher building multi-task prediction results and sample building ground truth data are then used as labels to train a preset building student model in combination with sample building SAR images and sample radar system parameters. This allows the preset building student model to transfer knowledge of building outline and height distribution from sample optical images, retaining richer structural details. Thus, with only the input of the SAR image of the building to be detected and the parameters of the radar system to be detected, a high-precision refined height map can be obtained, significantly improving the accuracy of building height estimation in a single SAR image.

[0034] It should be noted that the target building teacher model is trained using optical images of sample buildings, while the target building student model is trained using SAR images of sample buildings as input. During the training of the target building student model, radar system parameters such as downward viewing angle and polarization are embedded as feature conditions into each stage of the network. Both the target building teacher and student models employ a unified multi-task architecture, executing three core sub-tasks in parallel: binarization segmentation, height-level segmentation, and hierarchical regression. In the prediction stage, dictionary lookups and mask refinement operations are performed on the outputs of the three sub-tasks to fuse them into a final, refined height map. In the knowledge distillation stage, corresponding distillation loss functions (i.e., the target student building loss function) are designed for the feature attributes and output formats of different sub-tasks to drive the target building student model to fully learn the building outline structure information and multi-level height distribution features contained in the target building teacher model.

[0035] In another embodiment, acquiring the optical image and ground truth data of the sample building includes: Obtain the ground truth data of the sample building height, the ground truth data of the sample building binarized segmentation, and the sample height hierarchy table of the sample building; Based on the sample height hierarchy classification table, the true value data of building height is mapped hierarchically to obtain the sample hierarchy number; The sample level number, the sample building binarized segmentation ground value, and the sample building height ground value are used as the sample building ground value data.

[0036] In another embodiment, obtaining the sample building height ground truth data, the sample building binarized segmentation ground truth, and the height hierarchy table of the sample building includes: Obtain the ground truth data of the sample building height and the ground truth data of the sample building binarized segmentation; Obtain the maximum and minimum values ​​of the sample buildings; The sample height level division table is obtained based on the maximum value of the sample buildings, the minimum value of the sample buildings, and the preset total number of sample levels.

[0037] It should be noted that the specific method for obtaining the optical images and ground truth data of the sample buildings is as follows: First, obtain the ground truth data of the sample building height (e.g., through a Digital Surface Model (DSM), the binarized segmentation ground truth of the sample building, and a pre-constructed sample height hierarchy table. The sample height hierarchy table is obtained using the logarithmic spatial interval incremental discretization method, and its discrete interval threshold is calculated using the following formula: ; in, ; For the i-th discrete interval threshold, The maximum value of the sample buildings. Let C be the minimum height of the sample buildings, C be the total number of preset sample levels, and exp() represent the natural exponential function. Based on the discrete interval threshold, each pixel in the ground truth data of sample building heights obtained through Digital Surface Model (DSM) data is assigned an interval number. This number serves as the ground truth label for subsequent height-level segmentation tasks. By summarizing the interval numbers of all pixels, the sample height-level classification table is obtained. This method can automatically generate segmentation labels using only DSM data, eliminating the need for additional manual annotation. The output of the height-level segmentation task branch in the target building teacher model is the teacher grade classification map in the teacher building multi-task prediction results, which includes the interval number corresponding to each pixel value in this prediction. The subsequent hierarchical regression task branch outputs the candidate height values ​​for each teacher grade in the teacher building multi-task prediction results, where each channel is responsible for regressing the specific height value of the building within a specific height level. It is understandable that the output of the binarization task branch in the target building teacher model is the teacher building contour mask in the teacher building multi-task prediction results.

[0038] Furthermore, the process of training the student model for the target building is similar to the process of training the teacher model for the target building described above: Based on the aforementioned discrete interval threshold, each pixel in the ground truth data of sample building heights obtained from Digital Surface Model (DSM) data is assigned an interval number. This number serves as the ground truth label 's' for the subsequent height hierarchy segmentation task. By summarizing the interval numbers of all pixels, the sample height hierarchy partitioning table is obtained. The output of the height hierarchy segmentation task branch in the subsequent inference stage is the target level assignment map in the student building multi-task prediction results. , H is the height of the image, and W is the width of the image, where the target level attribution map includes the value of each pixel. The corresponding height-level prediction sub-result is a predicted interval number. .

[0039] After height-level segmentation, buildings within the same interval have similar heights, significantly reducing the complexity of subsequent regression predictions. When the number of levels is set to C, the subsequent hierarchical regression task branch outputs candidate height values ​​for each target level in the student building multi-task prediction results, i.e., a multi-channel height prediction map. , Each channel is responsible for retrieving the specific height value of a building within a specific height level.

[0040] In another embodiment, the step of using the optical image of the building as input value and the ground truth data of the sample building as label to train a preset architectural teacher model to obtain a target architectural teacher model, and obtaining multi-task prediction results of the teacher building based on the target architectural teacher model, includes: The optical image of the building is input into a preset building teacher model to obtain the teacher training level number, the teacher building binarized segmentation training value, and the teacher building height training probability. The teacher building height training data is obtained based on the teacher building height training probability. The first teacher building loss function is obtained based on the training values ​​of the binarized segmentation of the teacher building and the true values ​​of the binarized segmentation of the sample building. The second teacher building loss function is obtained based on the teacher training level number, the sample level number, and the preset height level weight; A third teacher building loss function is obtained based on the training data of the teacher building height and the ground value data of the sample building height. Construct a target teacher building loss function based on the first teacher building loss function, the second teacher building loss function, and the third teacher building loss function; The target teacher model is obtained by training the preset teacher model based on the target teacher building loss function, and the teacher building multi-task prediction results are obtained based on the target teacher building loss function.

[0041] It should be noted that for binarization segmentation tasks, i.e., simple binarization segmentation tasks that distinguish between buildings and non-buildings, a combination of Kullback-Leibler divergence (hereinafter referred to as KL divergence) and Focal Loss loss function is used: ; in, For the first teacher building loss function, The teacher's binarized Focal Loss function. The teacher binarized KL divergence is used. The training values ​​of the teacher building binarized segmentation and the ground truth values ​​of the sample building binarized segmentation are compared to obtain the teacher binarized KL divergence and the teacher binarized Focal Loss function; based on the teacher binarized KL divergence and the teacher binarized Focal Loss function, a first teacher building loss function is obtained.

[0042] For the height-level segmentation task, the teacher training level number and the sample level number are compared to obtain the teacher level number KL divergence and teacher level number Focal Loss loss function; based on the teacher level number KL divergence, teacher level number Focal Loss loss function, and preset height level weights, a second teacher building loss function is obtained. This is because the height-level segmentation task focuses on the division of building height levels. To avoid interference from large areas of ground background in the remote sensing image, the weight of the ground category is set to [value missing] in the KL divergence and Focal Loss loss function. The weights for the other height levels are set to... ,in To predetermine the total number of sample levels, the second teacher building loss function is finally obtained.

[0043] For hierarchical regression tasks, the L1 function between the training data of teacher building heights and the ground truth data of sample building heights is directly obtained as the third teacher building loss function.

[0044] Then, a target teacher building loss function is constructed based on the first teacher building loss function, the second teacher building loss function, and the third teacher building loss function; the preset teacher building model is trained based on the target teacher building loss function to obtain the target teacher building model, and the teacher building multi-task prediction results are obtained based on the target teacher building model.

[0045] In another embodiment, the step of using the sample building SAR image and sample radar system parameters as input values, and using the teacher building multi-task prediction results and the sample building ground truth data as labels to train a preset building student model to obtain a target building student model includes: The sample building SAR image and sample radar system parameters are input into the preset building student model to obtain the student training level number, the student building binarized segmentation training value and the student building height training data. The first student building loss function is obtained based on the training values ​​of the binarized segmentation of the student buildings and the true values ​​of the binarized segmentation of the sample buildings; The second student building loss function is obtained based on the student training level number, the sample level number, and the preset height level weight; A third student building loss function is obtained based on the student building height training data, the sample building height ground value data, and the teacher building height training probability. Construct a target student building loss function based on the first student building loss function, the second student building loss function, and the third student building loss function; The target student building model is obtained by training the preset student building model based on the target student building loss function, and the multi-task prediction results of student buildings are obtained based on the target student building model.

[0046] It should be noted that the loss functions for the first and second student buildings are similar to the loss functions for the first and second teacher buildings: For the binarization segmentation task, the student binarization KL divergence and the student binarization Focal Loss function are obtained based on the student building binarization segmentation training values ​​and the sample building binarization ground values, and then added together to obtain the first student building loss function. For the height-level segmentation task, the KL divergence and Focal Loss function for student level numbers are obtained based on the student training level number and the sample level number. A second student building loss function is obtained based on the student level number KL divergence, the student level number Focal Loss function, and preset height-level weights. To avoid interference from large areas of ground background in the remote sensing image, the weight of the ground category is set to [value missing] in the student level number KL divergence and student level number Focal Loss function. The weights for the other height levels are set to... ,in To preset the total number of sample levels, the second student building loss function is finally obtained.

[0047] For hierarchical regression tasks, unlike typical models that output single-channel height maps, the target building student model outputs alternative height values ​​for each target level, i.e., a multi-channel height prediction map. Therefore, a specific knowledge distillation method needs to be designed to enable the target architectural student model to learn the regression knowledge of the target architectural teacher model at various levels. A direct distillation strategy is to calculate the average between the outputs of the teacher and student models. Norm as a loss function: ; In the formula, superscript and Representing the target building teacher model and the target building student model respectively, the subscripts are... Indicates the channel index. Indicates spatial location index, and These are the width and height of the image, respectively. It is an L1 norm. Therefore The regression prediction of the height value of the target building teacher model at position j in channel c is used. The regression prediction height value of the target building student model at position j in channel c.

[0048] However, this strategy of uniformly weighting all pixels and channels has significant limitations in remote sensing scenarios. Since the ground area typically occupies a large portion of the image, equal weighting dilutes the distilled signals of key building features. More importantly, it forces the student model to indiscriminately learn all predictions from the teacher network, including potentially unreliable or erroneous regression targets in low-confidence regions.

[0049] To address the aforementioned issues and achieve more effective knowledge transfer, this paper focuses on the inherent logic of the segmentation-regression framework. In this framework, the probability distribution of the height-level segmentation prediction is perfectly aligned with the height map output by the hierarchical regression branch in both spatial and channel dimensions. This is due to the dictionary lookup mechanism in the segmentation-regression process, which selects the regression value corresponding to the channel with the highest segmentation prediction confidence as the final output. Therefore, a high segmentation confidence for a particular channel in the target architectural teacher model directly indicates that the regression value of that channel contributes more to the final prediction result and should thus be considered a more reliable supervisory signal.

[0050] Based on this, drawing on the idea of ​​Channel-Wise Distillation (CWD), we utilize the spatially normalized training probability of teacher building height generated by the hierarchical segmentation task branch of the target building teacher model. As a weight, a channel-weighted regression distillation CWRD loss is proposed: ; ; In the formula, In training the probability of building height for teachers, the normalized probability of the i-th spatial location on the c-th height level channel. In the training probability of unnormalized teacher building height, the output of the height level segmentation task is the unnormalized probability of the i-th spatial location on the c-th height level channel. This is the preset distillation temperature. The channel-weighted regression distillation CWRD loss (i.e., the third student building loss function) is used, where W is the image width, H is the image height, and WH represents the total number of pixels in the image. The regression prediction of the height value of the target building teacher model at position j in channel c is used. The regression prediction height value of the target building student model at position j in channel c.

[0051] The third student building loss function achieves dual optimization: First, it utilizes segment confidence as a proxy variable for regression reliability, implementing an uncertainty-weighted distillation mechanism to guide the target building student model to prioritize learning the most reliable regression knowledge. Second, by weighted distilling multi-channel "soft labels," it enables the target building student model's learning process to move beyond isolated numerical values, allowing it to perceive and mimic the continuous value prediction patterns of the teacher network at different height levels, thus learning more robust and discriminative height distribution characteristics. CWRD is formally very similar to CWD, both using channel-normalized teacher probabilities as weights. The fundamental difference lies in the measurement of teacher-student differences: CWD aims to minimize the difference in probability distribution between teacher and student models, using KL divergence; while CWRD focuses on reducing the difference in regression values, typically using the L-norm. This formal inheritance and the inherent difference in the task together ensure the effectiveness of the CWRD method in regression distillation tasks.

[0052] Finally, a target student building loss function is constructed based on the first student building loss function, the second student building loss function, and the third student building loss function. The target student building model is obtained by training the preset student building model based on the target student building loss function, and the multi-task prediction results of student buildings are obtained based on the target student building model.

[0053] In another embodiment, the student building multi-task prediction result includes a target building outline mask, a target level classification map, and candidate height values ​​for each target level. The step of obtaining a refined height map of the building to be detected based on the student building multi-task prediction result includes: Based on the target level attribution map and the height candidate values ​​for each target level, an initial assembly height map is obtained; Based on the target building outline mask and the initial height map, the refined height map is obtained.

[0054] In another embodiment, obtaining the initial height map based on the target level attribution map and the height candidate values ​​for each target level includes: Obtain the pixel coordinates and level index of each pixel in the target level attribution map; Based on the hierarchical number index, the corresponding index height channel is selected from the candidate height values ​​of each level of the target; An initial height value is selected from the index height channel based on the pixel coordinates; The initial stitching height map is obtained based on the initial height value corresponding to all the pixels.

[0055] It should be noted that the target building student model outputs multi-task prediction results for the student building. These results include the target building outline mask, the target level assignment map, and candidate height values ​​for each target level. To integrate the target level assignment map output from the height hierarchy segmentation task and the candidate height values ​​for each target level output from the hierarchical regression task, this invention designs a dictionary lookup mechanism, the process of which is as follows: Figure 3 , Figure 4 , Figure 5 As shown. The specific steps of this mechanism are as follows: The height of each target level output by the hierarchical regression task branch is selected as an alternative value. It can be viewed as a dictionary. It contains pixel coordinates. With channel number Combinations As the key, the corresponding predicted height value As a value; as a query: use the true labels during training and the model predictions (i.e., the target level attribution map) during prediction. Iterate through all pixel positions in the query. For the training phase, for each pixel position... Based on its training query value , with training key Search Dictionary The corresponding training height prediction value is obtained. For the prediction stage, for each pixel location... According to its test query value (i.e., hierarchical number index), using the test key Search Dictionary The corresponding predicted test height value is obtained. And record it as the initial height value. Summarize the initial height values ​​at all locations to obtain the initial jigsaw height map. .

[0056] It is important to note that different query mappings are used during the training and testing phases to decouple the gradient flow between the high-level segmentation and hierarchical regression subtasks during training. If the predicted values ​​were also used for queries during training, gradient backpropagation would couple the optimization objectives of the two tasks, increasing the instability of model convergence. Therefore, the strategy adopted in this paper ensures that the two subtasks are optimized relatively independently during training.

[0057] The following diagram illustrates the entire process: (1) Reference Figure 3 Specifically: the optical image corresponds to the optical image of the sample building, serving as the input data source for the teacher model; the teacher model corresponds to the preset building teacher model / target building teacher model, taking the optical image as input and outputting three types of prediction results in parallel; the teacher model has three outputs, corresponding to the teacher's multi-task prediction results for buildings, including: Binarization segmentation: the predicted output of the true value of the binarization segmentation of the corresponding sample buildings, i.e., the outline mask of the teacher building; Height level segmentation: the predicted output of the corresponding sample level number, i.e., the teacher level classification map; Hierarchical regression: the predicted output of the true value data of the height of the corresponding sample buildings, i.e., the candidate height values ​​for each teacher level.

[0058] The SAR imagery corresponds to the sample building SAR image / the building to be detected SAR image, serving as the input data source for the student model; the student model corresponds to the preset building student model / target building student model, taking SAR imagery and radar system parameters as input, and outputting three types of prediction results in parallel; the student model has three outputs, corresponding to the student building multi-task prediction results, including: Binarization segmentation, which corresponds to the outline mask of the target building; height level segmentation, which corresponds to the target level classification map; and hierarchical regression, which corresponds to the height candidate values ​​for each target level.

[0059] The radar parameter injection module corresponds to the input features of the student model, that is, the complete process of inputting the sample radar system parameters / detection radar system parameters into the student model: The radar parameter injection module takes sample radar system parameters, such as downward view and polarization mode, as input sources. These parameters are processed layer by layer, including: Reshape, Concat, Conv, LeakyReLu, and LeakyReLu. Finally, the processed radar parameter features are injected into the student model, corresponding to the technical feature of "using sample building SAR images and sample radar system parameters as input values ​​to the preset building student model".

[0060] (2) Reference Figure 5 The segmentation-regression knowledge distillation module corresponds to the student model training distillation loss features, specifically: The student model outputs student training level number, student building binarized segmentation training value, and student building height training data. KL divergence is applied between the student building binarized segmentation training value and the sample building binarized segmentation ground value, as well as between the student training level number and the sample level number. It should be noted that when KL divergence is applied to the student training level number and the sample level number, a preset height level weight needs to be introduced to perform weight balancing on the height level division interval, thereby realizing cross-modal knowledge distillation for the segmentation task.

[0061] For obtaining the loss function for the third student building, Figure 3 The middle channel dimension Softmax is the training probability of the teacher's building height obtained after normalization processing. When calculating the L1 loss function using the multi-channel L1 norm between the student building height training data and the sample building height ground truth data, the teacher's building height training probability is introduced into the calculation for significance weighting, resulting in channel dimension regression distillation, which is the third student building loss function.

[0062] (3) Reference Figure 4 The segmentation-regression prediction process module specifically includes: The segmentation-regression prediction workflow module fully corresponds to the entire process of student model prediction → dictionary lookup → mask refinement → height map refinement, including: The student model outputs student training level number, student building binarized segmentation training value, and student building height training data. During the training phase: The loss is the Focal Loss function, which operates between the student building binarized segmentation training value and the sample building binarized segmentation ground value, as well as between the student training level number and the sample level number. The L1 loss is applied between the student building height training data and the sample building height ground truth data. It should be noted that the teacher building height training probability is also introduced.

[0063] In summary, the loss functions for the first and second student buildings both utilize KL divergence and Focal Loss, with the second student building loss function also incorporating preset height level weights. The loss function for the third student building utilizes L1 loss and the training probability of the teacher's building height.

[0064] In the subsequent processing stage: The multi-task prediction results for student buildings include the target building outline mask, the target level assignment map, and the height candidate values ​​for each target level. argmax corresponds to the step of "obtaining the pixel coordinates and level index of each pixel in the target level attribution map", which is the mathematical operation for selecting the pixel level index.

[0065] The dictionary lookup corresponds to the steps of selecting the corresponding height channel from the height candidate values ​​of each target level based on the hierarchical number index, extracting the initial height value by pixel coordinates, and finally generating the initial mosaic height map.

[0066] Mask refinement corresponds to the step of "obtaining a refined height map based on the target building outline mask and the initial height map", which means using the binarized segmented target building outline mask to refine the outline of the initial height map.

[0067] The predicted height map is the initial height map output by the dictionary lookup.

[0068] What is understandable is that the sample building binarization segmentation ground truth, the sample building height ground truth data, and the sample height hierarchy table are all ground truth data.

[0069] Based on the above method embodiments, corresponding apparatus embodiments are provided; like Figure 2 As shown, one embodiment of the present invention provides a building height estimation device, comprising: The first basic data acquisition module acquires optical images and ground truth data of the sample buildings. The teacher model training module is used to take the optical image of the building as the input value and the ground truth data of the sample building as the label to train the preset building teacher model to obtain the target building teacher model, and obtain the teacher building multi-task prediction result based on the target building teacher model. The second basic data acquisition module is used to acquire sample building SAR images and sample radar system parameters of sample buildings; The student model training module is used to train the preset building student model by taking the sample building SAR image and sample radar system parameters as input values, and the teacher building multi-task prediction results and sample building ground truth data as labels, to obtain the target building student model. The data acquisition module is used to acquire the SAR image of the building to be detected and the parameters of the radar system to be detected. The student building multi-task prediction result generation module is used to input the SAR image of the building to be detected and the parameters of the radar system to be detected into the student model of the target building to obtain the student building multi-task prediction result. The refined height map acquisition module is used to obtain the refined height map of the building to be detected based on the multi-task prediction results of the student building.

[0070] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the building height estimation method provided by any of the above-described method embodiments of the present invention.

[0071] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0072] Based on the above-described embodiment of a building height estimation method, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a building height estimation method according to any embodiment of the present invention.

[0073] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0074] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0075] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0076] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute a building height estimation method as described in any of the above-described method embodiments of the present invention.

[0077] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0078] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for estimating building height, characterized in that, include: Obtain optical images and ground truth data of the sample buildings; The optical image of the building is used as the input value, and the ground truth data of the sample building is used as the label to train the preset building teacher model to obtain the target building teacher model, and the multi-task prediction result of the teacher building is obtained based on the target building teacher model. Acquire sample building SAR images and sample radar system parameters for the sample buildings; Using the sample building SAR image and sample radar system parameters as input values, and the teacher building multi-task prediction results and the sample building ground truth data as labels, the preset building student model is trained to obtain the target building student model. Acquire SAR images of the building to be detected and parameters of the radar system to be detected; The SAR image of the building to be detected and the parameters of the radar system to be detected are input into the student model of the target building to obtain the multi-task prediction result of the student building. Based on the multi-task prediction results of the student building, a detailed height map of the building to be tested is obtained.

2. The method for estimating building height according to claim 1, characterized in that, The acquisition of optical images and ground truth data of the sample buildings includes: Obtain the ground truth data of the sample building height, the ground truth data of the sample building binarized segmentation, and the sample height hierarchy table of the sample building; Based on the sample height hierarchy classification table, the true value data of building height is mapped hierarchically to obtain the sample hierarchy number; The sample level number, the sample building binarized segmentation ground value, and the sample building height ground value are used as the sample building ground value data.

3. The method for estimating building height according to claim 2, characterized in that, The process of obtaining the sample building height ground value data, the sample building binarized segmentation ground value, and the sample height hierarchy table includes: Obtain the ground truth data of the sample building height and the ground truth data of the sample building binarized segmentation; Obtain the maximum and minimum values ​​of the sample buildings; The sample height level division table is obtained based on the maximum value of the sample buildings, the minimum value of the sample buildings, and the preset total number of sample levels.

4. A method for estimating building height according to claim 2 or 3, characterized in that, The step of using the optical image of the building as input and the ground truth data of the sample building as labels to train a preset architectural teacher model to obtain a target architectural teacher model, and obtaining multi-task prediction results of the teacher building based on the target architectural teacher model, includes: The optical image of the building is input into a preset building teacher model to obtain the teacher training level number, the teacher building binarized segmentation training value, and the teacher building height training probability. The teacher building height training data is obtained based on the teacher building height training probability. The first teacher building loss function is obtained based on the training values ​​of the binarized segmentation of the teacher building and the true values ​​of the binarized segmentation of the sample building. The second teacher building loss function is obtained based on the teacher training level number, the sample level number, and the preset height level weight; A third teacher building loss function is obtained based on the training data of the teacher building height and the ground value data of the sample building height. Construct a target teacher building loss function based on the first teacher building loss function, the second teacher building loss function, and the third teacher building loss function; The target teacher model is obtained by training the preset teacher model based on the target teacher building loss function, and the teacher building multi-task prediction results are obtained based on the target teacher building loss function.

5. The method for estimating building height according to claim 4, characterized in that, The process involves using the sample building SAR image and sample radar system parameters as input values, and the multi-task prediction results of the teacher building and the ground truth data of the sample building as labels to train a preset building student model, thereby obtaining a target building student model. This includes: The sample building SAR image and sample radar system parameters are input into the preset building student model to obtain the student training level number, the student building binarized segmentation training value and the student building height training data. The first student building loss function is obtained based on the training values ​​of the binarized segmentation of the student buildings and the true values ​​of the binarized segmentation of the sample buildings; The second student building loss function is obtained based on the student training level number, the sample level number, and the preset height level weight; A third student building loss function is obtained based on the student building height training data, the sample building height ground value data, and the teacher building height training probability. Construct a target student building loss function based on the first student building loss function, the second student building loss function, and the third student building loss function; The target student building model is obtained by training the preset student building model based on the target student building loss function, and the multi-task prediction results of student buildings are obtained based on the target student building model.

6. The method for estimating building height according to claim 1, characterized in that, The multi-task prediction result for student buildings includes a target building outline mask, a target level classification map, and candidate height values ​​for each target level. The process of obtaining a refined height map of the building to be detected based on the multi-task prediction result includes: Based on the target level attribution map and the height candidate values ​​for each target level, an initial assembly height map is obtained; Based on the target building outline mask and the initial height map, the refined height map is obtained.

7. The method for estimating building height according to claim 6, characterized in that, The process of obtaining the initial height map based on the target level attribution map and the height candidate values ​​for each target level includes: Obtain the pixel coordinates and level index of each pixel in the target level attribution map; Based on the hierarchical number index, the corresponding index height channel is selected from the candidate height values ​​of each level of the target; An initial height value is selected from the index height channel based on the pixel coordinates; The initial stitching height map is obtained based on the initial height value corresponding to all the pixels.

8. A building height estimation device, characterized in that, include: The first basic data acquisition module acquires optical images and ground truth data of the sample buildings. The teacher model training module is used to take the optical image of the building as the input value and the ground truth data of the sample building as the label to train the preset building teacher model to obtain the target building teacher model, and obtain the teacher building multi-task prediction result based on the target building teacher model. The second basic data acquisition module is used to acquire sample building SAR images and sample radar system parameters of sample buildings; The student model training module is used to train the preset building student model by taking the sample building SAR image and sample radar system parameters as input values, and the teacher building multi-task prediction results and sample building ground truth data as labels, to obtain the target building student model. The data acquisition module is used to acquire the SAR image of the building to be detected and the parameters of the radar system to be detected. The student building multi-task prediction result generation module is used to input the SAR image of the building to be detected and the parameters of the radar system to be detected into the student model of the target building to obtain the student building multi-task prediction result. The refined height map acquisition module is used to obtain the refined height map of the building to be detected based on the multi-task prediction results of the student building.

9. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein, when the processor executes the computer program, it implements a building height estimation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a building height estimation method as described in any one of claims 1-7.

Images