Building height estimation method and device integrating multi-view building roof outlines

Abstract

The present invention provides a method and device for estimating the height of a building by integrating the roof outlines of a building from multiple perspectives, and relates to the technical field of building height estimation, including: extracting the roof outline of a satellite remote sensing image to obtain building roof extraction results of different perspectives; performing Hu invariant distance matching on the building roof extraction results; determining the unmatched roof targets in the building roof extraction results, and constructing an energy function based on the offsets corresponding to multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets; correcting the roof targets with adhesion; and estimating the height information corresponding to each building based on the offsets corresponding to each roof target. The present invention can effectively reduce the labor cost required for estimating the building height, while significantly improving the reliability of the building height, and has better generalization ability.

Description

Technical Field

[0001] The present invention relates to the technical field of building height estimation, and in particular to a method and device for estimating the building height by integrating multi-viewing angles of building roof outlines. Background Art

[0002] Currently, there are generally two methods for estimating the height of buildings in urban scenes based on satellite optical images. One is the traditional three-dimensional reconstruction method based on dense matching, and the other is the single-view image building height estimation method based on deep learning.

[0003] The 3D reconstruction method based on dense matching has been maturely applied in natural scenes, and can be roughly divided into steps such as stereo correction, dense matching, multi-view point cloud model fusion, and digital surface model generation. However, there are still some problems with the 3D reconstruction of satellite images in urban scenes: 1) Large-scale satellite stereo is relatively difficult to calibrate accurately. Unlike the commonly used array pinhole camera, the onboard optical sensor is generally a linear array push-broom type. During the push-broom imaging process, there are differences in the polar geometry between different linear arrays; 2) In urban scenes, the elevation of a large number of buildings is discontinuous, and weak texture areas such as flat ground and roofs limit the reliability of dense matching; 3) The 3D reconstruction method based on dense matching is computationally intensive and requires a lot of time in the matching process. Manual correction is required for poorly matched areas.

[0004] The single-view building height estimation method based on deep learning uses a deep learning model to directly regress the building height, which involves implicit learning of multiple clues such as building texture, tilt, covering relationship, size, shadow, etc. In practical applications, methods based on deep learning often have the following problems: 1) poor generalization ability. As one of the important clues in the building height regression model, building texture requires a large number of remote sensing image training in different regions to meet its adaptability to different textures; 2) it is more sensitive to building shadows. The presence or absence of shadows and the length of shadows may cause fluctuations in height estimation. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method and device for estimating the building height by integrating multi-perspective building roof contours, which can effectively reduce the labor cost required for estimating the building height, while significantly improving the reliability of the building height and having better generalization ability.

[0006] In a first aspect, an embodiment of the present invention provides a method for estimating building height by fusing multi-view building roof profiles, comprising:

[0007] Acquire satellite remote sensing images from multiple perspectives, and extract roof contours from the satellite remote sensing images to obtain building roof extraction results from different perspectives. The building roof extraction results include roof targets corresponding to each building displayed in the satellite remote sensing images;

[0008] Perform Hu invariant distance matching on the building roof extraction results from different perspectives to obtain multiple pairs of matching roof targets and their corresponding offsets;

[0009] Determine the unmatched roof targets in the building roof extraction results at different viewing angles, construct an energy function based on the offsets corresponding to multiple pairs of matched roof targets, and use it to determine the offsets corresponding to the unmatched roof targets, so as to obtain the initial roof matching results corresponding to the building roof extraction results at different viewing angles;

[0010] Correct the roof targets with adhesion in the initial roof matching results, and obtain the target roof matching results corresponding to the building roof extraction results at different viewing angles;

[0011] Based on the offset corresponding to each roof target in the target roof matching result, the height information corresponding to each building is estimated.

[0012] In one implementation, Hu invariant distance matching is performed on building roof extraction results from different perspectives to obtain multiple pairs of matched roof targets and their corresponding offsets, including:

[0013] The building roof extraction results of any two perspectives are recorded as the first building roof extraction result and the second building roof extraction result respectively;

[0014] For any first roof target in the first building roof extraction result, a matching range is constructed with the first roof target as the center, and a second roof target within the matching range is extracted from the second building roof extraction result;

[0015] Determine the Hu distance between the first rooftop target and the second rooftop target;

[0016] A second roof target whose Hu distance is less than a preset Hu distance threshold and whose area difference with the first roof target is within a preset range is determined to match the first roof target, and an offset between the matched first roof target and the second roof target is determined.

[0017] In one embodiment, after determining the offset between the matched first rooftop target and the second rooftop target, the method further includes:

[0018] Determine the mean and variance of the offset between the matched first roof target and the second roof target to determine the error range, and if the offset between the matched first roof target and the second roof target is within the error range, eliminate the matching relationship between the first roof target and the second roof target;

[0019] And, the first roof target is moved according to the offset, and if the intersection-and-union ratio between the moved first roof target and the second roof target matched with it is less than a preset intersection-and-union ratio threshold, the matching relationship between the first roof target and the second roof target is eliminated.

[0020] In one embodiment, unmatched roof targets in building roof extraction results from different viewing angles are determined, and an energy function is constructed based on offsets corresponding to multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, including:

[0021] Determine an unmatched first roof target from the first building roof extraction result;

[0022] Determine the maximum value, minimum value and average value of the offset between the matched first roof target and the second roof target, so as to determine the sliding matching range of the outer bounding box of the unmatched first roof target;

[0023] Slide the outer frame of the unmatched first roof target within the sliding matching range to obtain the sliding amount, and construct an energy function to take the weighted sum of the area intersection and union ratio and the edge intersection and union ratio between the unmatched first roof target and the sliding amount as the energy value corresponding to the sliding amount;

[0024] A plurality of candidate sliding amounts are screened out from the sliding amounts based on the energy values, and an offset corresponding to the unmatched first roof target is determined based on the candidate sliding amounts.

[0025] In one embodiment, after determining the offset corresponding to the unmatched first roof target based on the candidate sliding amount, the method further includes:

[0026] According to the offset amount, the first roof target in the first building roof extraction result is offset to obtain the offset first roof target;

[0027] For each offset first roof target, determine the IoF value between the offset first roof target and other offset first roof targets;

[0028] If the IoF value is greater than the first preset IoF threshold, the shifted first roof target or other shifted first roof targets are eliminated based on the edge intersection and union ratios corresponding to the shifted first roof target and other shifted first roof targets respectively.

[0029] In one embodiment, after determining the offset corresponding to the unmatched first roof target based on the candidate sliding amount, the method further includes:

[0030] From the first building roof extraction result, continue to determine the first roof target that is not matched;

[0031] Determine, based on the distance between the unmatched first roof target and the matched first roof target, an adjacent roof target corresponding to the unmatched first roof target;

[0032] The offset corresponding to the first rooftop object that is not matched is determined according to the offset corresponding to the rooftop outline object of the adjacent building.

[0033] In one implementation, the roof targets with adhesion in the initial roof matching results are corrected to obtain the target roof matching results corresponding to the building roof extraction results at different viewing angles, including:

[0034] According to the offset amount, the first roof target in the first building roof extraction result is offset to obtain the offset first roof target, and the second roof target covered by the offset first roof target is determined from the second building roof extraction result;

[0035] The IoF value between the first roof target after the offset and the second roof target covered by it is determined, and the correction strategy corresponding to the first roof target is determined based on the IoF value. The first roof target is corrected using the correction strategy to obtain the target roof matching result.

[0036] In one implementation, estimating the height information corresponding to each building based on the offset corresponding to each roof target in the target roof matching result includes:

[0037] Based on the height information of any building or the offset vector from the building roof to the ground in any satellite remote sensing image, combined with the imaging parameters of the satellite remote sensing image and the offset corresponding to each roof target in the target roof matching result, the height information corresponding to each building is estimated.

[0038] In a second aspect, an embodiment of the present invention further provides a building height estimation device integrating multi-view building roof profiles, comprising:

[0039] The roof extraction module is used to obtain satellite remote sensing images from multiple perspectives and extract roof contours from satellite remote sensing images to obtain building roof extraction results from different perspectives. The building roof extraction results include the roof targets corresponding to each building displayed in the satellite remote sensing images.

[0040] The first matching module is used to perform Hu invariant distance matching on the building roof extraction results from different perspectives to obtain multiple pairs of matched roof targets and their corresponding offsets;

[0041] The second matching module is used to determine the unmatched roof targets in the building roof extraction results of different viewing angles, and to construct an energy function based on the offsets corresponding to multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, so as to obtain the initial roof matching results corresponding to the building roof extraction results of different viewing angles;

[0042] The adhesion correction module is used to correct the roof targets with adhesion in the initial roof matching results, and obtain the target roof matching results corresponding to the building roof extraction results at different viewing angles;

[0043] The height estimation module is used to estimate the height information corresponding to each building based on the offset corresponding to each roof target in the target roof matching result.

[0044] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.

[0045] The method and device for estimating the building height by integrating the roof contours of a multi-view building provided by the embodiment of the present invention first obtain satellite remote sensing images of multiple viewpoints, extract the roof contours of the satellite remote sensing images to obtain building roof extraction results of different viewpoints, and the building roof extraction results include roof targets corresponding to each building displayed by the satellite remote sensing images; then Hu invariant distance matching is performed on the building roof extraction results of different viewpoints to obtain multiple pairs of matched roof targets and their corresponding offsets; then unmatched roof targets in the building roof extraction results of different viewpoints are determined, and an energy function is constructed based on the offsets corresponding to the multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, so as to obtain initial roof matching results corresponding to the building roof extraction results of different viewpoints; then the roof targets with adhesion in the initial roof matching results are corrected to obtain target roof matching results corresponding to the building roof extraction results of different viewpoints; finally, based on the offset corresponding to each roof target in the target roof matching results, the height information corresponding to each building is estimated. The above method obtains the target roof matching results corresponding to the building roof extraction results of different perspectives through three stages of building roof contour matching, which are mainly matching based on Hu invariant distance matching, matching based on energy function, and disassembling adhesion roof targets. Then, the offset corresponding to each roof target in the target roof matching result is used to realize the building height estimation. The embodiment of the present invention can effectively reduce the labor cost required for estimating the building height, while significantly improving the reliability of the building height, and has better generalization ability.

[0046] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A schematic flow chart of a method for estimating building height by fusing multi-view building roof profiles provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of mobile matching and sliding matching provided by an embodiment of the present invention;

[0051] Figure 3 An analysis diagram of the roof profile of a building from different viewing angles provided by an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of a building height estimation device integrating multi-view building roof profiles provided by an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] At present, the 3D reconstruction method based on dense matching has the following problems: 1) Large-scale satellite stereo is relatively difficult to calibrate accurately. Unlike the commonly used area array pinhole camera, the onboard optical sensor is generally a linear array push-broom type. During the push-broom imaging process, there are differences in the epipolar geometry between different linear arrays; 2) A large number of buildings in urban scenes are discontinuous in elevation, and weak texture areas such as flat ground and roofs limit the reliability of dense matching; 3) The 3D reconstruction method based on dense matching has the following problems: 1) Poor generalization ability. Building texture is one of the important clues in the building height regression model. It requires a large number of remote sensing image training in different regions to meet its adaptability to different textures; 2) It is more sensitive to building shadows. The presence or absence of shadows and the length of shadows may cause fluctuations in height estimation.

[0056] Based on this, the present invention provides a method and device for estimating the building height by integrating multi-perspective building roof contours, which can effectively reduce the labor cost required for estimating the building height, while significantly improving the reliability of the building height and having better generalization ability.

[0057] To facilitate understanding of this embodiment, firstly, a building height estimation method for integrating multi-view building roof profiles disclosed in an embodiment of the present invention is described in detail. Figure 1The flowchart of a method for estimating building height by fusing multi-view building roof profiles is shown, and the method mainly includes the following steps S102 to S110:

[0058] Step S102, acquiring satellite remote sensing images of multiple viewing angles, and extracting roof contours from the satellite remote sensing images to obtain building roof extraction results of different viewing angles.

[0059] The building roof extraction results include the roof targets corresponding to each building displayed in the satellite remote sensing image. In one example, the satellite remote sensing image is preprocessed to unify the image resolution, and high-precision building roof extraction results of dual-view or multi-view satellite remote sensing images are obtained. If there is high-precision annotation or open source data, the building roof extraction result of one of the views can be replaced.

[0060] Step S104, performing Hu invariant distance matching on the building roof extraction results from different viewing angles to obtain multiple pairs of matched roof targets and their corresponding offsets.

[0061] This stage is the matching based on Hu invariant distance matching, which can also be described as multi-view roof profile shape matching based on Hu invariant moment. In one example, for the building roof extraction results of any two views, the two sets of building roof extraction results are matched using Hu invariant distance, and unreliable matching results are eliminated through the 2sigma principle and the intersection-over-union ratio after alignment, so as to obtain multiple pairs of matched roof targets and their corresponding offsets.

[0062] Step S106, determine the unmatched roof targets in the building roof extraction results from different perspectives, construct an energy function based on the offsets corresponding to multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, and obtain the initial roof matching results corresponding to the building roof extraction results from different perspectives.

[0063] This stage is the matching based on the energy function, which can also be described as sliding matching based on the combined area intersection and edge intersection. In one example, sliding matching is performed based on the statistical results of the offsets corresponding to multiple pairs of matched roof targets, and an energy function is constructed based on the area intersection and edge intersection to determine the offsets corresponding to the unmatched roof targets, and the overlap after the offset is analyzed and corrected.

[0064] Step S108, correcting the roof targets with adhesion in the initial roof matching results, and obtaining target roof matching results corresponding to the building roof extraction results at different viewing angles.

[0065] This stage is to disassemble the adhered roof targets, which can also be described as disassembling the adhered building roof outlines by combining multi-view outlines. The initial roof matching results are further analyzed, and the adhered roof targets in the one-to-many matching case are disassembled into multiple non-adhesive roof targets, thereby obtaining the target roof matching results.

[0066] Step S110, estimating the height information corresponding to each building based on the offset corresponding to each roof target in the target roof matching result.

[0067] In one example, by combining the offset and imaging parameters corresponding to the roof target on the dual-view satellite remote sensing image, specifying the height of any building or the offset vector from the roof of the building to the foundation on any image, the building height can be calculated and a three-dimensional building model can be constructed.

[0068] The method for estimating the building height by integrating the multi-perspective building roof contours provided by the embodiment of the present invention obtains the target roof matching results corresponding to the building roof extraction results of different perspectives through three-stage building roof contour matching, which mainly includes three stages of multi-perspective roof contour shape matching based on Hu invariant moment, sliding matching of joint area intersection and edge intersection, and joint multi-perspective contour disassembly of the adhered building roof contours. Then, the offset corresponding to each roof target in the target roof matching result is used to realize the building height estimation. The embodiment of the present invention can effectively reduce the labor cost required for estimating the building height, while significantly improving the reliability of the building height, and has better generalization ability.

[0069] For ease of understanding, an embodiment of the present invention provides a specific implementation method of a building height estimation method that integrates multi-perspective building roof contours, which can cope with a variety of complex situations and achieve matching and height estimation of building roof contours under multiple perspectives.

[0070] For the aforementioned step S102, the embodiment of the present invention provides a specific implementation method of acquiring satellite remote sensing images from multiple perspectives, extracting roof contours from satellite remote sensing images, and obtaining building roof extraction results from different perspectives, including steps 1.1 to 1.4:

[0071] Step 1.1, firstly, the multispectral data and panchromatic data of the satellite remote sensing image need to be fused, and then the satellite remote sensing image needs to be orthorectified using RPC parameters and external DEM data to eliminate terrain deformation and ensure that the position of the building foundation outline is consistent under different viewing angles, so that the subsequent roof outline distance under different viewing angles is meaningful; then, radiation calibration and atmospheric correction are performed to improve the quantitative analysis capability of each satellite remote sensing image and the comparability between images;

[0072] Step 1.2: The subsequent matching process is carried out on the raster results, and its unit is pixel. In order to ensure that the pixel size of each building on different satellite remote sensing images is basically the same, all satellite remote sensing images need to be sampled to the same resolution;

[0073] Step 1.3, input the pre-processed dual-view / multi-view satellite remote sensing image into the building roof segmentation model to obtain the building roof extraction results of different viewpoints. If there is high-precision annotation or open source data, the extraction result of one viewpoint can be replaced. In the embodiment of the present invention, a high-precision building roof extraction result is obtained by using a deep learning network model for building roof extraction;

[0074] Step 1.4, convert the extracted building roofs from different perspectives into raster format to facilitate calculation in the subsequent matching process.

[0075] For the aforementioned step S104, the embodiment of the present invention provides a specific implementation method for performing Hu invariant distance matching on the building roof extraction results from different viewing angles to obtain multiple pairs of matched roof targets and their corresponding offsets, and adopts shape matching based on Hu moments to quickly obtain matching pairs of roof targets with similar contours under different viewing angles, which, on the one hand, reduces the number of contours that need to be matched subsequently, and on the other hand, provides an adaptive sliding matching range for subsequent sliding matching, reducing the amount of sliding matching calculations. For details, please refer to the following steps 2.1 to 2.7:

[0076] In step 2.1, morphological opening operations are performed on the building roof extraction results at different perspectives to reduce the small connections between the contours during building detection and try to avoid adhesion between two roof targets.

[0077] Step 2.2: The building roof extraction results from any two viewing angles are recorded as the first building roof extraction result and the second building roof extraction result, respectively.

[0078] In one example, two groups of building roof extraction results are taken out, which are respectively recorded as the first building roof extraction result A and the second building roof extraction result B. In order to facilitate subsequent calculations, the bounding box range and area of ​​each contour are also recorded.

[0079] Step 2.3: for any first roof target in the first building roof extraction result, construct a matching range with the first roof target as the center, and extract a second roof target within the matching range from the second building roof extraction result.

[0080] In one example, a first roof target a in the first building roof extraction result A is taken out, and all second roof targets b within a certain range of the first roof target a are taken out in the second building roof extraction result B. The certain range here is defined in the embodiment of the present invention as a range that is twice the length and width of the outer bounding box of the first roof target a. The Hu moment of the first roof target a and all second roof targets b is calculated. The Hu image moment is invariant to translation, rotation, and scaling, and has a relatively stable feature representation in shape matching.

[0081] Step 2.4, determine the Hu distance between the first rooftop target and the second rooftop target.

[0082] In one example, the distance of the Hu moment between the first rooftop object a and each second rooftop object b is calculated, and the smaller the distance, the higher the similarity.

[0083] Step 2.5, determine that a second roof target whose Hu distance is less than a preset Hu distance threshold and whose area difference with the first roof target is within a preset range matches the first roof target, and determine the offset between the matched first roof target and the second roof target.

[0084] For example, the preset Hu distance threshold is 0.5. In one example, when the Hu distance is less than the threshold 0.5, and the area difference between the first roof target a and the second roof target b is between 0.5 and 1.5 times, the second roof target b that is closest to the first roof target a is selected for matching, and the offset is calculated. and .

[0085] Step 2.6, determine the mean and variance of the offset between the matched first roof target and the second roof target to determine the error range. If the offset between the matched first roof target and the second roof target is within the error range, the matching relationship between the first roof target and the second roof target is eliminated.

[0086] In one example, iteratively execute steps 2.4 to 2.5 until all first roof targets a in the first building roof extraction result A are matched, and the mean of all offsets is calculated. and variance Then, using the 2sigma principle, we can eliminate the offsets that are beyond the error range. matching relationship.

[0087] Step 2.7, move the first roof target according to the offset. If the intersection-and-union ratio between the moved first roof target and its matching second roof target is less than a preset intersection-and-union ratio threshold, the matching relationship between the first roof target and the second roof target is eliminated.

[0088] For example, the preset intersection-and-union ratio threshold is 0.6. In one example, after the first rooftop target a is moved according to the offset, the intersection-and-union ratio is calculated with the matching second rooftop target b. If the intersection-and-union ratio is lower than the threshold 0.6, the matching relationship is eliminated.

[0089] For the aforementioned step S106, an embodiment of the present invention provides a specific implementation method for determining unmatched roof targets in the building roof extraction results from different perspectives, and constructing an energy function based on the offsets corresponding to multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets.

[0090] Due to the problem of overlapping roof outlines of buildings of different heights, shape matching cannot provide good matching results for combined shapes. Therefore, consider offsetting the roof target from one perspective in different directions until an offset is found so that it has a greater overlap with the roof outline from another perspective. However, such an operation requires a lot of time to move the target position. Using a window as large as the roof target to slide on the result of another perspective to simulate the movement process can greatly reduce the amount of calculation. At the same time, using the offset statistics of Hu shape matching to limit the range of sliding can further reduce the amount of operation. See steps 3.1 to 3.7 below for details:

[0091] Step 3.1, determining the first roof target that is not matched from the first building roof extraction result.

[0092] Step 3.2, determining the maximum value, minimum value and average value of the offset between the matched first roof target and the second roof target, so as to determine the sliding matching range of the outer bounding box of the unmatched first roof target.

[0093] In one example, the maximum offset, minimum offset, and average offset of the first rooftop target and the second rooftop target in the x and y directions are counted;

[0094] In one example, the sliding matching range of the sliding matching is determined, and a certain range is expanded based on the maximum and minimum offsets:

[0095] ;

[0096] ;

[0097] in, is the sliding matching range in the x direction, is the proportional coefficient of the expansion range, here it is 0.2, and similarly the sliding matching range in the y direction can be obtained .

[0098] Step 3.3, slide the outer frame of the unmatched first roof target within the sliding matching range to obtain the sliding amount, and construct an energy function to take the weighted sum of the area intersection and union ratio and the edge intersection and union ratio between the unmatched first roof target and the sliding amount as the energy value corresponding to the sliding amount.

[0099] In one example, see Figure 2 Specifically, the first roof target a that has not been matched in the first building roof extraction result A is taken out, and its bounding box range is , move the bounding box of the first roof target a within the sliding range:

[0100] ;

[0101] in, , , remove the range Result A of the extraction of the roof of the first building in ,as well as Sliding amount within ;

[0102] In one example, an energy function is constructed to calculate and The area intersection ratio and the marginal intersection-over-union ratio The weighted sum of:

[0103] ;

[0104] in, is the energy value, is the weight coefficient, here it is 1, and is defined as follows:

[0105] ;

[0106] ;

[0107] in, It refers to taking All contour surfaces Width of the profile, is 1% of the current window diagonal length, Refers to calculating area.

[0108] Step 3.4: Filter out a plurality of candidate sliding amounts from the sliding amounts based on the energy values, and determine the offset corresponding to the first roof target that is not matched based on the candidate sliding amounts.

[0109] In one example, step 3.3 is iterated until the entire sliding matching range is traversed, and the five groups of sliding amounts corresponding to the largest energy values ​​are selected. The five groups of offsets are clustered using the DBSCAN algorithm, and the sliding amount that is closer to the average offset is selected as the offset of the first roof target a.

[0110] Step 3.5: offset the first roof target in the first building roof extraction result according to the offset amount to obtain the offset first roof target.

[0111] In one example, steps 3.4 to 3.5 are iteratively executed until the remaining first roof targets a in the first building roof extraction result A are matched, and all first roof targets a in the first building roof extraction result A are offset according to the offset to obtain the offset first building roof extraction result A'.

[0112] Step 3.6, for each offset first roof target, determine the IoF value between the offset first roof target and other offset first roof targets; if the IoF value is greater than the first preset IoF threshold, based on the edge intersection and union ratio corresponding to each of the offset first roof target and other offset first roof targets, the offset first roof target or other offset first roof targets are eliminated.

[0113] In one example, the IoF of each first roof target a' after offset and other contours in the first building roof extraction result A' after offset is calculated, that is, the ratio of the intersection of the area of ​​the first roof target a' after offset and other contours to its own area. If the IoF of the first roof target a' after offset and a certain contour is greater than the threshold value 0.6, the first roof target a' after offset and a certain contour are removed. Lower goals.

[0114] Step 3.7, continue to determine the unmatched first roof target from the first building roof extraction result; based on the distance between the unmatched first roof target and the matched first roof target, determine the adjacent roof target corresponding to the unmatched first roof target; according to the offset corresponding to the adjacent building roof outline target, determine the offset corresponding to the unmatched first roof target.

[0115] In one example, for the first rooftop target a that has not been matched yet, the first building rooftop extraction result A is used to find a neighboring rooftop target whose distance from the first rooftop target a is less than a threshold. If the offset difference between several neighboring rooftop targets is less than a threshold of 10, the offset of the nearest neighboring rooftop target is directly applied; otherwise, the average offset is used.

[0116] For the aforementioned step S108, the embodiment of the present invention provides a specific implementation method for correcting the roof targets with adhesion in the initial roof matching results to obtain the target roof matching results corresponding to the building roof extraction results at different viewing angles.

[0117] At different viewing angles, the different building heights lead to different situations of building roof adhesion in satellite remote sensing images. Roofs that are adhered in one satellite remote sensing image may be separated at another viewing angle. By comparing the building roof extraction results at different viewing angles, the adhered roof targets can be disassembled into multiple non-adhesive roof targets, which is more reasonable in height estimation and 3D reconstruction. For details, please refer to the following steps 4.1 to 4.2:

[0118] Step 4.1, according to the offset amount, offset the first roof target in the first building roof extraction result to obtain the offset first roof target, and determine the second roof target covered by the offset first roof target from the second building roof extraction result.

[0119] In one example, all first roof targets a in the first building roof extraction result A are offset according to the offset to obtain the offset first building roof extraction result A', and the second roof targets b in the second building roof extraction result B covered by each offset first roof target a' in the offset first building roof extraction result A' are found.

[0120] Step 4.2, determine the IoF value between the first roof target after offset and the second roof target it covers, determine the correction strategy corresponding to the first roof target based on the IoF value, use the correction strategy to correct the first roof target, and obtain the target roof matching result.

[0121] In one example, the ratio of the coverage area of ​​the first roof target a' to its own area after each second roof target b is offset is calculated, and the second roof target b with an IoF greater than a threshold of 0.8 is marked as , the second roof target b marked with an IoF greater than the limit value 0.2 and less than the threshold value 0.8 is .

[0122] Based on this, see Figure 3 The following is an analysis diagram of the roof profile of a building from different perspectives. Figure 3 The following process is illustrated:

[0123] (1) If all The area ratio of the first roof target a' after the union of the offset exceeds the threshold value of 0.8, that is, , then Move in the opposite direction according to the offset to replace the first roof target a;

[0124] (2) If the new first roof target a'' is equal to The area occupied by the intersection of a'' exceeds the limit of 0.2, that is, , and the new first roof target a'' is combined with The intersection of is moved in the opposite direction according to the offset of a'' to replace the corresponding contour in A.

[0125] The process of determining the new first roof target a'' is as follows: after calculating the offset, the first roof target a' and The difference of the offset replaces the first roof target a' and finds The new first roof target a'' in A' covered by the profile;

[0126] (3) If all The area of ​​the first roof target a' after the union of the occupied offset does not exceed the threshold value 0.8, that is, ,calculate The intersection with the offset first rooftop target a' is moved in the reverse direction to replace the first rooftop target a.

[0127] For the aforementioned step S110, an embodiment of the present invention provides a specific implementation method for estimating the height information corresponding to each building based on the offset corresponding to each roof target in the target roof matching result. The height information corresponding to each building can be estimated based on the height information of any building or the building roof to foundation offset vector in any satellite remote sensing image, combined with the imaging parameters of the satellite remote sensing image and the offset corresponding to each roof target in the target roof matching result.

[0128] In practical applications, the offset from the building roof outline to the foundation outline has the following geometric relationship with the building height and imaging angle:

[0129] ;

[0130] is the building height, is the image incident angle, is the image resolution, usually in meters / pixel. is the angle between the offset vector and the x-axis of the pixel coordinate. , And unified resolution Known.

[0131] On two satellite remote sensing images with different imaging angles that have been orthorectified, the position of the foundation is the same, and the position of the roof outline is different, which can form two sets of offset vectors from the roof to the foundation and , the subtraction of the two vectors is the offset of the roof on the two satellite remote sensing images:

[0132]

[0133] The modulus of this offset can be expressed as:

[0134] ;

[0135] To simplify the expression, , , after simplifying the above formula:

[0136] ;

[0137] in, , , All are known. It can be obtained by steps S102-S108, as long as we know , we can solve the building height. For details, please refer to steps 5.1 to 5.2 below:

[0138] Step 5.1, specify the height of any building or the offset vector from the roof to the foundation of any building in the image, and calculate the unknown quantity ;

[0139] Step 5.2: Based on the offsets of the roof target under the two viewing angles obtained in steps S102-S108, the building height can be estimated:

[0140] ;

[0141] When there are building roof extraction results from more than two perspectives, the two-by-two combinations can form multiple pairs of roof profile offset values, which can improve the accuracy of the estimated building height on average.

[0142] In summary, there are now a large number of high-precision building roof outline extraction models with good generalization ability, which can provide effective information for the building height estimation task by combining the building roof outlines under different viewing angles. Therefore, the present invention provides a building height estimation method that integrates multi-view building roof outlines in view of the technical problems and practical application problems described in the background technology. The embodiment of the present invention applies the existing high-precision building roof extraction model to multi-view satellite images, obtains the roof outline offsets of different viewing angles and estimates the building height through three-stage building roof outline matching, mainly including: multi-view roof outline shape matching based on Hu invariant moments, sliding matching of combined area intersection and edge intersection, joint multi-view outline disassembly of adhered building roof outlines, and estimation of building height offset based on paired multi-view roof outlines. The process method described in the embodiment of the present invention can achieve large-scale, fast and automated building height estimation.

[0143] Specifically, the embodiments of the present invention have at least the following characteristics: 1. Utilize the difference of roof outlines from multiple sources to estimate the building height. That is, the embodiments of the present invention fully utilize the relationship between the difference of roof outline positions in multi-view images and the building height estimation, use a mature building roof segmentation model to obtain multi-view high-precision roof outlines, obtain the roof outline offset between different views through three-stage multi-view roof matching, and quickly obtain building height information by combining imaging parameters. 2. Mutual calibration of multi-source data is completed through multi-level matching.

[0144] Based on the above-mentioned embodiments, the present invention provides a device for estimating building height by fusing multi-view building roof profiles. Figure 4 The structure diagram of a building height estimation device integrating multi-view building roof profiles is shown in FIG. 1 , and the device mainly includes the following parts:

[0145] The roof extraction module 402 is used to obtain satellite remote sensing images of multiple perspectives, and extract the roof contours of the satellite remote sensing images to obtain building roof extraction results of different perspectives, wherein the building roof extraction results include the roof target corresponding to each building displayed in the satellite remote sensing images;

[0146] The first matching module 404 is used to perform Hu invariant distance matching on the building roof extraction results of different viewing angles to obtain multiple pairs of matched roof targets and their corresponding offsets;

[0147] The second matching module 406 is used to determine the unmatched roof targets in the building roof extraction results of different viewing angles, and construct an energy function based on the offsets corresponding to the multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, so as to obtain the initial roof matching results corresponding to the building roof extraction results of different viewing angles;

[0148] The adhesion correction module 408 is used to correct the roof targets with adhesion in the initial roof matching results, and obtain the target roof matching results corresponding to the building roof extraction results of different viewing angles;

[0149] The height estimation module 410 is used to estimate the height information corresponding to each building based on the offset corresponding to each roof target in the target roof matching result.

[0150] The building height estimation device for integrating multi-perspective building roof contours provided by the embodiment of the present invention obtains target roof matching results corresponding to building roof extraction results of different perspectives through three-stage building roof contour matching, which mainly includes three stages of matching based on Hu invariant distance matching, matching based on energy function, and disassembling adhesion roof targets, and then uses the offset corresponding to each roof target in the target roof matching result to realize building height estimation. The embodiment of the present invention can effectively reduce the labor cost required for estimating the building height, while significantly improving the reliability of the building height, and has better generalization ability.

[0151] In one implementation, the first matching module 404 is specifically configured to:

[0152] The building roof extraction results of any two perspectives are recorded as the first building roof extraction result and the second building roof extraction result respectively;

[0153] For any first roof target in the first building roof extraction result, a matching range is constructed with the first roof target as the center, and a second roof target within the matching range is extracted from the second building roof extraction result;

[0154] Determine the Hu distance between the first rooftop target and the second rooftop target;

[0155] A second roof target whose Hu distance is less than a preset Hu distance threshold and whose area difference with the first roof target is within a preset range is determined to match the first roof target, and an offset between the matched first roof target and the second roof target is determined.

[0156] In one implementation, the first matching module 404 is further configured to:

[0157] Determine the mean and variance of the offset between the matched first roof target and the second roof target to determine the error range, and if the offset between the matched first roof target and the second roof target is within the error range, eliminate the matching relationship between the first roof target and the second roof target;

[0158] And, the first roof target is moved according to the offset, and if the intersection-and-union ratio between the moved first roof target and the second roof target matched with it is less than a preset intersection-and-union ratio threshold, the matching relationship between the first roof target and the second roof target is eliminated.

[0159] In one implementation, the second matching module 406 is specifically configured to:

[0160] Determine an unmatched first roof target from the first building roof extraction result;

[0161] Determine the maximum value, minimum value and average value of the offset between the matched first roof target and the second roof target, so as to determine the sliding matching range of the outer bounding box of the unmatched first roof target;

[0162] Slide the outer frame of the unmatched first roof target within the sliding matching range to obtain the sliding amount, and construct an energy function to take the weighted sum of the area intersection and union ratio and the edge intersection and union ratio between the unmatched first roof target and the sliding amount as the energy value corresponding to the sliding amount;

[0163] A plurality of candidate sliding amounts are screened out from the sliding amounts based on the energy values, and an offset corresponding to the unmatched first roof target is determined based on the candidate sliding amounts.

[0164] In one implementation, the second matching module 406 is further configured to:

[0165] According to the offset amount, the first roof target in the first building roof extraction result is offset to obtain the offset first roof target;

[0166] For each offset first roof target, determine the IoF value between the offset first roof target and other offset first roof targets;

[0167] If the IoF value is greater than the first preset IoF threshold, the shifted first roof target or other shifted first roof targets are eliminated based on the edge intersection and union ratios corresponding to the shifted first roof target and other shifted first roof targets respectively.

[0168] In one implementation, the second matching module 406 is further configured to:

[0169] From the first building roof extraction result, continue to determine the first roof target that is not matched;

[0170] Determine, based on the distance between the unmatched first roof target and the matched first roof target, an adjacent roof target corresponding to the unmatched first roof target;

[0171] The offset corresponding to the first rooftop object that is not matched is determined according to the offset corresponding to the rooftop outline object of the adjacent building.

[0172] In one implementation, the adhesion correction module 408 is specifically configured to:

[0173] According to the offset amount, the first roof target in the first building roof extraction result is offset to obtain the offset first roof target, and the second roof target covered by the offset first roof target is determined from the second building roof extraction result;

[0174] The IoF value between the first roof target after the offset and the second roof target covered by it is determined, and the correction strategy corresponding to the first roof target is determined based on the IoF value. The first roof target is corrected using the correction strategy to obtain the target roof matching result.

[0175] In one implementation, the height estimation module 410 is specifically configured to:

[0176] Based on the height information of any building or the offset vector from the building roof to the ground in any satellite remote sensing image, combined with the imaging parameters of the satellite remote sensing image and the offset corresponding to each roof target in the target roof matching result, the height information corresponding to each building is estimated.

[0177] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0178] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0179] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 50, a memory 51, a bus 52 and a communication interface 53, wherein the processor 50, the communication interface 53 and the memory 51 are connected via the bus 52; the processor 50 is used to execute an executable module stored in the memory 51, such as a computer program.

[0180] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A building height estimation method integrating multi-view building roof profiles, characterized in that: include: Acquire satellite remote sensing images of multiple perspectives, and extract roof contours from the satellite remote sensing images to obtain building roof extraction results of different perspectives, wherein the building roof extraction results include a roof target corresponding to each building displayed in the satellite remote sensing images; Performing Hu invariant distance matching on the building roof extraction results at different viewing angles to obtain multiple pairs of matched roof targets and their corresponding offsets; Determine the unmatched roof targets in the building roof extraction results at different viewing angles, construct an energy function based on the offsets corresponding to the multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, and obtain initial roof matching results corresponding to the building roof extraction results at different viewing angles; Correcting the roof target with adhesion in the initial roof matching result to obtain the target roof matching results corresponding to the building roof extraction results at different viewing angles; Based on the offset corresponding to each of the roof targets in the target roof matching result, estimating the height information corresponding to each of the buildings; Determining the unmatched roof targets in the building roof extraction results at different viewing angles, and constructing an energy function based on the offsets corresponding to the multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, including: The building roof extraction results of any two viewing angles are recorded as a first building roof extraction result and a second building roof extraction result respectively; Determining an unmatched first roof target from the first building roof extraction result; Determine the maximum value, minimum value and average value of the offset between the matched first roof target and the second roof target, so as to determine the sliding matching range of the outer bounding box of the first roof target that is not matched; wherein the second roof target is a roof target in the second building roof extraction result that is within the matching range constructed with the first roof target as the center; Sliding the outer frame of the first roof target that is not matched within the sliding matching range to obtain a sliding amount, and constructing an energy function to use a weighted sum of an area intersection-and-union ratio and an edge intersection-and-union ratio between the first roof target that is not matched and the sliding amount as an energy value corresponding to the sliding amount; A plurality of candidate sliding amounts are screened out from the sliding amounts based on the energy value, and an offset corresponding to the first roof target that is not matched is determined based on the candidate sliding amounts.

2. The building height estimation method by fusing multi-view building roof profiles according to claim 1, characterized in that: Performing Hu invariant distance matching on the building roof extraction results from different viewing angles to obtain multiple pairs of matched roof targets and their corresponding offsets, including: For any first roof target in the first building roof extraction result, construct a matching range with the first roof target as the center, and extract a second roof target within the matching range from the second building roof extraction result; Determining a Hu distance between the first rooftop object and the second rooftop object; The second roof target, whose Hu distance is less than a preset Hu distance threshold and whose area difference with the first roof target is within a preset range, is determined to match the first roof target, and an offset between the matched first roof target and the second roof target is determined.

3. The building height estimation method of fusing multi-view building roof profiles according to claim 2, characterized in that: After determining the offset between the matched first roof target and the second roof target, the method further includes: Determine a mean and a variance of an offset between the first roof target and the second roof target that are matched, so as to determine an error range, and if the offset between the first roof target and the second roof target that are matched is within the error range, eliminate the matching relationship between the first roof target and the second roof target; And, the first roof target is moved according to the offset, and if the intersection-and-union ratio between the moved first roof target and its matching second roof target is less than a preset intersection-and-union ratio threshold, the matching relationship between the first roof target and the second roof target is eliminated.

4. The building height estimation method by fusing multi-view building roof profiles according to claim 1, characterized in that: After determining the offset corresponding to the first roof target that is not matched based on the candidate sliding amount, the method further includes: offsetting the first roof target in the first building roof extraction result according to the offset amount to obtain the offset first roof target; For each of the first rooftop targets after shifting, determining an IoF value between the first rooftop target after shifting and other first rooftop targets after shifting; If the IoF value is greater than a first preset IoF threshold, the shifted first roof target or other shifted first roof targets are eliminated based on the edge intersection-over-combination ratios corresponding to the shifted first roof target and other shifted first roof targets.

5. The building height estimation method by fusing multi-view building roof profiles according to claim 1, characterized in that: After determining the offset corresponding to the first roof target that is not matched based on the candidate sliding amount, the method further includes: Continue to determine the first roof targets that are not matched from the first building roof extraction results; Determining a neighboring rooftop target corresponding to the unmatched first rooftop target based on a distance between the unmatched first rooftop target and the matched first rooftop target; The offset corresponding to the first rooftop object that is not matched is determined according to the offset corresponding to the adjacent rooftop object.

6. The building height estimation method by fusing multi-view building roof profiles according to claim 2, characterized in that: Correcting the roof target with adhesion in the initial roof matching result to obtain the target roof matching result corresponding to the building roof extraction result at different viewing angles includes: offsetting the first roof target in the first building roof extraction result according to the offset amount to obtain the offset first roof target, and determining the second roof target covered by the offset first roof target from the second building roof extraction result; Determine an IoF value between the offset first roof target and the second roof target covered by it, determine a correction strategy corresponding to the first roof target based on the IoF value, and use the correction strategy to correct the first roof target to obtain a target roof matching result.

7. The building height estimation method by fusing multi-view building roof profiles according to claim 1, characterized in that: Based on the offset corresponding to each of the roof targets in the target roof matching result, estimating the height information corresponding to each of the buildings, including: Based on the height information of any of the buildings or the building roof-to-ground offset vector in any of the satellite remote sensing images, combined with the imaging parameters of the satellite remote sensing image and the offset corresponding to each of the roof targets in the target roof matching result, the height information corresponding to each of the buildings is estimated.

8. A building height estimation device integrating multi-view building roof outlines, characterized in that: include: A roof extraction module is used to obtain satellite remote sensing images of multiple viewing angles, and to extract roof contours from the satellite remote sensing images to obtain building roof extraction results of different viewing angles, wherein the building roof extraction results include roof targets corresponding to each building displayed in the satellite remote sensing images; The first matching module is used to perform Hu invariant distance matching on the building roof extraction results at different viewing angles to obtain multiple pairs of matched roof targets and their corresponding offsets; A second matching module is used to determine the unmatched roof targets in the building roof extraction results at different viewing angles, and to construct an energy function based on the offsets corresponding to the multiple pairs of matched roof targets to determine the offsets corresponding to the unmatched roof targets, so as to obtain initial roof matching results corresponding to the building roof extraction results at different viewing angles; An adhesion correction module is used to correct the roof target with adhesion in the initial roof matching result, and obtain the target roof matching results corresponding to the building roof extraction results at different viewing angles; A height estimation module, used for estimating the height information corresponding to each of the buildings based on the offset corresponding to each of the roof targets in the target roof matching result; The second matching module is specifically used for: The building roof extraction results of any two viewing angles are recorded as a first building roof extraction result and a second building roof extraction result respectively; Determining an unmatched first roof target from the first building roof extraction result; Determine the maximum value, minimum value and average value of the offset between the matched first roof target and the second roof target, so as to determine the sliding matching range of the outer bounding box of the first roof target that is not matched; wherein the second roof target is a roof target in the second building roof extraction result that is within the matching range constructed with the first roof target as the center; Sliding the outer frame of the first roof target that is not matched within the sliding matching range to obtain a sliding amount, and constructing an energy function to use a weighted sum of an area intersection-and-union ratio and an edge intersection-and-union ratio between the first roof target that is not matched and the sliding amount as an energy value corresponding to the sliding amount; A plurality of candidate sliding amounts are screened out from the sliding amounts based on the energy value, and an offset corresponding to the first roof target that is not matched is determined based on the candidate sliding amounts.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.

Images