Method and system for building reconstruction of single-view orthographic image of lod2 level and storage medium

Abstract

The application discloses a single-view orthographic image LOD2-level building reconstruction method and system and a storage medium, and the method comprises the following steps: inputting a single-view orthographic remote sensing image into a shared backbone network to generate a footprint mask probability graph; constructing a first modulation factor, inputting a multi-scale feature graph into a height decoder after element-by-element gating, and outputting a height field; normalizing the height field to obtain a normalized height graph, and constructing a second modulation factor; fusing the second modulation factor and the first modulation factor to form a composite modulation factor, inputting the multi-scale feature graph into a structure line decoder after double element-by-element gating, and outputting a distance field and a direction angle field; recovering a roof structure line based on the distance field and the direction angle field; combining the roof structure line, the height field and a footprint boundary to construct a roof surface, a wall surface and a ground surface, and generating a building LOD2 three-dimensional model; and the application can automatically generate a topologically complete and geometrically accurate LOD2 three-dimensional model only by relying on a single orthographic image.

Description

Technical Field

[0001] This invention relates to the field of building reconstruction technology, specifically to a method, system, and storage medium for LOD2 level building reconstruction based on single-view orthophotos. Background Technology

[0002] With the rapid development of 3D real-scene China construction, digital twin cities, and smart cities, 3D building models play a crucial role in urban planning, real estate management, emergency simulation, and visualization. The Level of Detail (LOD) of 3D building models is divided into five levels, from LOD0 to LOD4. LOD1 models only represent the approximate volume and height of a building as flat-top blocks, failing to reflect roof morphology. LOD2 models can not only represent the building's facade walls, roof surface, and ground reference plane, but also reflect the building's roof structure, including various types such as pitched roofs, gable roofs, and hipped roofs, along with their corresponding structural line elements such as ridge lines and eaves lines. However, most existing 3D building reconstruction methods rely on multi-source auxiliary data such as multi-view oblique photography, airborne LiDAR point clouds, and digital surface models (DSM), which are insufficient to meet the urgent needs for low-cost and high-efficiency 3D modeling of large-scale urban buildings.

[0003] In recent years, single-view remote sensing imagery building reconstruction methods based on deep learning have gradually emerged. However, automated reconstruction at the LOD2 level still faces the following core technical challenges: First, multi-task prediction lacks collaborative constraints, resulting in poor spatial consistency. LOD2 reconstruction requires the simultaneous acquisition of three types of information: building footprint, height, and roof structure line. Strict spatial geometric constraints exist among these three elements—the height field should only be valid within the footprint region, and the structure line should be located inside the building and correspond to elevation changes. However, most existing methods model these three elements as independent sub-tasks, lacking information transfer and constraint mechanisms between sub-networks. This leads to height estimation overflowing the footprint boundary, structure lines deviating from the building outline, and topological defects such as wall penetration, roof overhang, and structural incompleteness during assembling the 3D model.

[0004] Secondly, the conversion from pixel-level prediction to LOD2 geometric models lacks a complete technical chain. A systematic solution is still lacking for how to transform pixel-level footprint masks, height fields, and structure line predictions into topologically consistent 3D geometric models that meet the CityGML LOD2 standard. Existing methods largely rely on matching predefined roof type template libraries, which lacks generalization ability for atypical roof shapes and lacks a unified process in areas such as roof surface topology partitioning, shared vertex elevation consistency processing, and automatic wall generation. Summary of the Invention

[0005] To address the aforementioned technical problems in existing technologies, this invention provides a method, system, and storage medium for reconstructing LOD2 level buildings using single-view orthophotos. By using a single orthophoto remote sensing image, spatial consistency constraints between footprint, height, and structural lines are achieved, automatically generating topologically complete and geometrically accurate LOD2 building 3D models.

[0006] According to a first aspect of the present invention, a method for reconstructing LOD2 level buildings from single-view orthophotos is provided, the method comprising: S1: Input single-view orthophoto remote sensing images into a shared backbone network and extract multi-scale feature maps; S2: In response to the multi-scale feature map, generate a footprint mask probability map using the building footprint decoder; S3: Construct a first modulation factor using the footprint mask probability map, perform element-wise gating on the multi-scale feature map, input it to the height decoder, and output the height field; S4: Normalize the height field to obtain a normalized height map and construct a second modulation factor; fuse the second modulation factor with the first modulation factor to form a composite modulation factor, perform dual element-wise gating on the multi-scale feature map and input it into the structure line decoder to output the distance field and orientation angle field; S5: Reconstruct the roof structure line based on the distance field and orientation angle field; S6: Combine the roof structure lines, height field and footprint boundary to construct the roof surface, walls and ground, and generate the LOD2 3D model of the building.

[0007] According to a second aspect of the present invention, a LOD2 building reconstruction system based on single-view orthophotos is provided, comprising: The feature encoding module is used to input single-view orthophoto remote sensing images into a shared backbone network and extract multi-scale feature maps. The building footprint localization module is used to generate a footprint mask probability map based on the multi-scale feature map using the building footprint decoder. The building height estimation module is used to construct a first modulation factor using the footprint mask probability map, and after element-wise gating of the multi-scale feature map, it is input to the height decoder to output a height field. The serial gating and structure line prediction module is used to normalize the height field to obtain a normalized height map and construct a second modulation factor; the second modulation factor is fused with the first modulation factor to form a composite modulation factor, which is then input into the structure line decoder after performing dual element-wise gating on the multi-scale feature map, and outputs the distance field and orientation angle field. The structural line restoration module is used to restore the roof structural line based on the distance field and orientation angle field; the three-dimensional model construction module is used to combine the roof structural line, height field and footprint boundary to construct the roof surface, wall surface and ground, and generate the building LOD2 three-dimensional model.

[0008] According to a third aspect of the present invention, a computer-readable storage medium is provided storing computer software instructions, which, when executed by a processor, implement the steps of the building LoD2 level three-dimensional reconstruction method described in any embodiment.

[0009] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. By using a single orthophoto remote sensing image as input, it eliminates the need for auxiliary data such as DSM, LiDAR point clouds, or multi-view stereo image pairs, significantly reducing data acquisition costs and production cycles, making it suitable for the high-timeliness requirements of large-scale urban building 3D modeling; 2. Through the cascaded modulation of footprint, height, and structure lines, it achieves spatial consistency constraints between multi-task predictions, effectively avoiding topological defects such as height overflowing the footprint range and structure lines deviating from the building outline; 3. By combining the implicit representation scheme of the distance field and orientation angle field with post-processing restoration, it achieves unified modeling and extraction of roof structure lines of arbitrary shapes, without relying on a predefined roof type template library, and has strong generalization ability; 4. It provides a complete pipeline from implicit field prediction to explicit LOD2 model generation, and the output 3D model includes semantically labeled roof surfaces, walls, and ground, meeting the geometric integrity and semantic correctness requirements of the CityGML LOD2 standard.

[0010] Figure 1 This is a schematic diagram of the overall process of a method for reconstructing LOD2 level buildings using single-view orthophotos provided in an embodiment of the present invention;

[0011] Figure 2 This is a schematic diagram of the multi-scale feature extraction structure of the shared backbone network provided in an embodiment of the present invention;

[0012] Figure 3 This is a schematic diagram of the progressive serial gating multi-task decoding architecture provided in an embodiment of the present invention; it illustrates the modulation relationship between the three branches: footprint mask, height field, and structure line.

[0013] Figure 4 This is a schematic diagram of the building footprint output by the footprint mask decoder provided in an embodiment of the present invention;

[0014] Figure 5 This is a schematic diagram of the building height output by the height decoder provided in this embodiment of the invention;

[0015] Figure 6This is a visual schematic diagram of the distance field and orientation angle field output by the structure line decoder provided in an embodiment of the present invention;

[0016] Figure 7 This is a schematic diagram of recovering the roof structure line from the distance field and the orientation angle field provided in an embodiment of the present invention;

[0017] Figure 8 This is a schematic diagram of the reconstruction results provided in an embodiment of the present invention;

[0018] Figure 9 This is a schematic diagram of the architecture of a LOD2 building reconstruction system based on a single-view orthophoto provided in an embodiment of the present invention;

[0019] Figure 10 This is a schematic diagram of data processing for a single-view orthophoto of a building at LoD2 level 3D reconstruction, provided by an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0021] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0022] Existing technologies for LOD2 3D reconstruction of buildings mainly include: methods based on airborne LiDAR point clouds, which offer high accuracy but involve expensive equipment, complex data processing, and long update cycles; methods based on multi-view oblique photography, which require multi-angle image input and involve large post-processing computations; and methods based on digital surface models (DSM), which require additional high-precision DSMs as prior input, increasing data dependence. All of these methods rely on multi-source auxiliary data other than orthophotos, making it difficult to meet the urgent needs for low cost and high timeliness in large-scale urban building 3D modeling.

[0023] Example 1 To address the aforementioned problems in existing technologies, reference is made to... Figure 1This invention provides a method for reconstructing LOD2 level buildings from single-view orthophotos, specifically including the following steps: Step S101: Input the single-view orthophoto remote sensing image into the shared backbone network and extract the multi-scale feature map F. The shared backbone network is a Swing Transformer, which reduces the spatial resolution and doubles the number of channels in four stages to obtain the multi-scale feature map F.

[0024] Specifically, step S10 involves processing the single-view orthophoto remote sensing image. The input is fed into a shared backbone network, divided into non-overlapping 4×4 pixel image patches, and mapped to initial embedding features via a linear projection layer, resulting in a resolution of [resolution missing]. An initial feature map with channel dimension C0, where H and W are the height and width of the image, respectively, and C0 is the initial embedding dimension.

[0025] Step S102: The initial feature map from step S10 is sequentially fed into the four-stage Swin Transformer module for hierarchical feature extraction. Within each stage, a local self-attention mechanism based on a sliding window (window size 4×4) is used to model the context of the features; between adjacent stages, the spatial resolution is reduced to half of the original through Patch Merging operations, while the number of channels is doubled, thereby gradually increasing the receptive field while preserving global semantics.

[0026] Step S103: After four stages of extraction, feature maps at four spatial scales are obtained, which together constitute a multi-scale feature map set. : .

[0027] In the formula, l represents the feature level number, and C1 = C0·2 l-1 The F1 value represents the number of channels in the l-th level feature map; F1 preserves detailed information at higher spatial resolution, such as building edges and textures; F2 balances local details and preliminary semantic features at medium to high resolution; F3 captures a larger range of contextual and structural information at medium to low resolution; and F4 has the semantic information with the largest receptive field.

[0028] Step S104: Copy and distribute the multi-scale feature map set F to the footprint mask prediction branch, height field estimation branch and structure line extraction branch respectively, as the shared feature input of the three decoding branches, to achieve joint optimization of backbone network parameters.

[0029] Step S2: In response to the multi-scale feature map F, generate a footprint mask probability map M using the building footprint decoder.

[0030] Specifically, in step S201, in response to the building footprint decoder, the different number of channels of the multi-scale feature map are uniformly mapped to the common embedding dimension to obtain the projected feature map.

[0031] The multi-scale feature map The feature maps at each level are fed into the corresponding linear projection layers, and the different number of channels C1 at each level are uniformly mapped to the common embedding dimension d. e The projected feature map is obtained. : .

[0032] In the formula, MLP l d represents the linear projection layer corresponding to level l. e This indicates a unified embedding dimension.

[0033] Step S202: Upsample each projected feature map to the same spatial resolution using bilinear interpolation, and then stitch them together to obtain a fused feature map.

[0034] The result obtained in step S201 Upsampled to the same level using bilinear interpolation. Same spatial resolution Then, four levels of features are spliced ​​along the channel dimension to obtain the fused feature map F. cat : .

[0035] In the formula, This indicates a bilinear interpolation upsampling operation.

[0036] Step S203: In response to the fused feature map, perform 1×1 convolution, batch normalization and GELU activation in sequence to obtain the compressed fused feature map.

[0037] splicing feature map F cat Perform 1×1 convolution, batch normalization (BatchNorm), and GELU activation sequentially to reduce the number of channels from 4d. e Compress to d e Redundancy is eliminated and nonlinear representation is enhanced to obtain the fused feature map F. fuse .

[0038] .

[0039] Step S204: In response to the compressed fused feature map, random discarding is applied sequentially, and single-channel prediction logits are output using 1×1 convolution and restored to the original spatial resolution by bilinear interpolation upsampling. Then, a footprint mask probability map is generated by mapping with the Sigmoid function.

[0040] For F fuse After applying random dropout, the single-channel predicted logits are output through 1×1 convolution, and then upsampled through bilinear interpolation to restore the original image resolution H×W. Finally, the logits are mapped to a footprint mask probability map M by the Sigmoid function.

[0041] .

[0042] In the formula, σ() represents the Sigmoid function, and Conv 1×1 This represents a 1×1 convolution operation, where each pixel value in M ​​represents the probability that the location belongs to a building footprint.

[0043] Step S3: Construct a first modulation factor using the footprint mask probability map, perform element-wise gating on the multi-scale feature map F, and input it to the height decoder to output the height field H.

[0044] Step S301: The multi-scale feature map and the corresponding first modulation factor are gated element by element to obtain the gated height feature, which is then input into the height decoder.

[0045] The footprint mask probability map M is scaled to obtain the first modulation factors A1, A2, A3, and A4 corresponding to the feature maps at each scale; where the first modulation factor at the i-th scale is expressed as: A i =T i (M), i=1,2,3,4; In the formula, T i ( ) represents the scaling transformation mapping at the i-th scale, downsampling M to F. i Same spatial resolution .

[0046] The multi-scale feature map F={F1,F2,F3,F4} is gated element-wise with the corresponding first modulation factors A1, A2, A3, A4 to obtain the gated height feature F. h ={F h1 ,F h2 ,F h3 ,F h4}; where the gating result at the i-th scale is represented as: ;

[0047] In the formula, F represents element-wise multiplication. hi This represents the height feature map at the i-th scale after footprint constraint, enabling the decoder to extract effective height information only within the building area.

[0048] Step S302: In response to the height decoder, deep features are fused with shallow features step by step and upsampled. The height space is discretized into several depth intervals using interval regression and attractor mechanism. The probability distribution of each pixel interval is predicted and weighted summation is used to obtain the initial building height map.

[0049] The height feature F after gating h The input is fed into a height decoder for stepwise feature fusion and decoding; the height decoder starts from the deepest feature F h4 Starting from this point, features are fused with shallow features at each level and upsampled. Simultaneously, the height space is discretized into n using interval regression and attractor mechanisms. b Each depth interval is refined layer by layer, with the center of each interval being further refined. And predict the pixel-by-pixel interval probability distribution P k (i,j), the preliminary building height map R is obtained by weighted summation: ;

[0050] In the formula, p k (i,j) represents the probability that pixel (i,j) belongs to the k-th depth interval, and .

[0051] Step S303: In response to the initial building height map, scale recovery is performed and combined with the footprint mask probability map to output the building height field.

[0052] The initial height map R is scaled back and combined with the footprint mask probability map M to output the building height field H: ;

[0053] In the formula, U represents element-wise multiplication, and U() represents bilinear interpolation scale recovery mapping, which upsamples R to the original image resolution H×W. This represents a binary mask for footprints. This represents the mask confidence threshold, ensuring that the building height field H outputs valid height values ​​only within the building footprint area.

[0054] Step S4: Normalize the height field H to obtain a normalized height map. A second modulation factor is constructed; the second modulation factor is fused with the first modulation factor to form a composite modulation factor, and the multi-scale feature map is subjected to dual element-wise gating before being input into the structure line decoder, and the range field D and the orientation angle field Θ are output.

[0055] Step S401: In response to the height field, after normalization processing, a normalized height map is obtained.

[0056] The building height field H is normalized to obtain a normalized height map. : ;

[0057] Among them, H min H max Let H and ε represent the minimum and maximum values ​​of the height field H within the building footprint region, respectively, and let ε represent the minimum constant to prevent division by zero. It reflects the relative elevation of each pixel and serves as the geometric prior for structure line prediction.

[0058] Step S402: In response to the normalized height map, perform a scale transformation to obtain a second modulation factor corresponding to each scale feature map.

[0059] Normalized height map Perform a scale transformation to obtain the second modulation factors B1, B2, B3, and B4 corresponding to the feature maps at each scale; where the second modulation factor at the i-th scale is expressed as: ;

[0060] Among them, T i ( ) represents the scale transformation mapping at the i-th scale, which will Downsampled to the same spatial resolution as Fi.

[0061] Step S403: In response to the first modulation factor and the second modulation factor, perform element-wise multiplication and fuse them to form a composite modulation factor.

[0062] The first modulation factor A i With the second modulation factor B i Element-by-element fusion is performed to construct the composite modulation factor C. i : ;

[0063] in, This represents element-wise multiplication, with the composite modulation factor C. i Simultaneously, the spatial constraints of the footprint mask and the geometric constraints of the height field are encoded, enabling dual guidance for the prediction of the structure line.

[0064] Step S404: In response to the multi-scale feature map and the composite modulation factor, perform dual element-wise gating to obtain the gated structure line features.

[0065] The multi-scale feature map obtained in step S1 The structure line feature F is obtained by performing dual element-wise gating with composite modulation factors C1, C2, C3, and C4. l ={F l1 ,Fl2 ,F l3 ,F l4}: .

[0066] Step S405: In response to the gated structure line features, input the structure line decoder, perform stepwise deconvolution upsampling of deep features and add them element by element to shallow features to restore the original spatial resolution and obtain the fused feature map.

[0067] The gated structure line features Fl are input into the structure line decoder, starting from the deepest feature F. l4 The process involves progressively deconvolutional upsampling and element-wise addition with shallow features to restore the original image resolution, resulting in the fused feature map F. base Subsequently, two parallel prediction heads output the range field D and the orientation angle field Θ, respectively. ;

[0068] ;

[0069] Where (i,j) represents the spatial coordinates of a pixel. This indicates the range field prediction head, which outputs normalized range values ​​activated by ReLU. Let represent the orientation angle prediction head, σ(·) represent the Sigmoid function, and multiply the output by π to obtain the structure line orientation angle in the range [0, π); D(i, j) represents the normalized distance from pixel (i, j) to the nearest structure line. This indicates the local orientation angle of the structural line at that location.

[0070] Step S406: The range field and orientation angle field are output by the two parallel prediction heads respectively.

[0071] Step S5: Recover the roof structure line L based on the distance field D and the orientation angle field Θ.

[0072] Step S501: After the distance field undergoes inverse distance transformation, a pixel-level structure line distance map is obtained.

[0073] Performing an inverse range transform on the range field D yields a pixel-level structure line distance map D', where the pixel-level structure line distance map D' is denoted as D' = λ·D, where... λ This represents the preset line neighborhood scale parameter. A pixel value close to 0 indicates that the pixel is located on the structure line, while a pixel value close to λ indicates that the pixel is far from the structure line.

[0074] Step S502: In response to the footprint mask probability map, the pixel-level structure line distance map and orientation angle field are constrained to the effective area inside the building to obtain the set of candidate pixels for the structure line and the corresponding orientation information.

[0075] Based on the footprint mask probability map M, the pixel-level structure line distance map D' and orientation angle field Θ are constrained to a region within the effective area inside the building, thus obtaining the structure line candidate pixel set C and its corresponding orientation information. The constraint relationship is as follows: ;

[0076] Where, τ m The confidence threshold of the footprint mask is represented; the candidate regions of structural lines inside the building and their corresponding orientation information are obtained, and M(i,j) represents the probability value of the footprint mask probability map M at pixel position (i,j).

[0077] Step S503: In response to the candidate pixel set of the structure line, the distance map in the threshold discrimination set is used to extract pixels that meet the preset conditions as the initial structure line pixels, and morphological closing operation, connected component filtering, morphological opening operation and line thinning are performed in sequence to obtain the internal structure line set.

[0078] Thresholding is performed on the distance map D' within the candidate pixel set C, where D'(i,j)<τ d The pixels are used as the initial structure line pixels, where τ d The distance threshold is used; then, post-processing operations are performed sequentially: ① Morphological closing operation to fill line segment breaks; ② Connected component filtering to remove isolated noise components with an area smaller than Amin; ③ Morphological opening operation to eliminate glitch; ④ Line thinning operation to shrink line segments to a single-pixel wide skeleton, obtaining the set of internal structure lines L. in .

[0079] Step S504: Use the boundary contour of the footprint mask as the boundary line of the building footprint, and merge it with the set of internal structural lines to obtain a complete set of roof structural lines.

[0080] Extract the boundary contour of the footprint mask M and use it as the building footprint boundary line L. bd , will L in With L bd By merging, a complete set of roof structure lines is obtained: ;

[0081] The directional properties of the roof structure line set L are then assigned by the directional angle field Θ.

[0082] Step S6: Combine the roof structure line L, height field H and footprint boundary to construct the roof surface, wall surface and ground, and generate the LOD2 three-dimensional model of the building.

[0083] Step S601: The area enclosed by the footprint boundary is taken as the building area, and the structural line is used as the segmentation constraint. The building area is topologically divided to obtain a set of roof sub-regions.

[0084] Step S601: The area enclosed by the footprint boundary is taken as the building region, and the structural line is used as a partitioning constraint. The building region is topologically partitioned to obtain a set of roof sub-regions. The building region enclosed by the footprint boundary B is denoted as Ω. B And using the structural line L as a partitioning constraint, the building area Ω is... B Perform topological partitioning to obtain the set of rooftop subregions P = {P1, P2, ..., P} k}, where k is the total number of rooftop subregions; where the set of rooftop subregions is represented as P = Γ(Ω B \ L), Γ represents the connected component partitioning mapping, Ω B \L represents the area Ω of the building. B The remaining area after removing the structural line L.

[0085] Step S602: In response to the height field, extract the height information of each roof sub-region, estimate the roof surface parameters of each roof sub-region, and construct the corresponding roof surface equation.

[0086] Extracting each roof sub-region P by combining the height field H i The height information within the roof area, and the height information for each roof sub-area P. i Roof surface parameters are estimated, and the corresponding roof surface R is constructed using the least squares method. i The equation: ;

[0087] Where x and y represent the column and row coordinates of the pixel, respectively, and z i Let a represent the elevation value of the corresponding point on the i-th roof surface. i b i c represents the slope coefficient of the i-th roof surface. i This represents the intercept, thus constructing the corresponding roof surface R. i H(x,y) represents the elevation value of the building height field H at pixel coordinates (x,y).

[0088] Step S603: In response to the boundaries of each roof surface, perform contour extraction, Douglas-Peucker boundary simplification, and adjacent boundary snapping alignment in sequence; for shared vertices located at the intersection of multiple roof surfaces, take the average elevation of each associated roof surface equation as the three-dimensional elevation of the vertex, and obtain a topologically consistent set of roof surfaces.

[0089] R of each roof surface i The boundaries are sequentially subjected to contour extraction, Douglas-Peucker boundary simplification, and adjacent boundary snapping alignment, with a tolerance of ε for snapping alignment. snap For the shared vertex v=(x) located at the intersection of multiple roof surfaces v ,y v The average elevation of each associated roof surface equation is taken as the three-dimensional elevation of that vertex.

[0090] ;

[0091] Where, x v y v Let z be the column and row coordinates of vertex v. v Let be the 3D elevation of vertex v, be the set of roof surfaces containing vertex v, and be the number of elements in this set; thus, we obtain a topologically consistent set of roof surfaces R = {R1, R2, ..., R...} k}

[0092] Step S604: In response to the height information of the area surrounding the footprint boundary, take the average height of the low quantile points as the ground reference height and construct the ground reference surface.

[0093] Based on the height information of the area surrounding footprint boundary B, the average height of the low quantile points is taken as the ground reference height z. g And construct the ground reference surface G.

[0094] Step S605: In response to the outer boundary of the roof surface set that is referenced only by a single roof surface, which is taken as the upper edge of the wall, the two endpoints of the boundary are projected downwards to the ground reference height to obtain the corresponding ground endpoints. The four points are connected end to end to construct a quadrilateral wall, thus obtaining the wall set. The outer boundary of the roof surface set R that is referenced only by a single roof surface is taken as the upper edge of the wall, and the two endpoints (x1, y1, z1) and (x2, y2, z2) of the boundary are projected downwards to the ground reference height z. g The corresponding ground endpoints (x1, y1, z) are obtained. g (x2,y2,z) and (x2,y2,z) g Connect the four points one after the other to construct the j-th quadrilateral wall W. j Thus, we obtain the wall set W = {W1, W2, ..., W...} m}, where m is the total number of walls.

[0095] Step S606: Combine the roof surface set, wall surface set, and ground reference surface to generate a LoD2 3D model of the building.

[0096] The roof surface set R, the wall surface set W, and the ground reference surface G are combined to generate a LoD2 3D model C of the building, wherein the LoD2 3D model of the building is represented as follows: .

[0097] Example 2 This embodiment provides a specific scenario for illustration.

[0098] In this embodiment, the input remote sensing image is an orthorectified high-resolution satellite or aerial image with a spatial resolution of 0.3m and an image size of 512×512 pixels, containing RGB bands. The shared backbone network uses SwinTransformer V2-Base, with an initial embedding dimension C0=128, a window size s=7, and four-stage channel numbers of 128, 256, 512, and 1024 respectively, and a unified embedding dimension d. e =256, number of height intervals n b =256, Footprint mask confidence threshold τ m =0.5, distance threshold τ d =0.15, minimum area threshold A of connected components min =50 pixels, line neighborhood scale parameter λ=5.0, snap-in alignment tolerance pixel ε snap =2, excluding the zero constant ε=10 -6 .

[0099] Step S1, refer to Figure 2 Input orthophoto remote sensing images The image is divided into non-overlapping 4×4 pixel blocks, which are then mapped to initial embedding features of 128×128×128 through a linear projection layer. These features are then sequentially fed into a four-stage Swing Transformer module for hierarchical feature extraction.

[0100] The core of the Swin Transformer is a local multi-head self-attention mechanism based on sliding windows. Within each stage, the feature map is divided into a 7×7 non-overlapping window, and multi-head self-attention is computed within each window. To enable cross-window information exchange, successive Transformer layers alternate between regular window segmentation and shifted window segmentation, allowing features between adjacent windows to interact indirectly through shift operations. Downsampling is achieved between adjacent stages through Patch Merging: 2×2 adjacent pixel features are concatenated and then passed through a linear layer to double the number of channels while halving the spatial resolution.

[0101] After four stages, feature maps at four spatial scales are obtained. .

[0102] F1 preserves details such as building edges and textures, while F4 encodes the overall semantics and contextual relationships of the building. This involves a multi-scale feature map set. The parameters are copied and distributed to the footprint mask prediction branch, the height field estimation branch, and the structure line extraction branch, respectively. The three branches share the same set of backbone network parameters, and parameter optimization is achieved through joint training.

[0103] Step S2, refer to Figure 3 and Figure 4 The building footprint decoder uses a lightweight, full MLP decoder head in the SegFormer style. The feature maps at four scales are passed through linear projection layers to unify the number of channels to d. e =256, then the low-resolution feature map was upsampled to 128×128 using bilinear interpolation, and... The fused feature map is obtained by concatenating along the channel dimension, resulting in a 128×128×1024 map. After compression to 128×128×256 by 1×1 convolution, BatchNorm, and GELU activation, Dropout is applied, followed by another 1×1 convolution to output single-channel logits, which are upsampled to 512×512 and then processed by the Sigmoid function to obtain the footprint mask probability map M.

[0104] During the training phase, a weighted combination of binary cross-entropy loss and Dice loss is used: ;

[0105] M * represents the truth mask for footprint annotations. .

[0106] Step S3, refer to Figure 3 and Figure 5 The footprint mask probability map M is downsampled to four scales using adaptive average pooling and bilinear interpolation to construct the first modulation factors A1, A2, A3, and A4. The backbone features are then gated with the modulation factors through element-wise multiplication to obtain the gated height features.

[0107] The physical meaning of gating is that in the footprint mask probability map M, pixel values ​​within building areas are close to 1, while those in non-building areas are close to 0. Through element-wise multiplication, features in non-building areas are effectively suppressed, allowing the height decoder to focus solely on features within building areas for height regression, thus avoiding the problem of false heights generated in non-building areas. The height decoder employs the BinsFormer architecture, achieving high-precision height estimation through interval regression and attractor mechanisms. Specifically, the height value range is discretized into n... b=256 depth intervals, initially with uniformly distributed interval centers. During decoding, starting from the deepest feature Fh4, the data is upsampled level by level and fused with shallower features. At the same time, the center position of each interval is refined layer by layer through the Transformer attention mechanism. For each pixel, the probability distribution of its belonging to each interval is predicted, and the continuous height value is obtained by weighted summation.

[0108] The training loss for the height field uses SILog loss: ;

[0109] in, Indicates the truth height. Summation is performed only within the effective area of ​​the footprint mask.

[0110] Step S4, refer to Figure 3 This step is the core innovation of this invention. The height field H is normalized to the range [0, 1], and then downsampled to four scales to obtain the second modulation factors B1, B2, B3, and B4. The first modulation factor and the second modulation factor are multiplied element-wise to obtain the composite modulation factor. The physical meaning of the composite modulation factor is: A i Provides spatial constraints (range [0, 1]) on "which areas are buildings", B i Furthermore, a geometric constraint (range [0, 1]) is provided for the "relative height distribution within the building area". The C obtained by multiplying the two elements element by element is C. i It simultaneously encodes both spatial location and elevation geometry information. This is achieved through C... i After gating the backbone features, the features received by the structure line decoder already contain information about the spatial extent and height undulation of the building, thus enabling more accurate prediction of the location and direction of the roof structure line.

[0111] refer to Figure 6 The structure line decoder outputs two parallel fields: the distance field D represents the normalized distance from each pixel to the nearest structure line, and the orientation angle field Θ represents the local orientation angle of the structure line at that location. By employing an implicit representation scheme for the distance and orientation angle fields, rather than directly outputting the coordinates of the line segment endpoints, it is possible to uniformly model roof structure lines of arbitrary shapes using pixel-level dense prediction.

[0112] The training loss for structure line prediction is: ;

[0113] Among them, D * Θ * Let L1 represent the true values ​​of the range field and the orientation angle field, respectively, and L2 represent the mean absolute error (MAE). .

[0114] The overall training loss of the network is: ;

[0115] Step S5, refer to Figure 7 The distance field D is inversely transformed to obtain D' = λ·D. Pixels satisfying this condition are extracted from the internal region of the building footprint as initial structure line pixels. Subsequently, morphological closing operations (kernel size 3×3) are performed sequentially to fill line segment breaks, connected component filtering to remove isolated noise components with an area less than 50 pixels, morphological opening operations (kernel size 3×3) to eliminate glitch, and line thinning operations (Zhang-Suen thinning algorithm) to shrink line segments to a single-pixel wide skeleton, obtaining the internal structure line set L. in .

[0116] Simultaneously, the boundary contour of the footprint mask M is extracted as the building footprint boundary line L. bd (Using OpenCV's findContours function, setting an area threshold of 500 pixels to cull noisy facets), L in With L bd The fusion yields a complete set of roof structure lines. The directional properties of each structural line are assigned by the directional angle field Θ.

[0117] Step S6, refer to Figure 8 This step assembles the predicted footprint boundaries, height field, and structure lines into a complete LOD2 building 3D model.

[0118] The building area Ω enclosed by the footprint boundary B B After removing the structural line L, the set of roof sub-regions is obtained by partitioning the region into connected components. Each sub-region corresponds to a roof panel. For each roof sub-region P i Take the elevation values ​​of all pixels within it in the height field H, and fit the plane equation z using the least squares method. i =a i x+b i y+c i For flat roof areas, The plane is approximately horizontal; for the pitched roof area, a i b i This reflects the roof's slope and orientation. Douglas-Peucker simplification is applied to the boundaries of each roof surface (simplification threshold set to 2 pixels), and shared boundaries between adjacent roof surfaces are snapped together (tolerance). (pixels), ensuring topological consistency of adjacent facet boundaries. For shared vertices located at the intersection of multiple roof faces, the average elevation of each associated roof face equation at that point is taken as its three-dimensional elevation, thus obtaining a topologically consistent set of roof faces R. Then, based on the height information of the area surrounding the footprint boundary, the average height of the 5th percentile is taken as the ground reference height z. g Construct a ground reference plane G. Take the outer boundary of the roof surface set that is referenced only by a single roof surface as the upper edge of the wall surface, and project its two endpoints downwards to the ground reference height z. g Construct quadrilateral walls to obtain a wall set W. Combine the roof set R, the wall set W, and the ground reference plane G to generate a LoD2 3D model of the building. The output format is compatible with common 3D formats such as OBJ and CityGML, and can be directly integrated with smart city 3D platforms.

[0119] Reconstruction effect as Figure 8 As shown, the generated LOD2 model can accurately depict the roof shape of the building (including various types such as gable roof, hip roof, and flat roof), the topological relationship between the wall and the ground is complete, the model outline is regular, and it is visually highly consistent with the actual building structure.

[0120] Example 3 like Figure 9 As shown, this embodiment provides a LOD2 building reconstruction system based on a single-view orthophoto image, which may include a feature encoding module, a building footprint localization module, a building height estimation module, a serial gating and structural line prediction module, a structural line restoration module, and a 3D model construction module. These modules can work together, using a single orthophoto image as input, and can output a LOD2 3D building model that meets standard specifications without relying on auxiliary data such as digital surface models or LiDAR point clouds. In some embodiments, the various modules of the device can be deployed in at least one computing device, such as a cloud server, edge computing node, or local workstation, and this disclosure does not impose specific limitations in this regard.

[0121] The feature encoding module is used to input single-view orthophoto remote sensing images into a shared backbone network and extract multi-scale feature maps.

[0122] The feature encoding module is described in detail below. In some embodiments, the feature encoding module divides the input single-view orthophoto remote sensing image into non-overlapping pixel image blocks. These pixel image blocks are mapped to initial embedded features through a linear projection layer to obtain an initial feature map. The initial feature map is then sequentially fed into a multi-stage Swing Transformer module for hierarchical feature extraction. Within each stage, a sliding window-based local self-attention mechanism is used to perform contextual modeling of the features. This sliding window-based mechanism effectively limits the spatial range of self-attention computation, thereby significantly reducing computational complexity, while enabling cross-window information interaction through shift window operations. After extraction through multiple stages, feature maps at four spatial scales are obtained, which together constitute the multi-scale feature map. These four spatial scale feature maps correspond to different downsampling rates, such as 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image resolution, respectively. The shallow feature map retains detailed information at higher spatial resolution, such as the edges and textures of buildings; the deep feature map has semantic information with the largest receptive field, which helps to identify the overall structure of buildings.

[0123] The building footprint localization module is used to generate a footprint mask probability map based on the multi-scale feature map using the building footprint decoder.

[0124] The building footprint localization module is described in detail below. This module feeds the feature maps at each level of the multi-scale feature map into their respective linear projection layers, mapping them uniformly to a common embedding dimension to obtain the projected feature map. Since the multi-scale feature maps have different channel numbers at different stages, the corresponding linear projection layers can unify their channel numbers to facilitate subsequent feature fusion. Next, the projected feature maps are upsampled to the same spatial resolution using bilinear interpolation. The features from these four scales are then concatenated along the channel dimension to obtain the fused feature map.

[0125] The fused feature map is sequentially subjected to convolution, batch normalization, and activation operations to compress the number of channels. Then, random dropout is applied before convolution to output a single-channel prediction result. Random dropout effectively prevents overfitting and improves the model's generalization ability on unseen data. Finally, bilinear interpolation upsampling restores the original image resolution, which is then mapped to the footprint mask probability map using the Sigmoid function.

[0126] The building height estimation module is used to construct a first modulation factor using the footprint mask probability map, and then input the multi-scale feature map F into the height decoder after element-wise gating, and output the height field.

[0127] The building height estimation module is described in detail below. The module performs a scale transformation on the footprint mask probability map to obtain the first modulation factor corresponding to the feature maps at each scale. Subsequently, the multi-scale feature maps are multiplied and gated element-wise with the corresponding first modulation factors to obtain the gated height features. These gated height features are then input into the height decoder for step-by-step feature fusion and decoding. Finally, a preliminary building height map is obtained through weighted summation. This discretized interval regression method, compared to directly regressing continuous height values, better handles the long-tail effect of height distribution and improves the robustness of height estimation.

[0128] In some embodiments, the building height estimation module performs scale recovery on the preliminary building height map and outputs the height field by combining it with the footprint mask probability map. Scale recovery can be achieved by bilinear interpolation upsampling to the original image resolution. The operation of combining the footprint mask probability map can be achieved by setting a mask confidence threshold and adjusting the footprint mask probability. Figure 2 The height field is valued and multiplied with the preliminary building height map so that it outputs effective height values ​​only within the building footprint area, further ensuring strict alignment between the height field and the footprint boundary.

[0129] The serial gating and structure line prediction module is used to normalize the height field to obtain a normalized height map and construct a second modulation factor; the second modulation factor is fused with the first modulation factor to form a composite modulation factor, which is then input into the structure line decoder after performing dual element-wise gating on the multi-scale feature map, and outputs the distance field and orientation angle field.

[0130] The serial gating and structure line prediction module is described in detail. This module normalizes the height field to obtain a normalized height map. Normalization can be achieved by mapping the minimum and maximum values ​​of the height field within the building footprint region to a range of 0 to 1, thus reflecting the relative elevation fluctuations of each pixel. Subsequently, the normalized height map is scaled to obtain the second modulation factor corresponding to the feature maps at each scale. Next, the first and second modulation factors are fused element-wise to construct a composite modulation factor. This composite modulation factor simultaneously encodes the spatial constraints of the footprint mask and the geometric constraints of the height field.

[0131] The multi-scale feature map is subjected to dual element-wise gating with the composite modulation factor to obtain the gated structure line features. Through this dual gating mechanism, the features received by the structure line decoder implicitly contain information about the spatial extent and height undulations of the building, thus enabling more accurate perception of areas where structural lines such as ridge lines and eaves lines experience abrupt elevation changes or slope variations. Subsequently, the gated structure line features are input into the structure line decoder, which performs step-by-step deconvolution upsampling and element-wise addition with the shallow features to restore the original image resolution. Finally, two parallel prediction heads output the distance field and the orientation angle field, respectively.

[0132] The distance field represents the normalized distance from a pixel to the nearest structure line, and the orientation field represents the local orientation angle of that structure line. By employing implicit representations of the distance and orientation fields, rather than directly outputting the coordinates of line segment endpoints, we can uniformly model roof structure lines of arbitrary shapes using pixel-level dense prediction, avoiding the insufficient generalization capabilities of predefined roof type template libraries. Smaller pixel values ​​in the distance field indicate that the pixel is closer to the structure line; the orientation field provides crucial angular constraints for subsequent structure line vectorization.

[0133] The structure line restoration module is used to restore the roof structure line based on the distance field and orientation angle field.

[0134] The structure line reconstruction module is described in detail below. The module performs an inverse distance transformation on the distance field to obtain a pixel-level structure line distance map. This inverse distance transformation can map normalized distance values ​​back to the actual pixel distance scale. Subsequently, based on the footprint mask probability map, the pixel-level structure line distance map and the orientation angle field are subjected to regional constraints, limiting both to the effective area inside the building, thus obtaining a set of candidate pixels for structure lines and their corresponding orientation information.

[0135] A threshold judgment is performed on the distance map within the candidate pixel set of the structural lines, and pixels that meet the distance threshold condition are extracted as initial structural line pixels. Then, morphological closing, connected component filtering, morphological opening, and line thinning operations are performed sequentially to obtain the internal structural line set. The boundary contour of the footprint mask probability map is also extracted as the footprint boundary. Subsequently, the internal structural line set is fused with the footprint boundary to obtain the complete roof structural line, and the directional attributes of the roof structural line are assigned by the orientation angle field. The complete roof structural line includes not only the ridge line and gutter line inside the building, but also the eaves line on the building's exterior, laying a solid geometric foundation for subsequent 3D model topology construction.

[0136] The 3D model building module is used to combine the roof structure lines, height field and footprint boundary to build the roof surface, walls and ground, and generate the LOD2 3D model of the building.

[0137] The 3D model construction module is described in detail below. The module performs topological partitioning of the building region enclosed by the footprint boundary, using the roof structure line as a dividing constraint, to obtain a set of roof sub-regions. This partitioning process decomposes the complex building roof into multiple non-overlapping connected regions, each corresponding to a potential roof patch. Subsequently, the height information within each roof sub-region is extracted using the height field, and the corresponding roof surface equation is constructed using the least squares method. For each roof sub-region, an optimal 3D plane equation is fitted using the 3D coordinates of all pixels within it (2D planar coordinates plus the elevation provided by the height field). This equation accurately reflects the slope, orientation, and intercept of the roof surface.

[0138] The boundaries of each constructed roof surface are sequentially subjected to contour extraction, boundary simplification, and adjacent boundary snapping alignment. Further, for shared vertices located at the intersection of multiple roof surfaces, the average elevation of each associated roof surface equation is taken as the 3D elevation of that vertex, resulting in a topologically consistent set of roof surfaces. A ground reference plane is constructed based on the height information of the area surrounding the footprint boundary. Subsequently, the outer boundary of the roof surface set referenced only by a single roof surface is used as the upper edge of the wall, projected downwards onto the ground reference plane to construct a wall set. Each wall can be a quadrilateral patch formed by the two endpoints of its upper edge and its projection point on the ground reference plane. Finally, the roof surface set, the wall set, and the ground reference plane are combined to generate a LoD2 3D model of the building. This model can be exported to common 3D formats such as CityGML and OBJ, and directly applied to smart city, digital twin, and other scenarios.

[0139] The computer-readable storage medium or electronic device disclosed herein may store instructions that, when executed by a processor, enable the processor to perform the LOD2-level building reconstruction method based on single-view orthophotos. (See also...) Figure 10 When the instruction is executed, it can realize the complete technical chain from implicit pixel-level prediction to explicit LOD2 3D model generation.

[0140] All embodiments described above are not limited to the illustrated embodiments and can, of course, be combined with each other. Although the present invention has been described with reference to the embodiments shown in the accompanying drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments can be made therefrom.

Claims

1. A method for reconstructing LOD2 level buildings from single-view orthophotos, characterized in that, The method includes: S1: Input single-view orthophoto remote sensing images into a shared backbone network and extract multi-scale feature maps; S2: In response to the multi-scale feature map, generate a footprint mask probability map using the building footprint decoder; S3: Perform scale transformation on the footprint mask probability map to obtain the first modulation factor corresponding to each scale feature map, and input the multi-scale feature map into the height decoder after element-by-element gating to output the height field; S4: Normalize the height field to obtain a normalized height map, perform scale transformation on the normalized height map to obtain a second modulation factor corresponding to each scale feature map; multiply the second modulation factor and the first modulation factor element by element to form a composite modulation factor, perform dual element-wise gating on the multi-scale feature map and input it into the structure line decoder to output the distance field and orientation angle field. S5: Reconstruct the roof structure line based on the distance field and orientation angle field; S6: Combine the roof structure lines, height field and footprint boundary to construct the roof surface, walls and ground, and generate the LOD2 3D model of the building.

2. The method for reconstructing a LOD2 level building according to claim 1, characterized in that, The shared backbone network in step S1 is the Swing Transformer, which reduces the spatial resolution and doubles the number of channels in four stages to obtain the multi-scale feature map.

3. The method for reconstructing a LOD2 level building according to claim 1, characterized in that, Step S2 also includes: In response to the building footprint decoder, the different number of channels of the multi-scale feature map are uniformly mapped to the common embedding dimension to obtain the projected feature map; Bilinear interpolation is used to upsample the projected feature maps to the same spatial resolution, and then the fused feature maps are stitched together. In response to the fused feature map, 1×1 convolution, batch normalization and GELU activation are performed sequentially to obtain the compressed fused feature map; In response to the compressed fused feature map, random dropout is applied sequentially, and single-channel prediction logits are output using 1×1 convolution and then restored to the original spatial resolution by bilinear interpolation upsampling. Finally, a footprint mask probability map is generated by mapping with the Sigmoid function.

4. The method for reconstructing a LOD2 level building according to claim 1, characterized in that, Step S3 also includes: The multi-scale feature map is gated element-wise with the corresponding first modulation factor to obtain the gated height feature, which is then input into the height decoder. In response to the height decoder, deep features are fused with shallow features step by step and upsampled. The height space is discretized into several depth intervals using interval regression and attractor mechanism. The probability distribution of each pixel interval is predicted and weighted summation is used to obtain the initial building height map. In response to the initial building height map, scale recovery is performed and combined with the footprint mask probability map to output the building height field.

5. The method for reconstructing a LOD2 level building according to claim 1, characterized in that, Step S4 also includes: In response to the multi-scale feature map and the composite modulation factor, dual element-wise gating is performed to obtain the gated structure line features; In response to the gated structure line features, the input is given to the structure line decoder. The deep features are upsampled by deconvolution at each level and added element by element to the shallow features to restore the original spatial resolution and obtain the fused feature map. The range field and orientation angle field are output by two parallel prediction heads, respectively.

6. The method for reconstructing a LOD2 level building according to claim 1, characterized in that, Step S5 also includes: After the distance field undergoes inverse distance transformation, a pixel-level structure line distance map is obtained; In response to the footprint mask probability map, the pixel-level structure line distance map and orientation angle field are constrained to the effective area inside the building to obtain the set of candidate pixels for the structure line and the corresponding orientation information. In response to the candidate pixel set of the structural line, the distance map in the threshold discrimination set is used to extract pixels that meet the preset conditions as the initial structural line pixels. Then, morphological closing operation, connected component filtering, morphological opening operation and line thinning are performed in sequence to obtain the internal structural line set. The boundary contour of the footprint mask is used as the boundary line of the building footprint, and then merged with the set of internal structural lines to obtain a complete set of roof structural lines.

7. The method for reconstructing a LOD2 level building according to claim 1, characterized in that, Step S6 also includes: The area enclosed by the footprint boundary is taken as the building area, and the structural line is used as a dividing constraint. The building area is topologically divided to obtain a set of roof sub-regions. In response to the height field, height information of each roof sub-region is extracted, roof surface parameters of each roof sub-region are estimated, and corresponding roof surface equations are constructed. In response to the boundaries of each roof surface, contour extraction, Douglas-Peucker boundary simplification, and adjacent boundary snapping alignment are performed sequentially. For shared vertices located at the intersection of multiple roof surfaces, the average elevation of the equations of each associated roof surface is taken as the three-dimensional elevation of the vertex, resulting in a topologically consistent set of roof surfaces. In response to the height information of the area surrounding the footprint boundary, the average height of the low quantile points is taken as the ground reference height to construct the ground reference surface; In response to the outer boundary of the set of roof surfaces being referenced only by a single roof surface as the upper edge of the wall surface, the two endpoints of the boundary are projected downwards to the ground reference height to obtain the corresponding ground endpoints. The four points are connected end to end in sequence to construct a quadrilateral wall surface, thus obtaining the wall surface set. Combine the roof surface set, wall surface set, and ground reference surface to generate a LoD2 3D model of the building.

8. The method for reconstructing a LOD2 level building according to claim 1, characterized in that, The LoD2 3D model expression of the building is: C = {R, W, G}; In the formula, C represents the LoD2 3D model of the building, R represents the roof surface set, W represents the wall surface set, and G represents the ground reference surface.

9. A LOD2 building reconstruction system based on single-view orthophotos, characterized in that, include: The feature encoding module is used to input single-view orthophoto remote sensing images into a shared backbone network and extract multi-scale feature maps. The building footprint localization module is used to generate a footprint mask probability map based on the multi-scale feature map using the building footprint decoder. The building height estimation module is used to scale the footprint mask probability map to obtain the first modulation factor corresponding to each scale feature map, and to input the multi-scale feature map into the height decoder after element-by-element gating, and output the height field. The serial gating and structure line prediction module is used to normalize the height field to obtain a normalized height map, and to perform a scale transformation on the normalized height map to obtain a second modulation factor corresponding to the feature map at each scale. The second modulation factor is multiplied element-wise with the first modulation factor to form a composite modulation factor. After dual element-wise gating of the multi-scale feature map, it is input into the structure line decoder to output the range field and orientation angle field. The structure line restoration module is used to restore the roof structure line based on the distance field and orientation angle field. The 3D model building module is used to combine the roof structure lines, height field and footprint boundary to build the roof surface, walls and ground, and generate the LOD2 3D model of the building.

10. A computer-readable storage medium storing computer software instructions, characterized in that, When the computer software instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-8.

Images