A method and system for 3D building reconstruction based on domain-adaptive monocular remote sensing images

Abstract

This invention relates to the field of machine learning, specifically to a domain-adaptive method and system for 3D building reconstruction from monocular remote sensing images. The method includes: constructing and initializing a dual-model architecture consisting of a student model and a teacher model; training the student model using a source domain dataset, calculating semantic segmentation loss, height prediction loss, and offset prediction loss, and updating the teacher model; performing structured cross-domain mixing of source and target domain data to generate mixed data; calling the teacher model to generate high-confidence pixel category pseudo-labels for the mixed data and target domain data; constructing positive and negative sample pairs, introducing a contrastive learning mechanism to train the student model, and obtaining the target model; inputting the monocular remote sensing image to be processed into the target model, outputting building roof masks, roof footprint masks, building height, and offset vectors between the roof and footprints, constructing three-dimensional coordinates, and generating a 3D building model. This invention solves the domain offset problem caused by differences in imaging conditions across regions.

Description

Technical Field

[0001] This invention relates to the field of machine learning, specifically to a method and system for domain-adaptive monocular remote sensing image 3D building reconstruction. Background Technology

[0002] Although deep learning models have shown significant potential in height estimation and 3D building reconstruction tasks, and monocular imagery has become an important data support for building outline extraction and height estimation due to its high resolution advantage, it still faces two core bottlenecks in practical large-scale cross-regional applications. The lack of a domain adaptation mechanism is the key reason for the difficulty in implementing the technology:

[0003] First, existing methods are heavily reliant on data annotation. Fully supervised learning frameworks require massive amounts of samples with complete 3D annotations, such as roof outlines, building footprints, heights, and roof-to-ground offsets. However, architectural annotation, especially for irregular buildings or complex scenes, is time-consuming, labor-intensive, and extremely costly, resulting in a scarcity and limited coverage of high-quality 3D annotation data. Even weakly supervised strategies (such as WS-MTBR-Net) attempt to reduce the need for complete 3D labels, still require a certain scale of manual annotation assistance, such as annotating basic outlines, which is insufficient to meet the needs of large-scale reconstruction scenarios across cities and regions.

[0004] Secondly, the distribution shift problem caused by domain migration is particularly prominent. Imagery from different regions may exhibit significant differences in data distribution between the source domain (i.e., labeled training data) and the target domain (i.e., unlabeled application data) due to variations in sensor type, imaging angle, geographical environment, and architectural style. When a model trained in the source domain is directly transferred to the target domain, the mismatch between visual features and building structure distribution leads to problems such as decreased contour extraction accuracy, increased height estimation error, and distorted 3D reconstruction results. This becomes a core obstacle to cross-regional 3D building reconstruction.

[0005] Therefore, it is necessary to propose new technologies for 3D building reconstruction from monocular remote sensing images to solve the bottleneck problem of cross-regional applications. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a domain-adaptive monocular remote sensing image 3D building reconstruction method and system. It adopts a cross-domain adaptation technology based on pixel and block-level contrast learning. By constructing a targeted domain adaptation framework, it aligns the feature distributions of the source domain and the target domain, solves the domain offset problem caused by differences in imaging conditions across regions, and achieves high-precision 3D reconstruction of buildings in cross-domain scenes.

[0007] On one hand, embodiments of the present invention provide a method for reconstructing 3D buildings from domain-adaptive monocular remote sensing images, comprising the following steps:

[0008] S1. Construct a dual-model architecture consisting of a student model and a teacher model, and initialize the models;

[0009] S2. Train the student model using the labeled source domain dataset, and calculate the semantic segmentation loss, height prediction loss, and offset prediction loss; at the same time, update the teacher model based on the training results of the student model.

[0010] S3. Perform structured cross-domain mixing of source domain data and target domain data to generate mixed data that combines the annotation characteristics of the source domain with the scene characteristics of the target domain; call the teacher model to generate high-confidence pixel category pseudo-labels for the mixed data and target domain data, and filter pseudo-labels with confidence higher than the preset threshold.

[0011] S4. Construct positive and negative sample pairs, introduce a contrastive learning mechanism to further train the student model, learn domain-invariant features, optimize the total loss, and obtain the final student model as the target model.

[0012] S5. Input the monocular remote sensing image to be processed into the target model. The target model outputs the building roof mask, roof footprint mask, building height, and offset vector between the roof and the footprint. Using the roof footprint mask as the bottom contour and the building height as the vertical dimension, construct three-dimensional coordinates to generate a 3D building model.

[0013] On the other hand, embodiments of the present invention provide a domain-adaptive monocular remote sensing image 3D building reconstruction system, implemented using the aforementioned reconstruction method, the reconstruction system comprising:

[0014] The initialization module is used to build a dual-model architecture consisting of a student model and a teacher model, and to initialize the models.

[0015] The source domain pre-training module trains the student model using a labeled source domain dataset, calculates semantic segmentation loss, height prediction loss, and offset prediction loss; at the same time, it updates the teacher model based on the training results of the student model.

[0016] The cross-domain fusion and pseudo-label generation module performs structured cross-domain mixing of source domain data and target domain data to generate hybrid data that combines the annotation characteristics of the source domain with the scene characteristics of the target domain. It calls the teacher model to generate high-confidence pixel category pseudo-labels for the hybrid data and target domain data, and filters pseudo-labels with confidence scores higher than a preset threshold.

[0017] The loss optimization module constructs positive and negative sample pairs, introduces a contrastive learning mechanism to further train the student model, learns domain-invariant features, optimizes the total loss, and obtains the final student model as the target model.

[0018] The reconstruction module inputs the monocular remote sensing image to be processed into the target model. The target model outputs the building roof mask, roof footprint mask, building height, and offset vector between the roof and the footprint. Using the roof footprint mask as the bottom contour and the building height as the vertical dimension, a three-dimensional coordinate system is constructed to generate a 3D building model.

[0019] Compared with the prior art, the present invention has the following advantages and effects:

[0020] 1. By adopting a pixel-level contrast learning mechanism, the extraction of footprint contours of tall buildings enhances the intra-class compactness of pixels of the same type of buildings in the feature space and the separation between different types, thereby improving the accuracy of segmentation boundaries.

[0021] 2. By adopting a block-level contrastive learning mechanism, the model can mine domain-invariant features without relying on target domain annotations. This improves the robustness of the model to different imaging conditions (such as different viewpoints, lighting, and sensor characteristics), ensuring that the same building area maintains consistent feature representation under different environments, and effectively reducing the domain offset problem.

[0022] 3. A targeted domain adaptation framework is constructed using a teacher-student network architecture. Through the synergistic effect of pixel-level and block-level dual-dimensional contrastive learning, the feature distributions of the source and target domains are aligned. The contrastive learning is integrated with the multi-task characteristics of 3D building reconstruction (specifically including footprint segmentation, roof segmentation, height prediction, and offset prediction), which effectively alleviates the pain points of single-domain training and cross-domain failure in traditional methods. At the same time, it avoids the extremely high annotation cost of fully supervised methods and achieves high-precision 3D reconstruction in cross-domain scenarios. Attached Figure Description

[0023] Figure 1 This is a flowchart of the domain-adaptive monocular remote sensing image 3D building reconstruction method in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of pixel-level contrast learning in an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of block-level comparative learning in an embodiment of the present invention. Detailed Implementation

[0026] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the implementation of the present invention is not limited thereto.

[0027] Example

[0028] This embodiment proposes a domain-adaptive method for 3D building reconstruction from monocular remote sensing images. It uses a cross-domain adaptation technique based on pixel-level and block-level contrastive learning. By aligning the data distribution of the source and target domains, it reduces the dependence of domain adaptation on data annotation, solves the problem of cross-regional application failure in existing monocular remote sensing image 3D building reconstruction, and improves the robustness and generalization of the model.

[0029] The terms involved are explained below:

[0030] (1) Architectural Footprint

[0031] The architectural footprint refers to the two-dimensional projection area of ​​a building on the ground, that is, the outline of the part of the building's bottom that contacts the ground. It is equivalent to the shadow boundary formed by the building on the ground and is the most basic geometric reference information in 3D building reconstruction tasks.

[0032] (2) Roof-to-footprint Offset

[0033] Roof-footprint offset is a vector information generated by parallax effects in oblique remote sensing images. It describes the relative positional relationship between the roof outline and the footprint outline on the image plane, including both the distance and direction information between them. The essence of this offset is that the roof and the footprint are at different heights in real space, and the oblique shooting angle makes them appear to be at different positions in the image. The offset is a quantitative expression of this difference.

[0034] (3) Monocular remote sensing images

[0035] Monocular remote sensing images refer to 3D reconstruction data sources that use only a single satellite or aerial remote sensing image as input data, without the need for auxiliary data such as multi-view images or LiDAR. Monocular remote sensing image datasets contain complete 3D geometric information of buildings: building footprint, building roof, roof-footprint offset, and real-world height.

[0036] (4) Unsupervised Domain Adaptation (UDA)

[0037] Domain adaptation is a core preparatory step designed to achieve cross-domain knowledge transfer in unsupervised domain adaptation (UDA) scenarios. Its core goal is to enable the model to break free from its dependence on a single data source in the source domain and smoothly adapt to the new unlabeled scenario in the target domain. Essentially, it narrows the distribution gap between the source and target domains by finely mining the inherent patterns in the data, ultimately achieving a low-cost improvement in generalization ability.

[0038] Domain-invariant features refer to features that have a consistent representation of the same type of object (such as building roof or footprint) in both the source and target domains.

[0039] The 3D building reconstruction method proposed in this embodiment uses cross-domain fusion and contrastive learning as intermediate steps related to domain adaptation. The entire process includes: initialization to lay the foundation for domain adaptation, source domain pre-training to construct the starting point for domain adaptation, the core stage of domain adaptation, and 3D reconstruction to implement the domain adaptation effect. The core stage of domain adaptation, including cross-domain fusion, pseudo-labeling, and contrastive learning, is crucial in determining cross-domain performance. Figure 1 As shown, the 3D building reconstruction method of this embodiment includes the following steps:

[0040] S1. Construct a dual-model architecture consisting of a student model and a teacher model, and initialize the models.

[0041] A dual-model architecture consisting of a student model and a teacher model is constructed. Both models include a feature encoder, a multi-task prediction head, and a feature projection head. During initialization, the student and teacher models use the same network structure. The weights of the teacher model are initialized to the initial weight values ​​of the student model. During training, the weights of the teacher model are not directly updated via gradient backpropagation. Instead, they are dynamically adjusted based on the weights of the student model using an exponential moving average (EMA) method, thereby maintaining the stability and continuity of its parameters.

[0042] Through the above training process, the teacher model can provide stable, noise-free and highly consistent supervision signals for unlabeled data in the target domain, avoid training failure caused by pseudo-label fluctuations during domain adaptation, effectively reduce training instability caused by pseudo-label fluctuations or noise, and improve the model's adaptability in cross-domain scenarios and the accuracy of reconstruction tasks.

[0043] In addition, during the model initialization process, for the feature encoder, a feature extraction network commonly used in deep learning is selected. Using monocular remote sensing images as input, multi-scale visual features are extracted and used as input for the multi-task prediction head for sharing by the multi-task prediction head.

[0044] The multi-task prediction head further includes a roof mask segmentation head, a building footprint mask segmentation head, a height prediction head, and an offset prediction head, collectively adapting to the multi-objective requirements of building reconstruction tasks, i.e., adapting to the multi-task characteristics of building reconstruction. Specifically, the roof mask segmentation head is used to parse and output roof pixel classification results from input features; the building footprint mask segmentation head is used to parse and output footprint pixel classification results from input features; the height prediction head is used to output pixel-level building height values; and the offset prediction head is used to calculate the two-dimensional planar offset of roof pixels relative to their corresponding ground projection pixels and output a roof-footprint pixel-level offset vector field.

[0045] The feature projection head is used to map the extracted features to the low-dimensional feature space required for contrastive learning, achieving pixel-level or block-level feature space transformation to support subsequent domain adaptation processes based on contrastive learning. In this embodiment, the feature projection head is divided into a pixel-level projection head and a block-level projection head. The input of the pixel-level projection head is the features output by the feature encoder of the image, while the input of the block-level projection head is two features output by the feature encoder from two upsampled blocks of an image.

[0046] In this embodiment, the teacher model is defined by the formula: Perform weight updates, where For the first Teacher model weights, For the first Shift student model weights, For the first Teacher model weights, The attenuation coefficient provides a stable and up-to-date monitoring signal for the unlabeled target domain.

[0047] S2. Source domain pre-training: The student model is trained using only the labeled source domain dataset, and the three core losses—semantic segmentation loss, height prediction loss, and offset prediction loss—are calculated. At the same time, the teacher model is updated using the exponential moving average (EMA) mechanism based on the training results of the student model.

[0048] The purpose of source domain pre-training is to enable student models to master the basic features of buildings in the source domain dataset, such as the roof-footprint relationship and the correspondence between height and contour, and to generate certain correct pseudo-labels for the target domain dataset, so as to provide a reliable starting point for feature transfer in subsequent domain adaptation. If the source domain pre-training is insufficient, cross-domain transfer will fail during the domain adaptation process due to the weak feature foundation of the source domain.

[0049] The tasks of predicting rooftops and building footprints are both pixel-level semantic segmentation problems, with semantic segmentation losses of [missing information]. In this embodiment, it is uniformly referred to as .set up Represents the total number of pixels in the image. The total number of categories, For a one-hot vector, if the pixel The true category is Then the vector exist The value on the dimension is 1, and the rest are 0; Pixels predicted by the model Category The probability of the loss function is given by:

[0050] .

[0051] For highly predictive tasks, record To highly predict losses, , Let the model's predicted building height and the actual building height be the building height, respectively. Then, the formula for calculating the loss function is:

[0052] .

[0053] For the offset prediction task, let's call it... For offset prediction loss, This represents the offset predicted by the model for a single pixel. If the labels are true, then the formula for calculating the loss function (endpoint error) is:

[0054] .

[0055] Let the total loss of pre-training be... , Let the weights of the four loss functions be respectively, then the formula for calculating the total loss is:

[0056] ;

[0057] Then, by optimizing the total loss, we obtain student and teacher models that can be basically distinguished.

[0058] S3, Cross-domain fusion and pseudo-label generation, performs structured cross-domain mixing of source domain data and target domain data to generate hybrid data that combines the annotation characteristics of the source domain with the scene features of the target domain; calls a well-trained teacher model to generate high-confidence pixel category pseudo-labels for the hybrid data and target domain data, and filters pseudo-labels with confidence levels higher than a preset threshold to ensure the reliability of the supervision signal during domain adaptation.

[0059] This step introduces the source domain dataset and the target domain dataset, and through structured cross-domain fusion and high-confidence pseudo-label selection, provides data support with reliable supervision signals for subsequent model training. Specifically, this includes:

[0060] S31. Perform cross-domain data fusion operation: Using a category-level copy-paste method, randomly select 50% of the semantic regions from the source domain data according to their categories; for the target domain data, perform preliminary semantic prediction using a pre-trained teacher model to identify and locate the non-building regions; then, paste the building regions cropped from the source domain data into the corresponding non-building semantic regions in the target domain data according to their semantic categories, thereby constructing cross-domain hybrid data that simultaneously contains the annotation characteristics of the source domain and the scene features of the target domain.

[0061] By pasting the cropped building areas from the source domain data into the corresponding non-building semantic areas, such as ground areas, in the target domain data, potential buildings in the target domain can be avoided. This operation achieves initial alignment of the source and target domain distributions at the data level, laying the foundation for subsequent domain adaptation at the feature level.

[0062] S32. Perform pseudo-label generation and filtering: Call the teacher model that has been updated and trained stably using the exponential moving average (EMA) mechanism to perform forward inference on the cross-domain mixed data and the target domain data respectively, generating pixel-level category pseudo-labels to distinguish categories such as buildings and backgrounds, roofs and footprints; on this basis, a confidence threshold filtering mechanism is introduced to retain only pixel pseudo-labels with model prediction confidence higher than a set threshold, filtering out low-confidence parts to ensure the reliability of pseudo-labels as supervision signals; at the same time, the source domain real labels and target domain pseudo-labels are fused to generate hybrid labels for subsequent model training.

[0063] S4. Construct positive and negative sample pairs, introduce a contrastive learning mechanism to further train the student model, learn domain-invariant features, optimize the total loss, and obtain the final student model as the target model.

[0064] This step, based on the calculation of pseudo-label supervised loss, further introduces a pixel-level and block-level dual-dimensional contrastive learning mechanism to learn domain-invariant features and solve the distribution offset problem between the source and target domains. Specifically, the calculation of the pseudo-label supervised loss follows the loss function system from the source domain pre-training stage in step S2, using pseudo-labels as target domain data to replace the real labels.

[0065] Specifically, in the student-teacher dual-model architecture, the backbone encoder of the student model extracts multi-scale features from the input image. These multi-scale features are then processed by two independent projection heads: a pixel-level projection head and a block-level projection head. Both are used to map high-dimensional features to a low-dimensional contrastive learning feature space.

[0066] The specific process of pixel-level contrastive learning is as follows: Based on the pseudo-labels generated by the teacher model, pixels in the target domain image are classified into categories; pixel features belonging to the same category are formed into positive sample pairs, and pixel features belonging to different categories are formed into negative sample pairs; the scope of positive sample pairs can be extended to source and target domain images within the same training batch to cover feature diversity in cross-domain scenarios. Subsequently, the features extracted by the backbone encoder are mapped to a low-dimensional space through a pixel-level projection head, and the contrastive loss is calculated to shorten the feature distance of positive sample pairs and widen the feature distance of negative sample pairs, thereby enhancing the model's ability to discriminate features of categories such as buildings, backgrounds, roofs, and footprints, and improving the intra-class compactness and inter-class separability of its feature representation.

[0067] The specific process of block-level contrastive learning is as follows: partially overlapping feature blocks are sampled from the feature map. Feature vectors with the same spatial location within the same feature block are regarded as positive sample pairs, and feature vectors with different spatial locations are regarded as negative sample pairs. After the features are mapped by the block-level projection head, the contrast loss is calculated to enable the model to maintain a stable feature representation of the same building area under different imaging conditions, viewing angles or sensor differences, thereby improving the robustness of the model to domain shift.

[0068] Through the joint optimization of pixel-level and block-level comparative learning, the model can mine and strengthen domain-invariant features at different granularities, effectively alleviate feature distortion or misclassification caused by differences in data distribution, and achieve stable semantic segmentation in cross-domain scenarios.

[0069] In this embodiment, record This is a feature map extracted using a pixel-level projection head. Feature map The first One characteristic, The total number of pixels in the image is represented by the exponential cosine similarity formula. Calculate the cosine similarity. The contrastive learning loss is calculated using the following formula: .

[0070] By optimizing the contrastive learning loss, the distance between pixels of the same category is reduced in the low-dimensional feature space, while the distance between pixels of different categories is increased. Figure 2 The use of circles and squares in the design improves the compactness of intra-class features and the separation of inter-class features. Figure 2 In the image, the high-dimensional features extracted by the encoder on the left are mapped to the low-dimensional contrast feature space on the right by a pixel-level projection head. Figure 2 Different shapes represent different categories.

[0071] For block-level contrast loss, the core principle is to enhance the feature stability of local areas of the same building in different contexts through self-supervised constraints, adapting to the cross-domain scenario requirements of monocular remote sensing 3D reconstruction. First, block sampling is performed on the unlabeled image of the target domain. A core region containing complete local semantic units (such as a building and its surrounding background) is randomly selected from a single image. Based on this, two blocks covering this region are generated. , .like Figure 3 As shown, the large box represents the target domain image, while the small box represents the overlapping portion of two randomly selected regions, i.e., the sampled feature block. Next, the blocks... and For feature processing, high-dimensional semantic features are first extracted by the backbone encoder, and then input into the block-level projection head to be mapped to a low-dimensional contrast feature space.

[0072] set up The feature map is extracted using a block-level projection head. For blocks , The number of pixels in the union, The same pixel representing the overlapping area will now be divided into blocks. middle Location characteristics With blocks middle Location characteristics Features at non-corresponding positions are considered positive sample pairs, and all other features are considered negative sample pairs, where positive sample pairs must maintain feature consistency. This is first achieved using the exponential cosine similarity formula. The quantification of feature associations is then performed, and the contrastive learning loss is calculated using the following formula: .

[0073] By optimizing the contrastive learning loss, features of the same pixel in the overlapping area are forced to be closer to each other in the low-dimensional space and further away from each other, thus achieving strict constraints on the stability of features in the same spatial region under different contexts.

[0074] set up , The weight parameters for the contrastive learning loss are used to calculate the total loss during the contrastive learning phase. By optimizing the total loss of the contrastive learning, we can obtain the final student model as the target model.

[0075] S5. Input the monocular remote sensing image to be processed into the target model. The target model outputs the building roof mask, roof footprint mask, building height, and offset vector between the roof and the footprint. Using the roof footprint mask as the bottom contour and the building height as the vertical dimension, construct three-dimensional coordinates to generate a 3D building model.

[0076] The monocular remote sensing image to be processed can be a source domain image or a target domain image.

[0077] To improve the reconstruction accuracy and consistency of 3D buildings, this embodiment introduces a roof translation verification mechanism: the predicted roof mask is translated according to the offset vector to obtain the theoretical footprint mask; the intersection-union ratio (IoU) between the theoretical footprint mask and the footprint mask directly predicted by the model is calculated; only building instances with an IoU not lower than a preset threshold (e.g., 0.85) are retained, and inconsistent prediction results are filtered out.

[0078] The building footprint mask verified by roof translation is geometrically regularized to correct its outline shape. Specifically, irregular edges (such as uneven boundaries) in the outline that do not conform to the building physical design logic are smoothed into regular straight line segments to make them conform to the geometric characteristics of typical buildings, thereby obtaining the calibrated footprint outline.

[0079] Based on the above output, a 3D reconstruction is performed: the horizontal range of the 3D model is determined by the vertex coordinates of the calibrated footprint contour, and the vertical dimension is extended by the instance-level building height value. Through coordinate transformation and connection with the geometric structure, the 2D plane information is stretched and closed in the vertical direction to generate a 3D building model that conforms to the real physical space structure.

[0080] Through feature distribution alignment during the domain adaptation phase, the student model can maintain consistent segmentation and height estimation accuracy with the source domain on the target domain data, thereby ensuring the reliability of 3D reconstruction results in cross-domain scenarios.

[0081] This embodiment also provides a domain-adaptive monocular remote sensing image 3D building reconstruction system, which is implemented using the above-described reconstruction method. The reconstruction system of this embodiment includes the following modules:

[0082] The initialization module is used to build a dual-model architecture consisting of a student model and a teacher model, and to initialize the models.

[0083] The source domain pre-training module trains the student model using a labeled source domain dataset, calculates semantic segmentation loss, height prediction loss, and offset prediction loss; at the same time, it updates the teacher model based on the training results of the student model.

[0084] The cross-domain fusion and pseudo-label generation module performs structured cross-domain mixing of source domain data and target domain data to generate hybrid data that combines the annotation characteristics of the source domain with the scene characteristics of the target domain. It calls the teacher model to generate high-confidence pixel category pseudo-labels for the hybrid data and target domain data, and filters pseudo-labels with confidence scores higher than a preset threshold.

[0085] The loss optimization module constructs positive and negative sample pairs, introduces a contrastive learning mechanism to further train the student model, learns domain-invariant features, optimizes the total loss, and obtains the final student model as the target model.

[0086] The reconstruction module inputs the monocular remote sensing image to be processed into the target model. The target model outputs the building roof mask, roof footprint mask, building height, and offset vector between the roof and the footprint. Using the roof footprint mask as the bottom contour and the building height as the vertical dimension, a three-dimensional coordinate system is constructed to generate a 3D building model.

[0087] Each of the above modules is implemented using steps S1-S5, and the detailed implementation process is described in the corresponding steps, which will not be repeated here.

[0088] This invention addresses the bottleneck of cross-regional applications by designing a domain adaptation technology specifically suited for monocular remote sensing 3D building reconstruction. For example, when a 3D reconstruction model trained on labeled urban area images is applied to cities in different countries, its performance significantly degrades due to domain differences such as building density, style, and imaging conditions. The domain adaptation method, however, can enable the model to adapt to new regional scenes without relying on additional or minimal annotations, effectively mitigating the performance loss caused by distribution offset and providing a feasible path for large-scale cross-domain building reconstruction.

[0089] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for reconstructing 3D buildings from domain-adaptive monocular remote sensing images, characterized in that, Includes the following steps: S1. Construct a dual-model architecture consisting of a student model and a teacher model, and initialize the models; S2. Train the student model using the labeled source domain dataset, and calculate the semantic segmentation loss, height prediction loss, and offset prediction loss; at the same time, update the teacher model based on the training results of the student model. S3. Perform structured cross-domain mixing of source domain data and target domain data to generate hybrid data that combines the characteristics of source domain annotation with the scene features of target domain. The teacher model is invoked to generate high-confidence pixel category pseudo-labels for mixed data and target domain data, and pseudo-labels with confidence scores higher than a preset threshold are filtered out. S4. Construct positive and negative sample pairs, introduce a contrastive learning mechanism to further train the student model, learn domain-invariant features, optimize the total loss, and obtain the final student model as the target model. S5. Input the monocular remote sensing image to be processed into the target model. The target model outputs the building roof mask, roof footprint mask, building height, and offset vector between the roof and the footprint. Using the roof footprint mask as the bottom contour and the building height as the vertical dimension, construct three-dimensional coordinates to generate a 3D building model.

2. The reconstruction method according to claim 1, characterized in that, In step S1, both the student model and the teacher model include a feature encoder, a multi-task prediction head, and a feature projection head. The weights of the teacher model are initialized to the initial weights of the student model. The feature encoder takes a monocular remote sensing image as input, extracts multi-scale visual features, and uses them as input to a multi-task prediction head. The multi-task prediction head includes a roof mask segmentation head, a building footprint mask segmentation head, a height prediction head, and an offset prediction head, which together adapt to the multi-objective requirements of building reconstruction tasks. Among them, the roof mask segmentation head is used to parse and output the roof pixel classification result from the input features; the building footprint mask segmentation head is used to parse and output the footprint pixel classification result from the input features; the height prediction head is used to output the pixel-level building height value; and the offset prediction head is used to calculate the two-dimensional plane offset of the roof pixel relative to its corresponding ground projection pixel and output the roof-footprint pixel-level offset vector field. The feature projection head is used to map the extracted features to the low-dimensional feature space required for contrastive learning, achieving pixel-level or block-level feature space transformation.

3. The reconstruction method according to claim 2, characterized in that, Feature projection heads are divided into pixel-level projection heads and block-level projection heads. The input of a pixel-level projection head is the feature output by the feature encoder after the image is processed, while the input of a block-level projection head is the two features output by the feature encoder after two upsampled blocks of an image are processed.

4. The reconstruction method according to claim 1, characterized in that, In step S2, for the task of predicting rooftops and building footprints, the semantic segmentation loss is calculated using the following formula: ； Among them, the semantic segmentation loss for the prediction tasks of rooftops and building footprints are respectively , uniformly recorded as ; Represents the total number of pixels in the image. The total number of categories, It is a one-hot vector. Pixels predicted by the model Category The probability of; For highly predictive tasks, record To accurately predict the loss, the calculation formula is as follows: ； in, , These are the model's predicted building height and the actual building height label, respectively. For the offset prediction task, let's call it... The offset prediction loss is calculated using the following formula: ； in, This represents the offset predicted by the model for a single pixel. This is a real label; Let the total loss of pre-training be... , Let the weights of the four loss functions be respectively, then the formula for calculating the total loss is: 。 5. The reconstruction method according to claim 1, characterized in that, Step S3 includes: S31. Perform cross-domain data fusion operation: Using a category-level copy and paste method, randomly select 50% of the semantic regions from the source domain data according to their categories; for the target domain data, perform preliminary semantic prediction using a pre-trained teacher model to identify and locate the non-building regions; then, paste the building regions cropped from the source domain data into the corresponding non-building semantic regions in the target domain data according to their semantic categories, thereby constructing cross-domain hybrid data that simultaneously contains the annotation characteristics of the source domain and the scene features of the target domain. S32. Perform pseudo-label generation and filtering: Call the teacher model to perform forward inference on the cross-domain mixed data and the target domain data respectively, and generate pixel-level category pseudo-labels to distinguish between buildings and backgrounds, and roofs and footprints; introduce a confidence threshold filtering mechanism to retain only pixel pseudo-labels with model prediction confidence higher than the set threshold, and filter out low-confidence parts; fuse source domain real labels and target domain pseudo-labels to generate mixed labels.

6. The reconstruction method according to claim 1, characterized in that, Step S4 introduces a two-dimensional contrastive learning mechanism at the pixel and block levels; Pixel-level contrastive learning is as follows: Based on the pseudo-labels generated by the teacher model, the pixels in the target domain image are classified into categories; pixel features belonging to the same category are formed into positive sample pairs, and pixel features belonging to different categories are formed into negative sample pairs; the features extracted by the encoder are mapped to a low-dimensional space through a pixel-level projection head, and the contrastive loss is calculated to bring the feature distance of positive sample pairs closer and push the feature distance of negative sample pairs further apart. Block-level contrastive learning involves sampling partially overlapping feature blocks from the feature map, treating feature vectors at the same spatial location within the same feature block as positive sample pairs, and feature vectors at different spatial locations as negative sample pairs; after mapping the features through a block-level projection head, the contrastive loss is calculated.

7. The reconstruction method according to claim 6, characterized in that, In step S4, the scope of positive sample pairs is expanded to include source and target domain images within the same training batch to cover feature diversity in cross-domain scenarios.

8. The reconstruction method according to claim 1, characterized in that, Step S5 introduces a roof translation verification mechanism, which translates the predicted roof mask according to the offset vector to obtain the theoretical footprint mask; calculates the intersection-union ratio (IoU) between the theoretical footprint mask and the footprint mask directly predicted by the model; and retains only building instances with an IoU not lower than a preset threshold, filtering out inconsistent prediction results.

9. The reconstruction method according to claim 8, characterized in that, Step S5 performs geometric regularization on the building footprint mask verified by roof translation to correct its outline shape. Specifically, it smooths irregular edges in the outline that do not conform to the building physical design logic into regular straight line segments so that they conform to the geometric characteristics of typical buildings, thereby obtaining the calibrated footprint outline.

10. A domain-adaptive monocular remote sensing image 3D building reconstruction system, implemented using the reconstruction method described in any one of claims 1-9, characterized in that, The reconstruction system includes: The initialization module is used to build a dual-model architecture consisting of a student model and a teacher model, and to initialize the models. The source domain pre-training module trains the student model using a labeled source domain dataset, calculates semantic segmentation loss, height prediction loss, and offset prediction loss; at the same time, it updates the teacher model based on the training results of the student model. The cross-domain fusion and pseudo-label generation module performs structured cross-domain mixing of source domain data and target domain data to generate hybrid data that combines the annotation characteristics of the source domain with the scene characteristics of the target domain. It calls the teacher model to generate high-confidence pixel category pseudo-labels for the hybrid data and target domain data, and filters pseudo-labels with confidence scores higher than a preset threshold. The loss optimization module constructs positive and negative sample pairs, introduces a contrastive learning mechanism to further train the student model, learns domain-invariant features, optimizes the total loss, and obtains the final student model as the target model. The reconstruction module inputs the monocular remote sensing image to be processed into the target model. The target model outputs the building roof mask, roof footprint mask, building height, and offset vector between the roof and the footprint. Using the roof footprint mask as the bottom contour and the building height as the vertical dimension, a three-dimensional coordinate system is constructed to generate a 3D building model.

Images