A multi-task method and related apparatus for extracting architectural geometric features from remote sensing images

Abstract

This invention discloses a multi-task method and related apparatus for extracting building geometric features from remote sensing images, belonging to the field of remote sensing building image processing technology. The method includes: acquiring an RGB remote sensing image to be predicted; preprocessing the RGB remote sensing image to obtain a preprocessed RGB remote sensing image; inputting the preprocessed RGB remote sensing image into a trained multi-task dual-channel attention U-Net network; and obtaining the building geometric features based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder. This method and related apparatus can accurately extract the building geometric features.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing building image processing technology, and relates to a multi-task extraction method and related apparatus for building geometric features from remote sensing images. Background Technology

[0002] With the acceleration of urbanization, building energy consumption and carbon emission assessment has become an important research direction for urban sustainable development. Building geometric parameters (such as building height, building area, and building outline) are key input data for calculating building carbon emissions, directly affecting the accuracy of operational and implicit carbon emission estimations. However, traditional methods for obtaining building geometric information mainly rely on manual on-site measurements or building archive searches, which are insufficient to meet the needs of large-scale building information extraction at the urban scale.

[0003] With the development of deep learning, many deep learning-based methods have been applied to building information extraction and have achieved great success. These methods can solve the problems of low efficiency, high cost, and limited coverage inherent in traditional building geometry information acquisition methods.

[0004] Classic methods based on multi-source data fusion mainly combine multiple data sources such as RGB orthophotos, LiDAR point clouds, and digital surface models (DSMs) to extract building height and contours. While these methods offer high accuracy, they rely on expensive multi-source data acquisition equipment (such as airborne LiDAR systems), resulting in high data acquisition costs, complex processing procedures, and registration errors between different data sources, which limits their widespread application in large urban areas.

[0005] Single-task deep learning methods primarily employ deep learning models such as convolutional neural networks (CNNs) to extract building segmentation masks or building heights individually from remote sensing imagery. These methods typically design and optimize models for a single task, requiring the training of multiple independent models to obtain different building geometric parameters. This leads to problems such as model redundancy, wasted computational resources, and insufficient feature representation, and they cannot leverage inter-task correlations for joint optimization.

[0006] Traditional multi-task deep learning methods achieve parallel prediction across multiple tasks by sharing an encoder and multiple decoding branches. However, traditional deep learning models (such as Unet) employ a simple feature concatenation strategy in skip connections, failing to effectively model the semantic relationships between encoder and decoder features. This leads to problems such as task competition, gradient conflicts, and insufficient feature representation in multi-task learning scenarios. Especially when heterogeneous tasks such as building height regression and building segmentation coexist, the model often favors one task while ignoring others, resulting in a decline in overall performance and poor accuracy of extracted geometric features. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-task method and related apparatus for extracting geometric features of buildings from remote sensing images. This method and related apparatus can accurately extract the geometric features of buildings.

[0008] To achieve the above objectives, this invention discloses a multi-task method for extracting architectural geometric features from remote sensing images, comprising: Acquire RGB remote sensing images to be predicted; The RGB remote sensing image to be predicted is preprocessed to obtain the preprocessed RGB remote sensing image. The preprocessed RGB remote sensing image is input into a trained multi-task dual-channel attention U-Net network. The geometric features of the building are obtained based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder.

[0009] Furthermore, the dual-channel cross-attention module combines spatial cross-attention and channel cross-attention sub-modules to enhance multi-scale features and optimize collaborative learning between tasks; The spatial cross-attention submodule generates a query vector Q through 1×1 convolution, and combines it with the key vector K and value vector V generated by the encoder jump features to generate a spatial attention map. The value vector V is then weighted to obtain spatial attention-enhanced features. The channel cross-attention submodule generates a global descriptor through global adaptive average pooling, and combines the two to generate channel attention weights to enhance the representational ability of the encoder jump features.

[0010] Furthermore, the total loss function of the multi-task dual-channel attention U-Net network during training is constructed by weighted summation of the building height regression loss L_height, building area regression loss L_area, and building segmentation loss L_building.

[0011] Furthermore, the AdamW optimizer is used to train the multi-task dual-channel attention U-Net network, a mixed-precision training strategy is employed, and gradients are pruned during the training process.

[0012] Furthermore, the preprocessing process for the RGB remote sensing image to be predicted is as follows: The RGB remote sensing image to be predicted is sliced ​​and normalized.

[0013] Furthermore, the process of inputting the preprocessed RGB remote sensing image into the trained multi-task dual-channel attention U-Net network, and obtaining the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network is as follows: The preprocessed RGB remote sensing image is input into the trained multi-task dual-channel attention U-Net network to obtain building height prediction map, building area prediction map and building segmentation logits; The predicted building height map is denormalized and multiplied by the maximum height threshold to obtain the building height in meters. The predicted building area map is inversely normalized to obtain the building area in square meters. The building segmentation logits are binarized by applying the Sigmoid function and using a threshold of 0.5 to obtain the building segmentation mask.

[0014] This invention discloses a multi-task system for extracting architectural geometric features from remote sensing images, comprising: The acquisition module is used to acquire the RGB remote sensing image to be predicted; The preprocessing module is used to preprocess the RGB remote sensing image to be predicted to obtain the preprocessed RGB remote sensing image. The extraction module is used to input the preprocessed RGB remote sensing image into the trained multi-task dual-channel attention U-Net network, and obtain the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder.

[0015] Furthermore, the process of inputting the preprocessed RGB remote sensing image into the trained multi-task dual-channel attention U-Net network, and obtaining the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network is as follows: The preprocessed RGB remote sensing image is input into the trained multi-task dual-channel attention U-Net network to obtain building height prediction map, building area prediction map and building segmentation logits; The predicted building height map is denormalized and multiplied by the maximum height threshold to obtain the building height in meters. The predicted building area map is inversely normalized to obtain the building area in square meters. The building segmentation logits are binarized by applying the Sigmoid function and using a threshold of 0.5 to obtain the building segmentation mask.

[0016] This invention discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the multi-task extraction method for building geometric features from remote sensing images.

[0017] This invention discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the multi-task extraction method for building geometric features from remotely sensed images.

[0018] The present invention has the following beneficial effects: The multi-task extraction method and related apparatus for building geometric features from remote sensing images described in this invention, in specific operation, inputs the preprocessed RGB remote sensing image into a trained multi-task dual-channel attention U-Net network. The geometric features of the building are obtained based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder. The dual-channel cross-attention module, through the collaborative fusion of spatial attention and channel attention, utilizes high-level semantic information from the decoder to dynamically correct encoder skip features, effectively solving the semantic gap and multi-task competition problems caused by the simple feature stitching of traditional U-Net, thus accurately extracting the geometric features of the building and demonstrating strong practicality. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the U-Net encoder-decoder structure and feature map size changes; Figure 2 Diagram of shared encoder and three-branch decoder architecture; Figure 3 This is a schematic diagram of the DCAModule structure; Figure 4 This is a schematic diagram of an example of an input remote sensing image in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0025] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0026] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0028] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0029] Example 1 refer to Figure 1 , Figure 2 , Figure 3 and Figure 4 The multi-task extraction method for building geometric features from remote sensing images according to the present invention includes the following steps; 1) Acquire and preprocess remote sensing image data; The specific operation of step 1) is as follows: 11) Obtain a remote sensing dataset containing RGB orthophotos, ground height annotations (AGL), building footprint annotations (BLDG_FTPRINT), and semantic classification annotations (CLS); 12) The original large-scale remote sensing image was sliced ​​using a sliding window strategy. The slice size was set to 512×512 pixels, and an overlap area of ​​128 pixels was set between adjacent slices to ensure the integrity of the boundaries. 13) Normalize the RGB image, mapping the pixel values ​​from the [0,255] range to the [0,1] range; 14) Normalize the ground height labels by dividing the height value by the preset maximum height threshold (60 meters) to obtain the normalized height labels; 15) Perform a logical OR operation between the building footprint annotations and the building categories in the semantic classification to obtain binary building segmentation labels; 16) Count the number of pixels within the building mask and multiply it by the actual area corresponding to each pixel, then normalize it.

[0030] 2) Construct a multi-task, dual-channel attention U-Net network architecture; 21) Construct a shared encoder; 211) The input layer receives 3-channel RGB images with a size of 512×512×3; 212) The initial convolutional layer (inc) uses a dual convolutional module to map the input features from 3 channels to the basic number of channels C (C=32). Each dual convolutional module contains two 3×3 convolutional layers, a batch normalization layer and a ReLU activation function. 213) The first downsampling layer (down1) downsamples the features to 256×256 resolution through 2×2 max pooling and double convolution modules, expanding the number of channels to 2C; 214) The second downsampling layer (down2) downsamples the features to 128×128 resolution, expanding the number of channels to 4C; 215) The third downsampling layer (down3) downsamples the features to 64×64 resolution, expanding the number of channels to 8C; 216) The fourth downsampling layer (down4) downsamples the features to 32×32 resolution and expands the number of channels to 16C, thus obtaining the bottleneck layer features; 22) Construct a dual-channel cross-attention module (DCAModule): 221) Spatial Cross-Attention Submodule: 2211) Generate a query vector Q from the decoder features using a 1×1 convolution, with dimensions [B, E, H, W], where B is the batch size, E is the embedding dimension, and H and W are the feature map height and width;

[0031] 2212) Generate a key vector K with dimensions [B, E, H, W] by performing a 1×1 convolution on the skip features of the encoder. 2213) Generate a value vector V by 1×1 convolution of the encoder skip features, with dimensions [B, C_kv, H, W], where C_kv is the number of encoder skip feature channels; 2214) Concatenate Q and K along the channel dimension to obtain [B, 2E, H, W], and generate a spatial attention map with dimensions [B, 1, H, W] by using a 3×3 convolution and a sigmoid activation function;

[0032] 2215) Multiply the spatial attention map with V, and after batch normalization, perform residual connection with the original skip features to obtain the spatial attention enhancement features; 222) Channel Cross-Attention Submodule: 2221) Global adaptive average pooling is performed on the decoder features and encoder skip features respectively to obtain global descriptors with dimensions [B, C_q] and [B, C_kv]; 2223) Map the two global descriptors to the hidden dimension H_dim through a fully connected layer, and then activate them using ReLU;

[0033] 2224) After adding the two hidden vectors, map them back to the C_kv dimension through a fully connected layer, and then obtain the channel attention weights by sigmoid activation; 2225) After reshaping the channel attention weights to [B, C_kv, 1, 1], multiply them with the encoder skip features to obtain the channel attention enhancement features; 223) Feature fusion: 2231) Spatial attention enhancement features and channel attention enhancement features are concatenated along the channel dimension to obtain [B, 2C_kv, H, W]; 2232) The concatenated features are fused into [B, C_kv, H, W] through 1×1 convolution, batch normalization and ReLU activation, which serves as the final output of the dual-channel cross-attention module; 23) Construct a decoder upsampling layer (DCAUp) with dual-channel cross-attention: 231) The decoder features of the input are upsampled by a factor of 2 using bilinear interpolation or transposed convolution; 232) Align the upsampled features with the encoder jump features of the corresponding level in terms of spatial dimensions by using zero padding; 233) The upsampled decoder features are used as query Q and the encoder skip features are used as key-value pairs K / V, and then input into the dual-channel cross-attention module described in step S2.2 for feature enhancement; 234) Concatenate the attention-enhanced jump features with the upsampled features along the channel dimension; 235) The concatenated features are fused and extracted using a dual convolution module; 24) Construct a multi-task decoding branch: 241) Shared decoding path: 2411) The first upsampling layer (up1) upsamples the 32×32 features to 64×64 through the DCAUp module, reducing the number of channels to 8C; 2412) The second upsampling layer (up2) upsamples the 64×64 features to 128×128 through the DCAUp module, reducing the number of channels to 4C; 2413) The third upsampling layer (up3) upsamples the 128×128 features to 256×256 through the DCAUp module, reducing the number of channels to 2C; 2414) The fourth upsampling layer (up4) upsamples the 256×256 features to 512×512 through the DCAUp module, reducing the number of channels to C; 242) Building height prediction branch: The 512×512×C features are mapped to a 512×512×1 normalized height prediction map through 1×1 convolution, and the output is restricted to the [0, 1] interval by the Sigmoid activation function; 243) Building area prediction branch: The 512×512×C feature is mapped to a 512×512×1 normalized area prediction map through 1×1 convolution, and the output is restricted to the [0, 1] interval through the Sigmoid activation function; 244) Building segmentation prediction branch: The 512×512×C features are mapped to 512×512×1 building segmentation original confidence scores logits through 1×1 convolution, and output directly without going through an activation function; 3) Design a multi-objective weighted loss function; 31) Calculate the regression loss L_height for building height, specifically: 311) Calculate the squared error between the predicted height and the actual height; 312) Use building segmentation masks to weight the loss, with the weight of the loss within the building area set to w_h (w_h=10) and the weight of the loss within the non-building area set to 1; 313) Calculate the average of the weighted losses to obtain L_height; 32) Calculate the regression loss L_area for building area, specifically: 321) Calculate the squared error between the predicted area and the actual area; 322) Use building segmentation masks to weight the loss, with the weight of the loss within the building area set to w_a (w_a=5) and the weight of the loss within the non-building area set to 1; 323) The average of the weighted losses is used to obtain L_area; 33) Calculate the building segmentation loss L_building, specifically: 331) Employ a binary cross-entropy loss function with logits (L... BCE The predicted segmentation logits are compared with the true binary mask.

[0034] in This represents the raw confidence score logits of the model output. Indicates the true label (0 or 1).

[0035] This represents the Sigmoid activation function.

[0036] 332) The loss function automatically performs Sigmoid activation to ensure numerical stability; 34) Calculate the total loss L_total:

[0037] 341) Calculate the weighted total loss according to the formula, where the task weights are set as follows: height weight α = 1.0, area weight β = 1.0, and segmentation weight γ = 2.0. The segmentation weight γ is set to twice the weights of other tasks to address the problem that segmentation tasks are easily suppressed in multi-task learning; 4) Model training and optimization; 41) The AdamW optimizer is used for parameter updates, with an initial learning rate set to 1×10⁻⁶. -3 The weight decay factor is set to 1×10. -4 ; 42) A cosine annealing learning rate scheduling strategy is adopted, which decays the learning rate from the initial cosine value to the minimum value η_min (η_min=1×10) within the training period T_max. -6 ); 43) Enable Automatic Mixed Precision (AMP) training, using FP16 floating-point format for forward propagation and gradient calculation, and FP32 floating-point format for weight updates to accelerate training and reduce GPU memory usage; 44) Set the gradient clipping threshold to 1.0 to clip the L2 norm of the gradient to prevent gradient explosion; 45) Apply data augmentation strategies during the training data loading process, including random horizontal flipping, random vertical flipping, random rotation, random brightness adjustment, and random contrast adjustment; 46) Set the batch size to 4, the training cycle to 300 rounds, use 8 threads for parallel data loading and enable pin memory (pin_memory) to improve data loading efficiency; 47) After each training cycle, evaluate the model performance on the validation set, and calculate the total validation loss and the mean absolute error (MAE) of the height. height ), Area Mean Absolute Error (MAE)Area ) and the intersection-union ratio (IoU) of building segments;

[0038] in This represents the predicted height of the i-th pixel. This represents the actual height of the i-th pixel.

[0039] This represents the total number of pixels in the building area.

[0040]

[0041] in This represents the predicted area of ​​the i-th sample. Let N represent the true area of ​​the i-th sample, and N represent the total number of samples.

[0042]

[0043] Where TP represents a sample correctly predicted as a building, FP represents a sample incorrectly predicted as a building, FN represents a missed building, P represents the predicted building mask, and G represents the true building mask. Represents the correctly predicted building pixels. Represents all building pixels in the prediction or reality.

[0044] 48) When the total loss of the validation set reaches the historical minimum, save the current model weights as the best model (checkpoint_best.pth). 49) Save the current model checkpoint every 10 training cycles for breakpoint continuation training and ablation experiments; 5) Model reasoning and architectural geometric feature extraction; 51) Load the optimal model weights after training; 52) Perform the same normalization preprocessing as in step S1 on the RGB remote sensing image to be predicted; 53) Input the preprocessed image into the trained multi-task dual-channel attention U-Net network; 54) Extract the prediction results for the three tasks from the model output: 541) Perform inverse normalization on the building height prediction map and multiply it by the maximum height threshold (60 meters) to obtain the building height in meters; 542) Perform inverse normalization on the predicted building area map to obtain the building area in square meters; 543) Apply the Sigmoid function to the building segmentation logits and binarize it with a threshold of 0.5 to obtain the building segmentation mask; 55) Estimate building carbon emissions or other downstream applications based on extracted building geometry features (height, area, profile).

[0045] Example 2 The multi-task extraction method for building geometric features from remote sensing images based on dual-channel cross-attention, as described in this invention, includes the following steps: First, the remote sensing image data is preprocessed to construct input samples and labeled data suitable for multi-task learning; then, a multi-task dual-channel attention U-Net network architecture is built, basic features are extracted through a shared encoder, and a dual-channel cross-attention module is introduced in the decoding stage to enhance the interaction between the encoder's skip connection features and the decoder's semantic features; further, a multi-objective weighted loss function is designed for building height prediction, building area prediction, and building segmentation tasks, and the network is jointly trained and optimized; finally, the trained model is used to infer the input RGB remote sensing images, outputting geometric features such as building height, building area, and building outline for use in building carbon emission estimation or other downstream tasks. Specifically, the following steps are included: 1) Remote Sensing Image Data Preprocessing. A remote sensing dataset containing RGB orthophotos, ground elevation labels, building footprint labels, and semantic classification labels was acquired. Due to the large size of the original remote sensing images, a sliding window method was used to slice the original images to facilitate network training and inference. The slice size was set to 512×512 pixels, with a 128-pixel overlap between adjacent slices to avoid information loss when building targets are located at image edges. The input RGB images were normalized, mapping pixel values ​​from [0,255] to the [0,1] range. Ground elevation labels were normalized by dividing the height values ​​by a preset maximum height threshold of 60 meters to obtain normalized height labels. A logical OR operation was performed between the building footprint labels and the building categories in the semantic classification labels to construct binary building segmentation labels. Simultaneously, the number of pixels within the building mask area was counted, and the building area label was calculated based on the actual area corresponding to each pixel, then normalized to form training labels corresponding to the height prediction, area prediction, and building segmentation tasks.

[0046] 2) Constructing a multi-task dual-channel attention U-Net network architecture. The network adopts an encoder-decoder structure, where multi-scale features are transferred between the encoder and decoder via skip connections. First, a shared encoder is constructed. The input layer receives an RGB image of size 512×512×3. After passing through an initial double convolutional module, the number of channels is mapped to a base number of channels C, where C is 32. Then, four downsampling layers are passed sequentially. Each layer uses a combination of 2×2 max pooling and double convolutional modules to reduce spatial resolution and increase the number of channels, reducing the feature map size from 512×512 to 256×256, 128×128, 64×64, and 32×32, respectively. Simultaneously, the number of channels is expanded to 2C, 4C, 8C, and 16C, respectively, resulting in the bottleneck layer features. The shared encoder can extract texture, edge, semantic, and structural information of buildings layer by layer, providing a unified feature foundation for subsequent multi-task prediction.

[0047] In the decoder section, to overcome the semantic gap caused by simple feature concatenation in traditional U-Net skip connections, this invention introduces a dual-channel cross-attention module in each upsampling layer. Specifically, the low-resolution decoder features are first upsampled by a factor of 2, and then aligned in size with the corresponding encoder skip features. The dual-channel cross-attention module consists of two parts: a spatial cross-attention submodule and a channel cross-attention submodule. In the spatial cross-attention submodule, the decoder features are mapped to a query vector Q through a 1×1 convolution, and the encoder skip features are mapped to a key vector K and a value vector V through 1×1 convolutions, respectively. Then, Q and K are concatenated along the channel dimension, and a spatial attention map is generated through a 3×3 convolution and a sigmoid activation function. This spatial attention map is then used to weight V to obtain the spatial attention-enhanced features. In the channel cross-attention submodule, global adaptive average pooling is applied to both the decoder features and the encoder skip features to obtain two global descriptive vectors. These vectors are then mapped to the hidden dimension through a fully connected layer, activated, fused, and mapped back to the encoder feature channel dimension. Channel attention weights are generated using the Sigmoid function, and the encoder skip features are then recalibrated to obtain channel attention-enhanced features. Finally, the spatial attention-enhanced features and the channel attention-enhanced features are concatenated along the channel dimension and fused using 1×1 convolution, batch normalization, and ReLU activation to output the dual-channel cross-attention-enhanced features.

[0048] Based on the aforementioned dual-channel cross-attention module, this invention constructs a decoder upsampling layer with dual-channel cross-attention. The shared decoding path includes four consecutive upsampling stages, restoring 32×32 features to 64×64, 128×128, 256×256, and 512×512 respectively, with the number of channels decreasing to 8C, 4C, 2C, and C respectively. After completing the shared decoding, three task output branches are further set. Among them, the building height prediction branch maps 512×512×C features to a 512×512×1 normalized height prediction map through a 1×1 convolution, and limits the output range through a Sigmoid function; the building area prediction branch also outputs a 512×512×1 normalized area prediction map through a 1×1 convolution, and constrains it to the [0,1] interval through a Sigmoid function; the building segmentation prediction branch outputs building segmentation logits through a 1×1 convolution, which are used to generate subsequent binary segmentation results. By using a shared feature extraction and task branch decoupling design, the network can simultaneously predict building height, building area, and building outline within the same framework.

[0049] 3) Design of a multi-objective weighted loss function. Addressing the issue that building height prediction and building area prediction are regression tasks, while building segmentation is a classification task with different loss scales, this invention designs a multi-objective weighted loss function to balance the learning process of each task. For the building height prediction task, a weighted mean squared error is used as the height regression loss, with a higher weight (w_h=10) in the building area and a weight of 1 in the non-building area, guiding the model to focus on estimating the height of the building itself. For the building area prediction task, the same weighted mean squared error is used as the area regression loss, with a weight of w_a=5 in the building area and a weight of 1 in the non-building area, to improve the representation ability of the building area. For the building segmentation task, a binary cross-entropy loss function with logits is used, directly comparing the original confidence score output by the segmentation branch with the true binary mask to improve numerical stability. Finally, the losses from the three tasks are weighted and summed to form the total loss function, where the height loss weight α is set to 1.0, the area loss weight β is set to 1.0, and the segmentation loss weight γ is set to 2.0, in order to enhance the contribution of the segmentation task in multi-task joint training and avoid its suppression by the regression task.

[0050] 4) Model Training and Optimization. The AdamW optimizer was used to update model parameters, with an initial learning rate of 1×10^-3 and a weight decay coefficient of 1×10^-4. A cosine annealing learning rate scheduling strategy was employed to gradually decay the learning rate from its initial value to a minimum of 1×10^-6 during training, improving training stability. A mixed-precision training strategy was used during training, employing FP16 format for forward propagation and gradient calculation, and FP32 format for weight updates, thereby reducing memory usage and increasing training speed. A gradient clipping threshold of 1.0 was set to clip the L2 norm of gradients to prevent gradient explosion during training. During the data loading phase, random horizontal flipping, random vertical flipping, random rotation, random brightness adjustment, and random contrast adjustment were applied to the training samples to improve the model's generalization ability. The batch size was set to 4, the number of training epochs was set to 300, and 8 threads were used for parallel data loading. After each training cycle, the model is evaluated on the validation set, and the total validation set loss, mean absolute error of building height, mean absolute error of building area, and intersection-over-union ratio of building segmentation are calculated. When the total validation set loss reaches the historical best, the current model weights are saved as the best model.

[0051] 5) Model Inference and Building Geometric Feature Extraction. First, the optimal model weights from the trained system are loaded, and the RGB remote sensing image to be predicted undergoes the same normalization preprocessing as during the training phase. Then, the preprocessed remote sensing image is input into the trained multi-task dual-channel attention U-Net network to obtain building height prediction maps, building area prediction maps, and building segmentation logits. The height prediction map is inversely normalized by multiplying it by a maximum height threshold of 60 meters to obtain the building height result in meters; the area prediction map is inversely normalized to obtain the building area result in square meters; and the building segmentation logits are binarized using a Sigmoid function with a threshold of 0.5 to obtain the building outline mask. Through this method, three types of building geometric features—building height, building area, and building outline—can be simultaneously extracted from a single RGB remote sensing image. These extracted geometric features can then be used in downstream applications such as building carbon emission estimation, urban building information modeling, urban renewal assessment, or disaster emergency analysis.

[0052] This invention has the following characteristics: This invention requires only a single RGB remote sensing image as input to simultaneously extract three geometric parameters: building height, building area, and building outline, without relying on expensive multi-source auxiliary data such as LiDAR point clouds and digital surface models (DSMs). Compared to traditional methods that require airborne LiDAR systems (costing tens to hundreds of thousands of yuan per flight) or satellite DSM data (with limited coverage and long update cycles), this invention reduces data acquisition costs by more than 80%. Furthermore, RGB imagery has wide access channels (drones, satellites, aerial photography) and high timeliness, making it suitable for rapid building information extraction in large-scale urban areas. This provides a low-cost, high-efficiency technical solution for applications such as urban planning, carbon emission monitoring, and disaster assessment. In addition, the single data source avoids the registration error problem of multi-source data, simplifies the data processing workflow, and improves the robustness and practicality of the system.

[0053] The proposed Dual-Channel Cross-Attention Module (DCAModule) effectively addresses the semantic gap and multi-task competition issues caused by the simple feature concatenation in traditional U-Net by synergistically fusing spatial and channel attention and dynamically correcting encoder skip features using high-level semantic information from the decoder. Experimental results show that compared to the traditional U-Net architecture, this invention improves IoU by 15-20 percentage points (from 0.50 to 0.66) on building segmentation tasks, reduces MAE for height prediction by 25% (from 0.068 meters to 0.051 meters), and reduces MAE for area prediction by 18% (from 0.094 m² to 0.077 m²). The DCA module uses local convolutional attention instead of global self-attention, reducing computational complexity from O(N²) to O(N), reducing memory usage by 60%, and supporting larger batch training and higher resolution input. Furthermore, ablation experiments verify that the DCA module improves model convergence speed by 30% and produces a smoother and more stable loss reduction curve on the validation set, demonstrating the effectiveness and generalization ability of this mechanism in multi-task learning.

[0054] This invention addresses the problem of large differences in loss scales and training imbalance in three heterogeneous tasks: building segmentation, height regression, and area regression. It designs an adaptive weighted loss function and a region-weighted strategy. By setting the weight of the segmentation task to twice that of the regression task (γ=2.0), the prediction failure caused by the suppression of the segmentation task is effectively solved (the IoU was 0 when the original weight was 0.5, but increased to 0.66 after adjustment). Simultaneously, the height and area losses within the building area are weighted by 10 and 5 times respectively, allowing the model to focus on the accurate prediction of the building itself and reduce background noise interference. Experiments show that this loss balancing strategy synchronously improves the validation set metrics for all three tasks, reducing the total loss from the initial 1.20 to 0.78 (a 35% reduction), without any overfitting or underfitting of any task during training. Furthermore, by combining mixed precision training (AMP), cosine annealing learning rate scheduling, gradient pruning, and other optimization techniques, the model training speed is improved by 40%, and the training time per round is reduced from 12 minutes to 7 minutes, providing an efficient solution for training large-scale datasets. The extracted building geometric parameters can be directly used for building carbon emission calculations. The correlation between the parameters and the measured data has been verified to be above 0.92, demonstrating significant practical value and social benefits.

[0055] Example 3 The multi-task extraction system for building geometric features from remote sensing images according to the present invention includes: The acquisition module is used to acquire the RGB remote sensing image to be predicted; The preprocessing module is used to preprocess the RGB remote sensing image to be predicted to obtain the preprocessed RGB remote sensing image. The extraction module is used to input the preprocessed RGB remote sensing image into the trained multi-task dual-channel attention U-Net network, and obtain the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder.

[0056] Furthermore, the process of inputting the preprocessed RGB remote sensing image into the trained multi-task dual-channel attention U-Net network, and obtaining the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network is as follows: The preprocessed RGB remote sensing image is input into the trained multi-task dual-channel attention U-Net network to obtain building height prediction map, building area prediction map and building segmentation logits; The predicted building height map is denormalized and multiplied by the maximum height threshold to obtain the building height in meters. The predicted building area map is inversely normalized to obtain the building area in square meters. The building segmentation logits are binarized by applying the Sigmoid function and using a threshold of 0.5 to obtain the building segmentation mask.

[0057] The module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in each embodiment of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0058] Example 4 A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a multi-task method for extracting building geometric features from remotely sensed images. For example, the method includes: acquiring an RGB remotely sensed image to be predicted; preprocessing the RGB remotely sensed image to obtain a preprocessed RGB remotely sensed image; inputting the preprocessed RGB remotely sensed image into a trained multi-task dual-channel attention U-Net network; and obtaining the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder. The memory may include main memory, such as high-speed random access memory, or it may also include non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus, which may be an industry-standard architecture bus, a peripheral component interconnection standard bus, or an extended industry-standard architecture bus. The bus may be categorized as an address bus, a data bus, or a control bus. The memory is used to store programs; specifically, the program may include program code, which includes computer operation instructions. The memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0059] Example 5 A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a multi-task method for extracting architectural geometric features from remotely sensed images. For example, the method includes: acquiring an RGB remotely sensed image to be predicted; preprocessing the RGB remotely sensed image to obtain a preprocessed RGB remotely sensed image; inputting the preprocessed RGB remotely sensed image into a trained multi-task dual-channel attention U-Net network; and obtaining the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0060] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and disclosure of the invention. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0065] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

[0066] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A multi-task method for extracting architectural geometric features from remote sensing images, characterized in that, include: Acquire RGB remote sensing images to be predicted; The RGB remote sensing image to be predicted is preprocessed to obtain the preprocessed RGB remote sensing image. The preprocessed RGB remote sensing image is input into a trained multi-task dual-channel attention U-Net network. The geometric features of the building are obtained based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder.

2. The multi-task extraction method for architectural geometric features from remote sensing images according to claim 1, characterized in that, The dual-channel cross-attention module combines spatial cross-attention and channel cross-attention sub-modules to enhance multi-scale features and optimize collaborative learning between tasks. The spatial cross-attention submodule generates a query vector Q through 1×1 convolution, and combines it with the key vector K and value vector V generated by the encoder jump feature to generate a spatial attention map and weight the value vector V to obtain spatial attention enhancement features. The channel cross-attention submodule generates a global descriptor through global adaptive average pooling, and combines the two to generate channel attention weights to enhance the encoder's ability to represent skip features.

3. The multi-task extraction method for architectural geometric features from remote sensing images according to claim 1, characterized in that, The total loss function of the multi-task dual-channel attention U-Net network during training is constructed by weighted summation of the building height regression loss L_height, building area regression loss L_area, and building segmentation loss L_building.

4. The multi-task extraction method for architectural geometric features from remote sensing images according to claim 1, characterized in that, The AdamW optimizer is used to train the multi-task dual-channel attention U-Net network, a mixed-precision training strategy is used, and gradients are clipped during the training process.

5. The multi-task extraction method for architectural geometric features from remote sensing images according to claim 1, characterized in that, The process of preprocessing the RGB remote sensing image to be predicted is as follows: The RGB remote sensing image to be predicted is sliced ​​and normalized.

6. The multi-task extraction method for architectural geometric features from remote sensing images according to claim 1, characterized in that, The process of inputting the preprocessed RGB remote sensing image into a trained multi-task dual-channel attention U-Net network, and obtaining the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network is as follows: The preprocessed RGB remote sensing image is input into the trained multi-task dual-channel attention U-Net network to obtain building height prediction map, building area prediction map and building segmentation logits; The predicted building height map is denormalized and multiplied by the maximum height threshold to obtain the building height in meters. The predicted building area map is inversely normalized to obtain the building area in square meters. The building segmentation logits are binarized by applying the Sigmoid function and using a threshold of 0.5 to obtain the building segmentation mask.

7. A multi-task system for extracting architectural geometric features from remote sensing images, characterized in that, include: The acquisition module is used to acquire the RGB remote sensing image to be predicted; The preprocessing module is used to preprocess the RGB remote sensing image to be predicted to obtain the preprocessed RGB remote sensing image. The extraction module is used to input the preprocessed RGB remote sensing image into the trained multi-task dual-channel attention U-Net network, and obtain the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network. The multi-task dual-channel attention U-Net network includes a shared encoder, a dual-channel cross-attention module, and a decoder. Feature enhancement is performed through the dual-channel cross-attention module, and multi-task output is performed through the decoder.

8. The multi-task extraction system for architectural geometric features from remote sensing images according to claim 7, characterized in that, The process of inputting the preprocessed RGB remote sensing image into a trained multi-task dual-channel attention U-Net network, and obtaining the geometric features of the building based on the output of the trained multi-task dual-channel attention U-Net network is as follows: The preprocessed RGB remote sensing image is input into the trained multi-task dual-channel attention U-Net network to obtain building height prediction map, building area prediction map and building segmentation logits; The predicted building height map is denormalized and multiplied by the maximum height threshold to obtain the building height in meters. The predicted building area map is inversely normalized to obtain the building area in square meters. The building segmentation logits are binarized by applying the Sigmoid function and using a threshold of 0.5 to obtain the building segmentation mask.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the multi-task extraction method for building geometric features from remote sensing images as described in any one of claims 1-6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-task extraction method for building geometric features from remote sensing images as described in any one of claims 1-6.

Images