A historical remote sensing image built-up area extraction method based on a semantic segmentation model

Abstract

This invention discloses a method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model, belonging to the fields of geographic information, artificial intelligence, and remote sensing technology. The method includes: S1: acquiring multi-source data and preprocessing it; S2: training a model based on the preprocessed multi-source data and a semantic segmentation model; S3: after training, inputting the historical remote sensing image into the panchromatic image semantic segmentation model and outputting the image segmentation result. This invention, through effective data preprocessing and multi-source data fusion methods, fully utilizes the rich information in historical remote sensing data, solving problems such as inconsistent data formats and poor spatiotemporal consistency, thereby improving data usability and the accuracy of analysis results.

Description

Technical Field

[0001] This invention belongs to the fields of geographic information, artificial intelligence and remote sensing technology, and specifically relates to a method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model. Background Technology

[0002] Historical remote sensing imagery is crucial data recording the geographic information and spatial state of the Earth's surface during specific historical periods, and it is of great significance for understanding urban changes and land use shifts. Currently, land cover extraction techniques based on remote sensing imagery have evolved to a stage where multiple methods are applied in parallel, including traditional methods based on spectral features, machine learning methods, and deep learning methods. However, existing technologies have several shortcomings when processing historical remote sensing data:

[0003] (1) Historical remote sensing data comes from diverse sources, has inconsistent data formats, and poor spatiotemporal consistency, which makes data fusion difficult and affects the accuracy of analysis results.

[0004] (2) The methods for extracting urban features are relatively simple and lack in-depth exploration of the complex changes in urban planar morphology.

[0005] (3) The complexity and heterogeneity of historical remote sensing data increase the technical difficulty of data processing and fusion, making it difficult for existing technologies to fully extract the rich historical information contained in historical remote sensing data.

[0006] (4) The lack of spectral information in single-channel panchromatic images limits the accuracy of ground feature extraction, especially when dealing with complex ground features or ground features with small details.

[0007] Currently, historical remote sensing data has been applied to some extent in urban area extraction. By acquiring remote sensing images from different periods, changes in the urban built-up area and land use types can be observed intuitively.

[0008] In recent years, the technology of extracting ground features based on remote sensing images has gradually developed into a stage where multiple methods are applied in parallel, mainly including traditional methods based on spectral features, machine learning methods, and deep learning methods.

[0009] However, existing technologies have several shortcomings in processing historical remote sensing data to extract urban built-up areas. First, historical remote sensing data comes from diverse sources, including images from different sensors and at different resolutions. These data have inconsistent formats and poor spatiotemporal consistency, making data fusion difficult and affecting the accuracy of the analysis results. Second, urban feature extraction methods are relatively simplistic, mostly focusing on statistical analysis of urban built-up area or simple spatial distribution analysis, lacking in-depth exploration of the complex changes in urban morphology. Furthermore, the inherent complexity and heterogeneity of historical remote sensing data increase the technical difficulty of data processing and fusion, making it impossible for existing technologies to fully extract the rich historical information contained within the data. Summary of the Invention

[0010] This invention aims to address the problems of data fusion difficulties and incomplete feature extraction when extracting urban building areas and road networks using historical remote sensing data in existing technologies. It proposes a method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model, which can efficiently integrate multi-source historical remote sensing data, accurately extract geographic elements, and perform semantic segmentation.

[0011] Technical Solution: To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0012] A method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model is proposed. This method utilizes an adaptive framework based on deep learning and semantic segmentation, combined with georegistration using modern imagery. It employs an adversarial domain adaptation mechanism, a hybrid loss function design, and a lightweight model architecture to achieve efficient and automated segmentation of historical panchromatic images from Keyhole satellites, extracting core geographic features. The specific steps are as follows:

[0013] S1: Acquire multi-source data and preprocess it;

[0014] S2: Model training is performed based on preprocessed multi-source data and a general semantic segmentation model;

[0015] S3: After training, input the historical remote sensing images into the panchromatic image semantic segmentation model and output the image segmentation results.

[0016] As a preferred option, the specific content of S1 is as follows:

[0017] S11: Acquire historical panchromatic image data from the Keyhole satellite, perform strip image mosaicking and geometric registration, georegister remote sensing images, overlay vector layers, and extract the global image by masking.

[0018] S12: Acquire raw GF-2 satellite imagery data and perform cropping, mean averaging, and data augmentation.

[0019] As a preferred option, the specific implementation process in S11 is as follows:

[0020] Preprocessing of historical panchromatic imagery data from Keyhole satellite in ArcGIS Pro environment:

[0021] First, create a mosaic dataset and import the original scanned image, while setting the correct coordinate system and pixel type; then perform geometric correction: using the georegistration toolbar, select multiple evenly distributed ground feature points as control points, apply a first-order polynomial transformation, and ensure that the root mean square error is controlled within 1 pixel.

[0022] Radiometric correction was performed using a grid calculator, followed by local adaptive histogram equalization. For areas with significant noise, appropriate adjustments were made. Filtering the window's focus statistics by averaging;

[0023] To stitch together strip images, first select multiple corresponding control points at the overlap of the two strips, then use the mosaic tool, set the original resolution output, select the blend mosaic method, and enable the color balance function.

[0024] When extracting using a mask, it is necessary to ensure that the vector boundary of the study area completely matches the output range, while maintaining the original resolution; based on the range of the study area, extract the image data of the target area.

[0025] As a preferred option, the specific implementation process in S12 is as follows:

[0026] First, the original GF-2 satellite imagery data was acquired and cropped to... The standard image size for pixels;

[0027] Then, multiple bands in the GF-2 satellite imagery data are averaged to generate single-channel data with a style similar to the historical panchromatic imagery data of the Keyhole satellite.

[0028] Finally, data augmentation is performed, including random horizontal and vertical flipping.

[0029] As a preferred option, the specific implementation process in S2 is as follows:

[0030] S21: Preprocess the FBP dataset;

[0031] S22: Input the preprocessed data into the general semantic segmentation model for geographic feature extraction;

[0032] S23: Output the data, calculate the loss function, and update the model parameters.

[0033] As a preferred option, the specific preprocessing steps for the FBP dataset in S21 are as follows:

[0034] Data category merging: Merge multiple categories in the FBP dataset into categories including background, building areas, roads, vegetation, and water bodies;

[0035] Mean averaging: The multispectral bands of the FBP dataset are averaged and then converted into a single-channel panchromatic image;

[0036] Image cropping: Cropping a single-channel panchromatic image;

[0037] Data augmentation includes random horizontal flipping, vertical flipping, random scaling, random brightness adjustment, and injection of minor noise.

[0038] As a preferred option, the specific implementation details in S22 are as follows:

[0039] Segformer was chosen as the basic semantic segmentation model, which includes feature encoding and feature decoding. Input data, keyhole satellite imagery data, and discriminator outputs were fed into feature encoding. A multi-head self-attention mechanism was used to establish long-range dependencies between pixels, capture global contextual information, and form multi-level feature representations. The hidden representations output from feature encoding were fed into feature decoding for multi-scale feature fusion. A multilayer perceptron classification head generated a pixel-by-pixel class probability distribution map, which was finally transformed into ground feature extraction results with clear semantic annotations through argmax operation.

[0040] As a preferred approach, an adversarial training strategy is adopted, in which the hidden representations of the input data and keyhole satellite imagery data are input into the discriminator, and feature compression is performed based on a multi-layer MLP network to minimize the discrimination loss.

[0041] Preferably, in S23, the loss function is calculated based on the output data, specifically including:

[0042] Semantic segmentation loss: Combining cross-entropy loss and Focal Loss loss functions to optimize the model's pixel-wise classification accuracy and class balance;

[0043] Cross-entropy loss is used to optimize the model's pixel-wise classification accuracy. By calculating the difference between the predicted result and the true label, it drives the model to achieve higher classification accuracy at the pixel level. The specific calculation is as follows:

[0044]

[0045] in, This represents the cross-entropy loss value, where K represents the total number of categories; The actual label representing the pixel; This represents the probability distribution predicted by the model;

[0046] The specific formula for calculating the Focal Loss function is as follows:

[0047]

[0048] in, This represents the Focal Loss function. Indicates the probability of prediction. Represents the category balance factor. Indicates the modulation factor;

[0049] Combining the advantages of the two loss functions mentioned above, the final loss is obtained by summing them up, resulting in the overall segmentation loss function. for:

[0050] ,

[0051] Adversarial loss: A binary cross-entropy loss function is used. The discriminator improves its ability to identify the image source by minimizing the binary cross-entropy loss; the specific calculation formula is as follows:

[0052]

[0053] in, Indicates discriminator loss. Represents the binary cross-entropy function. This represents the discriminator model. This represents the source domain hidden features from GF-2 satellite imagery data. This represents the hidden features of the target domain derived from historical panchromatic imagery data from the Keyhole satellite. , These are label vectors with values ​​of 1 and 0, respectively.

[0054] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0055] (1) This invention can improve data utilization and analysis accuracy: Through effective data preprocessing and multi-source data fusion methods, it makes full use of the rich information in historical remote sensing data, solves problems such as inconsistent data formats and poor spatiotemporal consistency, and improves the availability of data and the accuracy of analysis results.

[0056] (2) This invention achieves comprehensive extraction of geographical features: it can not only obtain basic features such as changes in the area of ​​urban built-up areas, but also deeply explore the changes in urban form, spatial structure, functional zoning, etc., providing richer information and perspectives for a comprehensive understanding of the process of urban expansion and migration.

[0057] (3) The present invention improves the generalization ability of the model: Through adversarial training strategy and adaptive framework, the generalization ability and adaptability of the model on historical remote sensing images of different sensors and different resolutions are significantly improved, solving the problem of model transfer difficulty in the prior art.

[0058] Efficient Extraction and Mapping: By combining modern imagery with georeferencing, it can efficiently extract geographic elements and perform vectorized mapping, providing visualization support for historical geography research.

[0059] (4) This invention has broad application value: This invention can be widely applied in fields such as urban geography, historical geography, urban planning, and land resource management, providing efficient and accurate technical support for related applications and practical activities, helping to improve the scientificity and rationality of urban planning, and promoting the sustainable development of cities.

[0060] (5) The historical remote sensing image semantic segmentation method proposed in this invention has produced significant beneficial effects in many aspects through innovative technical solutions.

[0061] Firstly, regarding accuracy improvement, the adversarial training mechanism improved the model's mIoU by 7% on unlabeled historical images, and the segmentation accuracy of key categories such as building areas and roads by 6.7% and 9.6% respectively, effectively overcoming the problems of low resolution and noise interference in historical images. This was combined with Focal Loss (… The hybrid loss function, which combines cross-entropy loss with cross-entropy loss, significantly improves the segmentation effect of small targets, increases road IoU by 42%, and solves the problem of missed detection caused by fragmented ground features and blurred boundaries in historical images.

[0062] In terms of computational efficiency, the lightweight Segformer-B1 model (13.7M parameters) reduces the number of parameters by 93% compared to traditional large models, reduces training memory usage by 60%, increases inference speed by 3 times, and processes 512×512 images in less than 0.1 seconds, while also dynamically adjusting adversarial weights. The change from 0.05 to 0.15 shortened the training cycle by 20% and reduced hardware costs by 50%. This method demonstrates excellent generalization ability, with mIoU fluctuations of less than 2% across different eras (1970s-1980s) and different sensors (GF-2 to Keyhole satellite). It exhibits strong robustness to poor-quality images such as those with cloud cover and low contrast, maintains an IoU of over 55% for building segmentation, and reduces road breakage issues by 50%.

[0063] (6) In terms of practical application value, the technical solution of using modern image tags to drive historical image segmentation can save more than 90% of the annotation cost. The lightweight design supports deployment on edge devices and can still achieve real-time processing of 5 FPS in low computing power environments such as Jetson TX2, providing efficient and reliable technical support for fields such as urban history research and national defense reconnaissance. These technical effects directly stem from the innovative features of this application, such as the adversarial domain adaptation mechanism, hybrid loss function design, and lightweight model architecture. It is the first time that efficient and automated segmentation of keyhole satellite historical panchromatic images has been achieved, which has important academic value and engineering application significance. Attached Figure Description

[0064] Figure 1 This is a diagram of the training framework for panchromatic remote sensing image semantic segmentation of the present invention;

[0065] Figure 2 This is a roadmap for training the historical remote sensing image semantic segmentation model of the present invention;

[0066] Figure 3 This is a schematic diagram of the keyhole satellite image preprocessing of the present invention;

[0067] Figure 4 This is a diagram of the training data preprocessing framework of the present invention;

[0068] Figure 5 This is a flowchart of the adaptive training process of the Segformer model of the present invention;

[0069] Figure 6 The following are sample images of the FBP dataset verification results of this invention, where (a) is sparse mountain roads, (b) is dense urban areas, (c) is sparse rural areas, (d) is complex roads and industrial areas, and (e) is complex roads and dense urban areas. Detailed Implementation

[0070] The present invention will be further illustrated below with reference to specific embodiments. These embodiments are implemented based on the technical solutions of the present invention, and it should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0071] The core of this invention lies in addressing the difficulty of extracting key geographic features (such as urban building areas and road networks) from historical panchromatic imagery of Keyhole satellites. It proposes an adaptive framework based on deep learning and semantic segmentation, combined with modern imagery for georegistration. This embodiment provides a method for extracting built-up areas from historical remote sensing imagery based on a semantic segmentation model. Figures 1-5 As shown, the specific implementation steps are as follows:

[0072] S1: Acquire multi-source data and preprocess it;

[0073] The study area selected four ancient capitals, each representing the unique characteristics of cities in different regions of China: Beijing represents cities in northern China, Nanjing in eastern China, Xi'an in western China, and Luoyang in central China. These four cities each possess distinct characteristics, encompassing diverse geographical, cultural, and historical backgrounds, thus fully reflecting the model's applicability and robustness across different regional cities.

[0074] The study area in Nanjing was selected from Gulou, Xuanwu, Qinhuai and Jianye districts of Nanjing in the late 1970s. This area embodies the typical spatial characteristics of Nanjing as a historical and cultural city, can accurately capture the core spatial structure of the historical city, and can avoid data noise and computational redundancy caused by low-density built-up areas.

[0075] The research area for Beijing is the area within the Fifth Ring Road in 1973. This area not only encompasses the core urban area of ​​Beijing but also represents the main pattern of urban development during this period. Choosing this area for research helps to analyze the spatial layout, transportation network, population distribution, and economic development of major cities in northern China at that time, thus providing a historical perspective and theoretical support for a better understanding of contemporary Beijing's urban transformation.

[0076] The study area in Xi'an encompasses historical images of Lianhu, Xincheng, Beilin, and Yanta districts in 1974. These areas represent the main urban areas of Xi'an at that time, reflecting the city's development in the early 1970s, particularly changes in transportation, architectural style, and population distribution.

[0077] The study area in Luoyang selected historical images of Jianxi, Xigong, Laocheng and Chanhe districts in 1978. These areas were also the core urban areas of Luoyang at that time, reflecting the landscape and development characteristics of small and medium-sized cities in central China in the late 1970s.

[0078] S11. Obtain historical panchromatic image data from the Keyhole satellite, perform strip image stitching and geometric registration, georegister remote sensing images, overlay vector layers, and extract the global image by mask.

[0079] like Figure 3 As shown, processing keyhole satellite imagery in ArcGIS Pro requires a systematic preprocessing workflow, as detailed below:

[0080] When creating the mosaic dataset and importing the original scanned images, special attention must be paid to setting the correct coordinate system (such as WGS 1984 Web Mercator) and pixel type as 16-bit unsigned integers. During the geometric correction process, use the georegistration toolbar, select at least nine evenly distributed feature points as control points, apply a first-order polynomial (affine) transformation, and ensure that the RMS error is controlled within one pixel. If necessary, orthorectification can be performed in conjunction with DEM data.

[0081] During the image enhancement stage, it is recommended to first perform radiometric correction using a raster calculator, and then run a local adaptive histogram equalization tool. For areas with significant noise, additional methods can be used. The window uses a focus statistical average filtering method, and Fourier transform filtering can be enabled in special cases. This enhances the image and improves its quality.

[0082] For stitching between strip images, it is recommended to use the "Mattling to New Grid" tool. First, select 6 or more corresponding control points at the overlapping area of ​​the two strips, then use the mosaic tool, set the original resolution output, select the blend mosaic method, and enable the color balance function.

[0083] When extracting data using a mask, it is crucial to ensure that the vector boundary of the study area perfectly matches the output range, maintaining the original resolution. Based on the study area's extent, extract image data for the target region. During geometric correction, the rotation angle must be controlled within 5°, and bilinear interpolation resampling should be prioritized. Final quality control ensures that feature point distance errors and edge-matching errors are both less than one pixel, the histogram distribution is reasonable, and the proportion of data-free areas is controllable. This process fully leverages the advantages of ArcGIS Pro tools, improving work efficiency while ensuring processing quality, making it particularly suitable for the systematic preprocessing of historical remote sensing imagery.

[0084] Historical satellite imagery refers to KH series Keyhole satellites, specifically high-resolution optical reconnaissance satellites. The influence band information for the study area is shown in Table 1 below.

[0085] Table 1. Image strip information of the study area

[0086]

[0087] S12. Acquire the original GF-2 satellite image data and crop it to a standard 512×512 pixel map size to match the input requirements of the convolutional neural network, reduce the computational burden on the hardware, and improve data processing efficiency.

[0088] The basic training data for remote sensing semantic segmentation comes from the Five-Billion Pixels (FBP) benchmark dataset released in 2023: Tong, X.-Y., Xia, G.-S., Zhu, XX (2023). Enabling country-scale land cover mapping with meter-resolution satellite imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 196, 178-196 (Tong Xinyi, Xia Guisong, Zhu Xiaoxiang (2023). Enabling country-scale land cover mapping with meter-resolution satellite imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 196, 178-196).

[0089] This dataset, acquired via multispectral imagery from the Gaofen-2 (GF-2) satellite, includes four typical spectral bands: red, green, blue, and near-infrared, achieving a spatial resolution of approximately 4 meters. GF-2 is China's first civilian optical remote sensing satellite to break the 1-meter spatial resolution barrier. Equipped with two high-resolution cameras, it boasts panchromatic and multispectral resolutions of 1 meter and 4 meters respectively, while also providing a wide swath coverage of 45 kilometers. Its sub-meter spatial resolution and efficient observation capabilities have made it crucial for applications in urban planning, land resource surveys, environmental monitoring, and agricultural assessment.

[0090] To ensure that the model training data is consistent with the features of historical panchromatic images of KH, the multispectral bands of Gaofen-2 images in the FBP dataset were averaged and converted into single-channel panchromatic images.

[0091] The specific operation involves averaging multiple bands from the GF-2 imagery to generate single-channel data with a style similar to that of the KH panchromatic imagery (historical panchromatic imagery data from the Keyhole satellite). This method not only effectively simulates the single-band characteristics of the KH panchromatic imagery but also reduces domain differences between data, helping to improve the model's adaptability and segmentation accuracy on KH panchromatic imagery. This preprocessing step ensures a high degree of consistency between the training data and the target data in terms of spectral characteristics, laying a solid foundation for subsequent tasks.

[0092] Several common data augmentation methods were employed to enhance the model's robustness to geographic features. Specifically, these included random horizontal and vertical flipping to improve the model's ability to identify geographic features from different directions; random scaling to adapt the model to changes in geographic features at different scales; and random brightness and saturation, along with random small noise, to help the model adapt to image brightness differences and noise issues caused by the instability of early KH Keyhole satellite imaging, thereby increasing the model's generalization performance. These augmentation operations not only effectively enriched the diversity of the training data but also significantly improved the model's generalization ability in complex scenes, thus better addressing differences in direction, scale, and perspective in KH panchromatic historical imagery. Through these data augmentation techniques, this application further optimized the model's performance, providing more reliable technical support for accurately extracting building areas and roads from historical imagery.

[0093] S2: Model training is performed based on preprocessed multi-source data and semantic segmentation models;

[0094] S21: Preprocess the FBP dataset;

[0095] The FBP dataset contains 150 GF-2 satellite images used in the study. See the Supplementary Sources of GF-2 Satellite Imagery for data sources.

[0096] like Figure 4 As shown, data category merging: The 25 categories of the FBP dataset are merged into 6 categories: background, building area, road, vegetation, water body and others, which reduces the training difficulty and reduces the error caused by the ambiguity of category division.

[0097] Mean averaging: The multispectral bands of the FBP dataset are averaged and converted into single-channel panchromatic images to ensure that the training data is consistent with the spectral characteristics of the Keyhole satellite imagery.

[0098] Image cropping: The single-channel panchromatic image processed above is cropped.

[0099] Data augmentation: Various data augmentation methods are employed, such as random horizontal flipping, vertical flipping, random scaling, random brightness adjustment, and injection of subtle noise, to improve the model's robustness to geographic features.

[0100] like Figure 2 As shown, the source of the public remote sensing dataset is the FBP dataset. First, the GF-2 satellite image data (multispectral image) is averaged and converted into panchromatic single-channel data. The processed data is used as the training dataset for panchromatic remote sensing images to simulate the single-channel panchromatic characteristics of the Keyhole satellite.

[0101] S22: Input the preprocessed data into the general semantic segmentation model for geographic feature extraction;

[0102] Base model selection: Segformer was selected as the base semantic segmentation model and trained in conjunction with the adaptive framework proposed in this embodiment.

[0103] like Figure 5 As shown, model training is performed;

[0104] 1. Encoder and Decoder: The GF-2 image is input into the Segformer encoder, which establishes long-range dependencies between pixels through a multi-head self-attention mechanism, captures global context information, and forms multi-level feature representations.

[0105] First, feature preprocessing is performed on the input GF-2 satellite image data feature map. (H and W represent the height and width of the feature map, and C is the number of channels in the input feature map.) Flattened into a sequence:

[0106]

[0107]

[0108] in, This indicates the input raw 3D feature map. The two-dimensional sequence obtained after flattening; In this context, N represents the length of the sequence, which is the total number of pixels in the image, and C is the number of channels in the input feature map.

[0109] The input features are then transformed into a query matrix (Q), a key matrix (K), and a value matrix (V) using three sets of learnable projection weight matrices; the calculation formula for linear projection (separate processing) is as follows:

[0110]

[0111]

[0112]

[0113] in, , , These are the query (Q), key (K), and value (V) matrices for the h-th head, respectively. , , The projected weight matrix is ​​a learnable matrix. , , ; For each dimension of the attention head, d h =C / He (C represents the number of channels in the input feature map, and He is the number of attention heads).

[0114] When calculating attention weights, on the one hand, scaled dot product attention is calculated for each head h:

[0115]

[0116] in, Let h be the query matrix for the h-th head; Let be the key matrix of the h-th head; T represents the matrix transpose operation; Dimensions for each attention head.

[0117] On the other hand, a relative position encoding matrix M is also introduced, the elements of which are calculated as follows:

[0118]

[0119] in, This represents the bias resulting from the relative positional relationship between pixels i and j; ReLU represents the activation function, used here to introduce non-linearity into the relative positional bias and ensure that the bias value is non-negative. w is a learnable parameter. T represents the transpose operation; Encode the two-dimensional coordinates of pixel i (embedded representation of x and y coordinates). Represents the two-dimensional coordinate encoding of pixel j; This represents the encoding vector p that represents the position of pixel i. i and the encoding vector representing the position of pixel j Simply connect them together to form a longer joint vector that contains absolute information about both positions.

[0120] Calculated by each attention head It will be spliced ​​along the feature dimension to form a multi-head attention output. (in, The output of the He-th attention head is represented; N represents the sequence length, i.e., the total number of pixels in the image; C represents the number of channels in the input feature map; He is the number of attention heads, which is then processed by the output projection matrix W. O The linear transformation yields the final output Z = Concat(·)W O The output is ultimately reconstructed back to the original spatial dimensions as the encoder's output features. Here, Z represents the spatial features of the encoder's final output; Concat represents concatenating the outputs of all attention heads along the channel dimension. Indicates the output projection matrix. C represents the number of channels in the input feature map.

[0121] Features are fused at multiple scales through a decoder, and a pixel-by-pixel class probability distribution map is generated by a multilayer perceptron (MLP) classification head. Finally, the argmax operation is used to transform the ground feature extraction results with clear semantic annotations.

[0122] Specifically, the four levels of features output by the encoder (corresponding to downsampling rates of 1 / 4, 1 / 8, 1 / 16, and 1 / 32) are first upsampled to a uniform 1 / 4 input size using bilinear interpolation, as follows:

[0123]

[0124] in, This represents the original feature map output by the i-th level encoder (i=1,2,3,4). express Feature map upsampled to a uniform size. bilinear It is a bilinear interpolation upsampling function, with an output size equal to the input size. times. This represents the spatial downsampling rate of the i-th level feature map relative to the input image.

[0125] Then use Convolution adjusts the number of feature channels at each level to a uniform 256 dimensions, specifically:

[0126]

[0127] in, Indicates the process After convolution, the i-th level feature map has a unified number of channels of 256. for The weight tensor used in convolution on the i-th level feature. express Convolution operation; express Feature maps upsampled to a uniform size; and These represent the spatial height and width of the decoder at 1 / 4 input resolution, respectively.

[0128] These features, after spatial alignment and channel unification, are initially fused through element-by-element addition, and then... Depthwise separable convolutions are used for feature enhancement, specifically:

[0129]

[0130]

[0131] in, This represents the feature map after depthwise separable convolution enhancement. express Depthwise separable convolution, Its weight parameters. It is a preliminary fusion feature obtained by adding the four levels of features element by element; Indicates the process After convolution, the i-th level feature map has a unified number of channels of 256.

[0132] The fused features are fed into a multilayer perceptron classification head for processing. First, the feature dimension is expanded to 1024 dimensions (expansion ratio of 4) through a linear layer, then compressed back to 256 dimensions after applying the GELU activation function. The training process is stabilized through layer normalization and residual connections. Specifically:

[0133]

[0134]

[0135]

[0136]

[0137]

[0138] Where Z represents the pair of pairs ... Normalized features after layer normalization; Presentation layer normalization operation. This represents the feature map after depthwise separable convolution enhancement, which is used as the input to the MLP head; This indicates that the first linear layer (expansion ratio 4) increases the output of the channels from 256 to 1024. and This indicates the spatial height and width of the decoder at 1 / 4 of the input resolution; Indicates The result after activation, and Same shape; Represents a non-linear activation function, applied to Obtain a tensor of the same shape; This means that the second linear layer compresses the channel back to 256, completing the bottleneck structure of "expansion then contraction"; express The linear mapping implemented by convolution is called a fully connected layer. This represents the final decoded feature output by the MLP classification head, with a dimension of 256.

[0139] The final classification layer projects features onto the dimension of the number of categories (such as buildings, roads, vegetation, etc.), and generates the category probability distribution for each pixel using the Softmax function, specifically:

[0140]

[0141] Where P is the class probability distribution map for each pixel, with a size of ; and This represents the spatial height and width of the decoder at 1 / 4 of the input resolution; C represents the number of channels in the input feature map. express The linear mapping implemented by convolution is called a fully connected layer. This represents the final decoded features output by the MLP classification header; It is the weight tensor of the final classification layer, which maps the 256-dimensional features to the total number of categories in the semantic segmentation task (which is the number of land cover categories when the background is excluded).

[0142] To obtain the prediction results at the original resolution, the classification output is upsampled by 4 times, and the final class label for each pixel is determined by the Argmax operation, specifically:

[0143]

[0144] in, The final output semantic label map (each pixel value is...) Here, C1 represents the total number of categories in the segmentation task. P is the category probability distribution map for each pixel. This means that at pixel (h,w), we determine which class of c has the highest probability and use the number of that class as the final label for that pixel. This refers to upsampling P by a factor of 4 to restore it to the original input image resolution. c represents the traversal category; h represents the traversal row number; and w represents the traversal column number.

[0145] Throughout the process, the hierarchical fusion of multi-scale features enables the model to simultaneously capture local details and global contextual information, while the lightweight MLP design significantly reduces computational overhead while maintaining accuracy, making it particularly suitable for large-scale semantic segmentation tasks of high-resolution remote sensing images. This design, while maintaining the powerful representational capabilities of the Transformer architecture, achieves a good balance between computational efficiency and segmentation accuracy through optimization techniques such as spatial alignment, channel unification, and depthwise separable convolution.

[0146] 2. Adversarial Training: An adversarial training strategy is adopted to enhance the model's cross-domain adaptability. The input data (preprocessed GF-2 satellite imagery data) and the hidden representations of Keyhole satellite imagery are fed into a multi-layer MLP network for feature compression. The discriminator minimizes the discrimination loss by enhancing its discriminative ability, and the encoder learns the image style by reducing feature differences, ultimately achieving a sensor-independent general feature representation.

[0147] S23: Output data, calculate the loss function, and update the model parameters;

[0148] The segmentation result is output based on feature decoding, and the loss function is calculated based on the output data, specifically including:

[0149] Semantic segmentation loss: Combining cross-entropy loss and focal loss functions to optimize the model's pixel-wise classification accuracy and class balance.

[0150] Cross-entropy loss is used to optimize the model's pixel-by-pixel classification accuracy. By calculating the difference between the predicted result and the true label, it drives the model to achieve higher classification accuracy at the pixel level. This loss function is particularly suitable for handling complex backgrounds and multi-class land cover distributions in remote sensing imagery, effectively improving the model's ability to identify various land cover types. The specific calculation is as follows:

[0151]

[0152] in, This represents the cross-entropy loss value, where K is the total number of categories; The actual label representing the pixel; It is the probability distribution predicted by the model.

[0153] The Focal Loss function was originally proposed to address the class imbalance problem, especially in tasks such as remote sensing image processing. Often, certain classes (e.g., background, large objects) dominate the dataset, while other smaller classes (e.g., roads, small buildings) have far too few pixels or samples. This class imbalance leads to excessive bias in the model during training, affecting its ability to recognize smaller objects. The model may ignore these smaller targets and focus more on the larger, more prevalent ground features. To improve the model's ability to extract these smaller targets (especially important features like roads), Focal Loss effectively solves this problem by introducing a mechanism that prioritizes hard-to-classify samples. The specific calculation formula is as follows:

[0154]

[0155] in, It is the focal loss value. It is the predicted probability, the predicted probability for the correct class. It is a category balancing factor, typically used to adjust the importance of different categories and avoid bias towards certain categories. It is a modulation factor, usually with a value greater than 0, which controls the degree of penalty for samples classified as easy or difficult; when Focal Loss degenerates into cross-entropy loss. This formula indicates that in Focal Loss, the loss for easily classified samples is attenuated, while more attention is paid to samples that are difficult to classify, especially for categories with a small percentage of pixels (such as roads and small buildings).

[0156] Combining the advantages of the two loss functions mentioned above, the final loss is obtained by summing them up, resulting in the overall segmentation loss function. for:

[0157]

[0158] in, It is the focal loss value. This represents the cross-entropy loss value.

[0159] Adversarial loss: The binary cross-entropy (BCE) loss function is adopted. The discriminator improves its ability to distinguish the source of the image by minimizing the BCE loss, and the encoder reduces the feature difference by minimizing the adversarial loss.

[0160] The specific formula for calculating the discriminator's adversarial loss is as follows:

[0161]

[0162] in, This indicates that the discriminator is susceptible to loss. Represents the binary cross-entropy function. This represents the discriminator model. This represents the source domain hidden features from GF-2 satellite imagery data. This represents the hidden features of the target domain derived from historical panchromatic imagery data from the Keyhole satellite. , These are label vectors with values ​​of 1 and 0, representing their respective actual origins. It is the source domain label vector of the GF-2 satellite image data. It is the ideal label for the target domain corresponding to the keyhole data.

[0163] In adversarial training, the optimization objective of the Segformer feature encoder module is to mislead the discriminator, causing the target domain to also be judged as 1, thus achieving domain adaptation. The adversarial loss function of the feature encoder is defined as:

[0164]

[0165] in, This represents the adversarial loss of the feature encoder. This represents the binary cross-entropy loss function. Describes the discriminator model, f t This represents the hidden features of the target domain derived from historical panchromatic imagery data from the Keyhole satellite. It is the source domain label vector of the GF-2 satellite image data.

[0166] As can be seen, the optimization objective of the Segformer feature encoding module is exactly the opposite of that of the discriminator. By minimizing this loss function, the feature differences between GF-2 satellite image data and Keyhole satellite historical panchromatic image data are continuously reduced to facilitate feature extraction by the subsequent decoder.

[0167] S3: After training, input the historical remote sensing images into the panchromatic image semantic segmentation model and output the image segmentation results;

[0168] Run the panchromatic image semantic segmentation model to output the model segmentation probability map, and the historical building area data.

[0169] In this embodiment, the calculations for training settings and evaluation are as follows:

[0170] 1. Training framework: PyTorch framework is used, AdamW is selected as the optimizer, the initial learning rate is 2e-4, training is performed for 40 epochs, and the learning rate is adjusted to 2e-5 on the 30th epoch.

[0171] 2. Evaluation metrics: The most mainstream evaluation metric for semantic segmentation tasks is the mean intersection-union ratio (mIoU) to measure the segmentation performance of the model. At the same time, IoU values ​​for individual classes are provided to evaluate the segmentation accuracy of the model on specific classes.

[0172] mIoU calculates the degree of overlap between the predicted and ground truth regions and averages the IoU across all categories, providing an intuitive and standardized evaluation of a model's performance in the overall segmentation task. This metric not only reflects the model's comprehensive ability to identify various land features but also provides a unified benchmark for comparing the performance of different models. The formula for calculating mIoU is as follows:

[0173]

[0174] Where C1 represents the total number of all categories in the segmentation task (the total number of land cover categories when the background is not included; if the background is included, then C1+1). This represents the number of pixels correctly predicted as category c, i.e., the total number of pixels predicted as c and whose true label is also c. This represents the number of pixels that were incorrectly predicted as belonging to category c, i.e., the total number of pixels that were predicted as c but whose true label was not c. This represents the number of pixels that were not predicted for category c, i.e., the total number of pixels whose true label is c but were not predicted as c.

[0175] Figure 6 This paper presents several representative semantic segmentation results of the model on the FBP validation set. Each set of results consists of the following parts: the original remote sensing image, the panchromatic image input to the model, the ground truth labels, and a comparison of the model's predicted output with and without adaptive frame transfer. The comparison shows that, with a single-channel optical panchromatic image as input, the model's segmentation accuracy is significantly improved after introducing the adaptive frame transfer strategy. This improvement is particularly evident in the recognition of small-sized ground features such as roads. The model after transfer learning effectively alleviates the challenge caused by the lack of spectral information in panchromatic images and enhances the extraction and recognition of image detail features by transferring and utilizing pre-trained knowledge from the source domain.

[0176] In this embodiment, the extracted geographic element results are applied to fields such as historical geography research, urban planning, and land resource management, providing precise technical support for related applications and practical activities, such as creating historical maps of ancient city areas based on remote sensing images.

[0177] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model, characterized in that: By employing an adaptive framework based on deep learning and semantic segmentation, combined with modern imagery for georegistration, and utilizing an adversarial domain adaptation mechanism, hybrid loss function design, and lightweight model architecture, efficient and automated segmentation of historical panchromatic images from Keyhole satellites is achieved, extracting core geographic features. The specific steps are as follows: S1: Acquire multi-source data and preprocess it; S2: Model training is performed based on preprocessed multi-source data and a general semantic segmentation model. The specific implementation steps are as follows: S21: Preprocess the FBP dataset, specifically as follows: Data category merging: Merge multiple categories in the FBP dataset into categories including background, building areas, roads, vegetation, and water bodies; Mean averaging: The multispectral bands of the FBP dataset are averaged and then converted into a single-channel panchromatic image; Image cropping: Cropping a single-channel panchromatic image; Data augmentation includes random horizontal flipping, vertical flipping, random scaling, random brightness adjustment, and injection of minor noise. S22: Input the preprocessed data into the general semantic segmentation model for geographic feature extraction; Segformer was chosen as the basic semantic segmentation model. The model includes feature encoding and feature decoding. Input data, keyhole satellite image data and discriminator output results are input into the feature encoding. A multi-head self-attention mechanism is used to establish long-range dependencies between pixels, capture global context information, and form multi-level feature representations. The hidden representation output from the feature encoding is input into the feature decoding for multi-scale feature fusion. A pixel-by-pixel classifier head generates a class probability distribution map, which is then transformed into a ground feature extraction result with clear semantic annotations through argmax operation. An adversarial training strategy is adopted, incorporating the hidden representations of the input data and keyhole satellite imagery into the discriminator. Feature compression is performed based on a multi-layer MLP network to minimize the discrimination loss. S23: Output data, calculate the loss function, and update the model parameters; The loss function is calculated based on the output data, specifically including: Semantic segmentation loss: Combining cross-entropy loss and focus loss to optimize the model's pixel-wise classification accuracy and class balance; Cross-entropy loss is used to optimize the model's pixel-wise classification accuracy. By calculating the difference between the predicted result and the true label, it drives the model to achieve higher classification accuracy at the pixel level. The specific calculation is as follows: =－ ， in, This represents the cross-entropy loss value, where K represents the total number of categories; The actual label representing the pixel; This represents the probability distribution predicted by the model; The specific formula for calculating focus loss is as follows: =－ ， in, Indicates the focal loss value. Indicates the probability of prediction. γ represents the category balance factor, and γ represents the modulation factor; Combining the advantages of the two loss functions mentioned above, the final loss is obtained by summing them up, resulting in the overall segmentation loss function. for: ， Adversarial loss: A binary cross-entropy loss function is used. The discriminator improves its ability to identify the image source by minimizing the binary cross-entropy loss; the specific calculation formula is as follows: ， in, Let BCE(·,·) represent the discriminator loss, BCE(·,·) represent the binary cross-entropy function, and D(·) represent the discriminator model. This represents the source domain hidden features from GF-2 satellite imagery data. This represents the hidden features of the target domain derived from historical panchromatic imagery data from the Keyhole satellite. This indicates that the numerical value is the source domain label vector of the GF-2 satellite image data. This represents the ideal label for the target domain corresponding to the keyhole data; S3: After training, input the historical remote sensing images into the panchromatic image semantic segmentation model and output the image segmentation results.

2. The method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model according to claim 1, characterized in that: In S1, the specific content is as follows: S11: Acquire historical panchromatic image data from the Keyhole satellite, perform strip image mosaicking and geometric registration, georegister remote sensing images, overlay vector layers, and extract the global image by masking. S12: Acquire raw GF-2 satellite imagery data and perform cropping, mean averaging, and data augmentation.

3. The method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model according to claim 2, characterized in that: In S11, the specific implementation process is as follows: Preprocessing of historical panchromatic imagery data from Keyhole satellite in ArcGIS Pro environment: First, create a mosaic dataset and import the original scanned image, while setting the correct coordinate system and pixel type; then perform geometric correction: using the georegistration toolbar, select multiple evenly distributed ground feature points as control points, apply a first-order polynomial transformation, and ensure that the root mean square error is controlled within 1 pixel. Radiometric correction was performed using a grid calculator, followed by the local adaptive histogram equalization tool. For areas with significant noise, a 3×3 window focal statistical average filter was applied. To stitch together strip images, first select multiple corresponding control points at the overlap of the two strips, then use the mosaic tool, set the original resolution output, select the blend mosaic method, and enable the color balance function. When extracting using a mask, it is necessary to ensure that the vector boundary of the study area completely matches the output range, while maintaining the original resolution; based on the range of the study area, extract the image data of the target area.

4. The method for extracting built-up areas from historical remote sensing images based on a semantic segmentation model according to claim 2, characterized in that: In S12, the specific implementation process is as follows: First, the original GF-2 satellite imagery data was acquired and cropped to a standard 512×512 pixel map sheet. Then, multiple bands in the GF-2 satellite imagery data are averaged to generate single-channel data with a style similar to the historical panchromatic imagery data of the Keyhole satellite. Finally, data augmentation is performed, including random horizontal and vertical flipping.

Images