An urban ground feature extraction method and device

Abstract

The present application belongs to the field of geographic information system and artificial intelligence technology, and relates to a kind of urban ground feature extraction method and device;Remote sensing image is input into SAM encoder, and output multi-scale feature map, multi-scale feature map is input into RPN network for target detection to generate target boundary box;The target boundary box is expanded, and the foreground point prompt and the background point prompt are obtained based on the expansion target boundary box obtained by expansion;The expansion target boundary box and its foreground point prompt and background point prompt are input into prompt generator, and the prompt token is output, and the expansion target boundary box and its prompt token are input into SAM decoder, and the ground mask graph, vertex confidence graph, vertex offset and boundary mask graph are output;Using mask guide vertex connection algorithm, based on the ground mask graph of expansion target boundary box, vertex confidence graph, vertex offset and boundary mask graph, the ground vector contour is obtained.

Description

Technical Field

[0001] This invention relates to the fields of geographic information systems and artificial intelligence technology, and in particular to a method and apparatus for extracting urban features. Background Technology

[0002] Buildings, bridges, and other urban artificial features are the carriers of urban spatial structure and human activities. Their spatial distribution and morphological characteristics directly reflect the functional zoning of the city and the level of construction of building facilities. Therefore, developing an automated method and system that can stably and accurately extract the vector contours of urban features such as buildings and bridges under different regions and spatial resolutions is a key problem that urgently needs to be solved in the field of remote sensing urban analysis.

[0003] Compared to traditional extraction methods that rely on manual feature design, deep learning methods, with their powerful feature representation capabilities, have gradually become the mainstream technology for remote sensing urban feature extraction. Among them, Vision Transformer (ViT), leveraging its self-attention mechanism's advantages in global context modeling, demonstrates strong adaptability in complex urban scenarios. With the development of the ViT architecture, a series of vision-based models based on ViT have been proposed, providing new technical approaches for remote sensing feature extraction. Among these, the Segment Anything Model (SAM) achieves strong general visual representation capabilities through large-scale pre-training and introduces a cueing mechanism, enabling flexible segmentation methods.

[0004] However, in the task of automatically extracting vector contours of large-scale urban features such as buildings and bridges, current general segmentation methods based on SAM still have significant limitations and shortcomings in practical engineering applications. On the one hand, SAM models heavily rely on manual interaction prompts for target localization and segmentation. In complex urban scenes with large-scale, high-resolution, and high-density distribution, manual pointing and bounding box prompts are required for each target, making the process cumbersome and lacking in automation. This makes it difficult to support efficient batch processing of large-scale urban features, greatly limiting the model's application efficiency in engineering scenarios such as surveying and remote sensing interpretation. On the other hand, SAM models primarily aim to output pixel-level dense masks. Their native structure does not have the ability to directly generate regularized, high-precision vector contours. Relying solely on probabilistic masks makes it difficult to accurately depict the boundary structures, corner features, and contour directions of features such as buildings and bridges with strong geometric constraints. This fails to directly meet the requirements of geographic information rendering, 3D modeling, and smart city construction for the accuracy, standardization, and practicality of vector contours. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problems of existing methods for extracting ground features, which require manual annotation of prompt boxes or prompt points to achieve ground feature target location and segmentation, resulting in a cumbersome process and low extraction efficiency. At the same time, these methods can only output probability masks and cannot directly generate accurate ground feature vector contours.

[0006] To address the aforementioned technical problems, this invention provides a method for extracting urban feature elements, comprising:

[0007] Obtain a trained feature extraction model, which includes a SAM encoder, an RPN network, a cue generator, and a SAM decoder;

[0008] The remote sensing image is input into the SAM encoder, which outputs a multi-scale feature map; the multi-scale feature map is then input into the RPN network to perform target detection using ground features as targets, generating target bounding boxes.

[0009] Each target bounding box is expanded to obtain an expanded target bounding box. The center point of the expanded target bounding box is used as the foreground point cue. The background point cue of the expanded target bounding box is obtained based on the center points of the expanded regions along the positive y-axis, negative y-axis, positive x-axis, and negative x-axis directions of the target bounding box.

[0010] Input each extended target bounding box and its foreground and background point cues into the cue generator, and output the cue token; input each extended target bounding box and its cue token into the SAM decoder, and output the feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box;

[0011] Using a mask-guided vertex connection algorithm, the vector contours of ground features in remote sensing images are obtained based on the ground feature mask map, vertex confidence map, vertex offset, and boundary mask map of each extended target bounding box.

[0012] Preferably, the remote sensing image is input into the SAM encoder, and the output is a multi-scale feature map, including:

[0013] The remote sensing image is input into the image block embedding layer and divided into blocks to obtain multiple non-overlapping image blocks;

[0014] Convolution and position encoding are performed on each non-overlapping image patch to obtain the feature vector and position embedding vector of each non-overlapping image patch.

[0015] The feature vectors and position embedding vectors of each non-overlapping image block are added together to obtain the position feature sequence of each non-overlapping image block. The position feature sequence of the non-overlapping image block is then input into the Transformer encoder.

[0016] The intermediate features output by the m-th Transformer Block in the Transformer encoder are reduced in dimensionality using a feature aggregator to obtain the m-th dimensionality-reduced intermediate feature. The m-th dimensionality-reduced intermediate feature is then superimposed with the (m-2)-th fused feature by adding the residuals, and the superimposed feature is enhanced to obtain the m-th fused feature. Channel recovery is then performed on the last fused feature to obtain the target fused feature. Here, m is an odd number and m>1, and the first fused feature is the feature after dimensionality reduction of the intermediate features output by the first Transformer Block.

[0017] The target fusion features are upsampled using transposed convolution in the feature decomposer to generate first and second mapping features with spatial scales magnified by one and two times, respectively; the target fusion features are downsampled using max pooling in the feature decomposer to generate third and fourth mapping features with spatial scales reduced by one and two times, respectively.

[0018] Based on target fusion features, first mapping features, second mapping features, third mapping features and fourth mapping features, multi-scale feature maps with different spatial resolutions but consistent channel dimensions are obtained.

[0019] Preferably, each extended target bounding box and its cue token are input into the SAM decoder, and the output includes a ground feature mask, vertex confidence map, vertex offset, and boundary mask for each extended target bounding box, including:

[0020] In the multi-scale feature map, perform region of interest alignment operation on the region where each extended target bounding box is located, and extract the extended region of interest features of each extended target bounding box;

[0021] The mask token of the extended region of interest features, the mask intersection-union score token, and the preset polygon token used to fit the shape of the ground feature vector outline of each extended target bounding box are concatenated with the cue token of the extended target bounding box to obtain the concatenated token sequence of each extended target bounding box.

[0022] The extended region of interest features of each target bounding box and their concatenated token sequence are input into the SAM decoder. The SAM decoder flattens the extended region of interest features and maps them to obtain a one-dimensional feature vector of each extended target bounding box.

[0023] The one-dimensional feature vectors of each extended target bounding box and their concatenated token sequences are input into the bidirectional Transformer module in the multi-task decoder to perform cross-attention calculation, and the updated one-dimensional feature vectors and concatenated token sequences are output.

[0024] Two-dimensional shape restoration is performed on the updated one-dimensional feature vectors of each extended target bounding box, and the restored two-dimensional feature vectors are upsampled to the target space resolution; the mask token, mask intersection-union score token, cue token and polygon token are separated from the stitched token sequence updated by each extended target bounding box.

[0025] The mask token, mask intersection-union score token, cue token, and polygon token of each separated extended target bounding box are multiplied by matrix multiplication with the upsampled two-dimensional feature vector of each extended target bounding box to obtain the ground feature mask, mask intersection-union score, vertex confidence map, vertex offset, and boundary mask of each target bounding box.

[0026] Preferably, a mask-guided vertex connection algorithm is used to obtain the vector contours of ground features in the remote sensing image based on the ground feature mask map, vertex confidence map, vertex offset, and boundary mask map of each extended target bounding box, including:

[0027] Local max pooling is performed on the vertex confidence map of each expanded target bounding box to obtain the pixel with the maximum confidence score in each local neighborhood. The pixels with a confidence score greater than a preset threshold among the pixels with the maximum confidence score are selected as candidate vertices of the expanded target bounding box.

[0028] The pixel-level coordinates of each candidate vertex of the expanded target bounding box are corrected based on the vertex offset of each expanded target bounding box, and the vertex set of the expanded target bounding box is obtained based on the corrected candidate vertices.

[0029] Based on the feature mask map of each extended target bounding box, extract the feature contour point sequence, and search in the feature contour point sequence for the contour point with the smallest distance to each vertex in the vertex set of the extended target bounding box to obtain the contour point corresponding to each vertex.

[0030] Connect each vertex in the vertex set of each extended target bounding box to its adjacent vertices to obtain the vertex vector of each vertex; connect the contour point corresponding to each vertex to the contour point corresponding to the adjacent vertices of that vertex to obtain the contour point vector of each vertex.

[0031] If the angle difference between the vertex vector and the contour point vector of a vertex in the expanded target bounding box is less than a preset angle difference, then the vertex is taken as a candidate vector contour point of the expanded target bounding box.

[0032] Based on all candidate vector contour points of each extended target bounding box, the feature vector contour of that extended target bounding box is obtained; based on the feature vector contours of all extended target bounding boxes, the feature vector contour of the remote sensing image is obtained.

[0033] Preferably, the training process of the feature extraction model includes:

[0034] S11: Based on the land cover features in all remote sensing image samples in the training set, calculate the proportion of remote sensing image samples containing each type of land cover feature to the total number of remote sensing image samples, and obtain the occurrence frequency of each type of land cover feature.

[0035] S12: Calculate the category repetition factor of each category of land cover based on the preset frequency threshold and the occurrence frequency of each category of land cover; calculate the repetition factor of each remote sensing image sample based on the land cover category contained in each remote sensing image sample and the category repetition factor of each category of land cover.

[0036] S13: Based on the repetition factor of each remote sensing image sample, sample the remote sensing image samples in the training set to obtain the target sample;

[0037] S14: Input the target sample into the SAM encoder, RPN network, cue generator and SAM decoder, output the land feature vector contour prediction result of the target sample, and then calculate the value of the land feature extraction loss function;

[0038] S15: Return to execute S13 to iteratively train the SAM encoder, RPN network, cue generator, and SAM decoder until the value of the feature extraction loss function converges, thus obtaining the trained SAM encoder, RPN network, cue generator, and SAM decoder, and thus obtaining the trained feature extraction model.

[0039] Preferably, during iterative training, multiple low-rank adaptation modules are set on the self-attention layer of the Transformer encoder of the SAM encoder, running parallel to the original linear mapping branch. The output of the original linear mapping branch is added to the output of the multiple low-rank adaptation modules as the output of the self-attention layer. In each iteration, the parameters of the original linear mapping branch are frozen, and only the parameters of the multiple low-rank adaptation modules are updated.

[0040] After iterative training, the parameters of multiple low-rank adaptation modules are fused with the parameters of the original linear mapping branch by matrix addition as the final weights of the self-attention layer linear mapping.

[0041] Preferably, the formula for calculating the category repetition factor for each category of land cover features is as follows:

[0042] ,

[0043] in, The category repetition factor represents the category of the land feature in category c; This indicates the frequency of occurrence of the c-th category of land feature; Indicates the preset frequency threshold;

[0044] The formula for calculating the repetition factor of each remote sensing image sample is:

[0045] ,

[0046] in, Represents remote sensing image samples The repeating factor; Represents remote sensing image samples It is a collection of all categories of geographic features.

[0047] Preferably, the calculation process of the feature extraction loss function includes:

[0048] The RPN loss is calculated based on the classification loss and regression loss of each target bounding box in the target sample output by the RPN network.

[0049] The sum of the regression loss and the feature vector contour prediction loss of each target bounding box is conditionally calculated using the indicator function to obtain the geometric constraint loss of each target bounding box after constraint.

[0050] The geometric constraint loss and classification loss for each target bounding box are summed to obtain the multi-task loss for each target bounding box;

[0051] The average of the multi-task loss of all target bounding boxes in the target sample is calculated, and the value of the feature extraction loss function is obtained based on the sum of the average value and the RPN loss.

[0052] Preferably, the loss function for extracting ground features Represented as:

[0053] ,

[0054] in, Indicates RPN loss; This indicates the number of bounding boxes of targets in a remotely sensed image sample. Let represent the classification loss for the i-th target bounding box; This represents the geometric constraint loss for the i-th target bounding box; Let represent the regression loss for the i-th target bounding box; The feature vector profile prediction loss represents the loss for the i-th target bounding box. This represents an indicator function used to distinguish between positive and negative samples. The i-th target bounding box is considered a positive sample when the intersection-union ratio (IoU) between it and the ground truth bounding box is greater than 0.5. When the intersection-union ratio (IoU) between the i-th target bounding box and the ground truth bounding box is less than or equal to 0.5, the i-th target bounding box is considered a negative sample. ;

[0055] The formula for calculating the loss in ground feature vector profile prediction is:

[0056] ,

[0057] in, The ground feature mask prediction loss represents the bounding box of the i-th target. The weights representing the prediction loss of the ground feature mask map; Let represent the vertex confidence graph prediction loss for the i-th target bounding box; The weights represent the prediction loss of the vertex confidence graph; This represents the vertex offset prediction loss for the i-th target bounding box; The weights represent the vertex offset prediction loss; The boundary mask prediction loss represents the bounding box prediction loss for the i-th target bounding box. The weights represent the prediction loss of the boundary mask image; The mask intersection-union score prediction loss represents the loss of the i-th target bounding box. This represents the weight of the mask intersection ratio score prediction loss.

[0058] The present invention also provides a device for extracting ground features, comprising:

[0059] The model acquisition module is used to acquire a trained ground feature extraction model, which includes a SAM encoder, an RPN network, a cue generator, and a SAM decoder.

[0060] The feature extraction and target detection module is used to input remote sensing images into the SAM encoder and output multi-scale feature maps; the multi-scale feature maps are then input into the RPN network to perform target detection using ground features as targets and generate target bounding boxes.

[0061] The prompt generation module is used to expand each target bounding box to obtain an expanded target bounding box, use the center point of the expanded target bounding box as the foreground point prompt, and obtain the background point prompt based on the center point of the expanded region along the positive y-axis direction, the center point of the expanded region along the negative y-axis direction, the center point of the expanded region along the positive x-axis direction, and the center point of the expanded region along the negative x-axis direction.

[0062] The mask generation module is used to input each extended target bounding box and its foreground and background point cues into the cue generator and output the cue token; it also inputs each extended target bounding box and its cue token into the SAM decoder and outputs the ground feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box.

[0063] The ground feature vector contour acquisition module is used to obtain the ground feature vector contour of the remote sensing image based on the ground feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box using a mask-guided vertex connection algorithm.

[0064] The urban feature extraction method provided in this application has the following beneficial effects:

[0065] 1. Using SAM as the base model, this application leverages its general segmentation and cross-scene generalization capabilities to provide a foundational model for the accurate extraction of urban features in complex remote sensing scenarios. Simultaneously, after extracting multi-scale feature maps, an RPN network is introduced to generate target bounding boxes for locating ground objects on each scale feature map. An automatic hybrid prompting mechanism is designed, generating foreground and background prompts based on the expansion of the target bounding boxes and the extraction of center points. This automatically generates prompt boxes and points without requiring manual interaction for each target, conforming to the prompting input method of SAM and reducing manual intervention, thus improving overall processing efficiency while maintaining segmentation accuracy. Furthermore, addressing the issue that existing SAM models can only output pixel-level mask results and cannot obtain fine vector contours, this application utilizes a mask-guided vertex connection algorithm to sort and connect unordered vertices under the geometric constraints of instance masks, thereby stably and accurately outputting ground object vector contour results.

[0066] 2. During the model training process, this application adopts a dynamic sampling strategy based on the image sample category frequency. By adjusting the participation frequency of different categories of samples during the training process, the minority category samples receive more attention during model training, thereby improving the model's learning ability for the features of the minority category of land cover. In this way, while maintaining stable learning of the majority of land cover categories, the feature learning and detection capabilities of the minority land cover categories can be effectively improved, thus improving the vector contour extraction performance of the model in class imbalance scenarios. Attached Figure Description

[0067] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0068] Figure 1 Flowchart of the urban feature extraction method provided for this application;

[0069] Figure 2 A schematic diagram of the SAM encoder framework based on efficient parameter fine-tuning provided in this application;

[0070] Figure 3 A schematic diagram of the lightweight multi-scale feature enhancement module provided in this application;

[0071] Figure 4This application provides a schematic diagram of RPN network for target bounding box detection;

[0072] Figure 5 A schematic diagram of the ROI-based automatic hybrid enhancement generation process provided for this application;

[0073] Figure 6 A schematic diagram of the multi-task decoder framework provided in this application;

[0074] Figure 7 A schematic diagram of the vector contour extraction framework for geographic features provided in this application;

[0075] Figure 8 A schematic diagram illustrating the principle of the efficient parameter fine-tuning strategy provided in this application;

[0076] Figure 9 Provide a CitySAM plugin system architecture diagram for this application;

[0077] Figure 10 A schematic diagram of the urban feature extraction framework provided for this application;

[0078] Figure 11 A schematic diagram illustrating the visualization of detection results on test data for the method provided in this application and existing methods; wherein, Figure 11 Image (a) shows the detection results of the FFL method on three remote sensing images. Figure 11 (b) in the figure is a schematic diagram of the detection results of the SAM_Adapter method on three remote sensing images. Figure 11 (c) in the figure is a schematic diagram of the detection results of the SAM_Prefix-Tuning method on three remote sensing images. Figure 11 (d) in the figure is a schematic diagram of the detection results of the SAM_LoRA (r=32) method on three remote sensing images. Figure 11 (e) in the figure is a schematic diagram of the detection results of the SAM_LoRA (r=96) method on three remote sensing images. Figure 11 (f) in the figure is a schematic diagram of the detection results of the method provided in this application on three remote sensing images. Figure 11 (g) in the figure is a schematic diagram of the detection results of the Ground Truth method on three remote sensing images. Detailed Implementation

[0079] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0080] Please see Figure 1 , Figure 1The diagram shown is a flowchart of the urban feature extraction method provided in this application, which specifically includes steps S10 to S50:

[0081] S10: Obtain the trained feature extraction model, which includes a SAM encoder, RPN network, cue generator, and SAM decoder.

[0082] S20: Input the remote sensing image into the SAM encoder and output a multi-scale feature map; input the multi-scale feature map into the RPN network to perform target detection using ground features as targets and generate target bounding boxes.

[0083] S30: Expand each target bounding box to obtain an expanded target bounding box. Use the center point of the expanded target bounding box as the foreground point cue. Obtain the background point cue of the expanded target bounding box based on the center points of the expanded regions along the positive y-axis, negative y-axis, positive x-axis, and negative x-axis directions.

[0084] S40: Input each extended target bounding box and its foreground and background point cues into the cue generator, and output the cue token; input each extended target bounding box and its cue token into the SAM decoder, and output the feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box.

[0085] S50: Using a mask-guided vertex connection algorithm, based on the ground feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box, the ground feature vector contour of the remote sensing image is obtained.

[0086] Specifically, this application introduces a highly efficient fine-tuning SAM encoder using a multi-branch low-rank adaptation fine-tuning strategy, such as... Figure 2 The diagram shows a schematic of the SAM encoder framework based on efficient parameter fine-tuning. The SAM encoder is based on the ViT architecture and consists of a patch embedding layer, a positional embedding layer, and a Transformer encoder. SAM currently offers three encoder-scale variants: ViT-B, ViT-L, and ViT-H. The difference between the three variants lies in the number of parameters in the main encoder. The Transformer encoder uses ViT-B / 16, which contains 12 Transformer Blocks and introduces a global attention mechanism in layers 3, 6, 9, and 12, while using a 14*14 window attention mechanism in other layers.

[0087] Furthermore, this application introduces a lightweight multi-scale enhancement module for multi-scale feature extraction. This module consists of a feature aggregator and a feature decomposer, such as... Figure 3 As shown, based on this, in step S20, the remote sensing image is input into the SAM encoder, and multi-scale feature maps are output, including S200~S205:

[0088] S200: The remote sensing image input image block embedding layer is divided into blocks to obtain multiple non-overlapping image blocks.

[0089] S201: Perform convolution and position encoding on each non-overlapping image block to obtain the feature vector and position embedding vector of each non-overlapping image block.

[0090] For example, first scale the remote sensing image to... Then divide it into sizes of Non-overlapping image patches are processed by a convolution kernel of size 1. The convolution operation maps each non-overlapping image patch to a 768-dimensional feature vector.

[0091] S202: Add the feature vectors and position embedding vectors of each non-overlapping image block to obtain the position feature sequence of each non-overlapping image block, and input the position feature sequence of the non-overlapping image block into the Transformer encoder.

[0092] Specifically, each Transformer Block in the Transformer encoder mainly consists of a multi-head self-attention layer and a multilayer perceptron (MLP). The multi-head self-attention layer is mainly used to extract image features. In the self-attention mechanism, the position feature sequences of each input image block are mapped to a query matrix Q, a key matrix K, and a value matrix V respectively through a self-attention linear layer. This allows the calculation of attention weights between image blocks, which are then applied to the value matrix to obtain the fused features.

[0093] Optionally, when the remote sensing image is large, directly extracting and extracting ground features from the entire remote sensing image requires significant computational resources and has low extraction efficiency. Therefore, in step S202, the positional feature sequences of each non-overlapping image block can be input into the Transformer encoder to obtain multi-scale feature maps for each non-overlapping image block. This allows for separate extraction of ground features from each non-overlapping image block. Finally, the extraction results of ground features from all non-overlapping image blocks are fused to obtain the ground feature vector contour of the entire remote sensing image. By employing a slicing and shift interpolation inference strategy, a large-scale, high-resolution urban remote sensing image is divided into multiple image blocks of appropriate scale. Ground feature vector contours are extracted from each image block separately and then fused. This effectively alleviates the problem of missed detections and misjudgments of small ground features when directly inferring from the entire image, reduces the memory and computational resource overhead required for a single inference, and improves the extraction efficiency of the model in large-scale remote sensing scenarios.

[0094] S203: Use a feature aggregator to reduce the dimensionality of the intermediate features output by the m-th Transformer Block in the Transformer encoder to obtain the m-th dimensionality-reduced intermediate feature; add the m-th dimensionality-reduced intermediate feature with the residuals and superimpose it with the (m-2)-th fused feature, and perform feature enhancement on the superimposed feature to obtain the m-th fused feature; perform channel recovery on the last fused feature to obtain the target fused feature; where m is an odd number and m>1, and the first fused feature is the feature after dimensionality reduction of the intermediate features output by the first Transformer Block.

[0095] For example, the dimensionality reduction module is used to reduce the dimensionality of the m-th intermediate feature to 32. The dimensionality reduction module consists of... It consists of convolutional blocks and ReLU activation functions; a feature enhancement module is used to enhance the superimposed features. The feature enhancement module consists of... It consists of convolutional blocks and ReLU activation functions; the restoration module restores the channel dimension of the last fused feature to 256. The restoration module consists of two... Convolutional block and a It consists of convolutional blocks.

[0096] S204: Upsample the target fusion features using transposed convolution in the feature decomposer to generate first and second mapping features with spatial scales magnified by one and two times respectively; downsample the target fusion features using max pooling in the feature decomposer to generate third and fourth mapping features with spatial scales reduced by one and two times respectively.

[0097] S205: Based on target fusion features, first mapping features, second mapping features, third mapping features and fourth mapping features, a multi-scale feature map with different spatial resolutions but consistent channel dimensions is obtained.

[0098] Furthermore, to obtain reliable candidate regions and high-quality bounding boxes, this application inputs multi-scale feature maps into the RPN network to generate multiple candidate regions on feature maps at various scales for initial localization of potential building targets. Then, a region of interest alignment operation is used to crop the bounding boxes for each candidate region on the corresponding scale feature maps. ROI features are input into the bounding box prediction head. For each candidate region, target category prediction and bounding box regression are performed, resulting in more accurate target bounding boxes. Figure 4 The diagram shown is a schematic of target bounding box detection using the RPN network provided in this application.

[0099] Furthermore, this application also designs an automatic blending prompt generation mechanism based on the aforementioned target bounding box, such as... Figure 5 The diagram shows the automatic blending and lifting generation process based on ROI provided in this application. For example, it illustrates the process of generating bounding boxes for each target. The scale is expanded, and the center point of the expanded target bounding box is used as the foreground point tooltip with coordinates (0.5, 0.5). Then, the center points of the top, bottom, left, and right regions of the expanded target bounding box are selected as background point tooltips, with coordinates as follows: , , , Based on the bounding boxes of each extended target, foreground point hints, and background point hints, box hints and point hints can be obtained.

[0100] Furthermore, in order to achieve fine extraction of ground feature vector contours while maintaining the original structure of the SAM decoder, this application introduces a multi-task prediction mechanism for ground feature vector contours, namely a multi-task decoder, based on the SAM decoder framework. Figure 6 As shown. Specifically, each extended target bounding box and the cue token are input into the SAM decoder, which outputs the feature mask map, vertex confidence map, vertex offset, and boundary mask map for each extended target bounding box, including S400~S405:

[0101] S400: Perform region of interest alignment operation on the regions where each extended target bounding box is located in the multi-scale feature map, and extract the extended region of interest features of each extended target bounding box.

[0102] S401: Concatenate the mask token of the extended region of interest features, the mask intersection-union score token, and the preset polygon token used to fit the shape of the ground feature vector contour of each extended target bounding box with the cue token of the extended target bounding box to obtain the concatenated token sequence of each extended target bounding box.

[0103] S402: Input the extended region of interest features of each target bounding box and their concatenated token sequence into the SAM decoder. The SAM decoder flattens the extended region of interest features and maps them to obtain a one-dimensional feature vector of each extended target bounding box.

[0104] S403: Input the one-dimensional feature vectors of each extended target bounding box and their concatenated token sequence into the bidirectional Transformer module in the multi-task decoder to perform cross-attention calculation, and output the updated one-dimensional feature vectors and concatenated token sequence.

[0105] S404: Perform two-dimensional shape restoration on the updated one-dimensional feature vectors of each extended target bounding box, and upsample the restored two-dimensional feature vectors to the target space resolution; separate the mask token, mask intersection-union score token, cue token and polygon token from the stitched token sequence updated by each extended target bounding box.

[0106] S405: Perform matrix multiplication operations on the mask token, mask intersection-union score token, cue token, and polygon token of each separated extended target bounding box with the upsampled two-dimensional feature vector of each extended target bounding box to obtain the ground feature mask map, mask intersection-union score, vertex confidence map, vertex offset, and boundary mask map of each target bounding box.

[0107] It should be noted that after obtaining the tokens of each subtask, the dimension of each subtask's token needs to be restored to 256 before matrix multiplication with the two-dimensional vector.

[0108] Furthermore, such as Figure 7 As shown, using a mask-guided vertex connection algorithm, based on the feature mask map, vertex confidence map, vertex offset, and boundary mask map of each extended target bounding box, the feature vector contours of the remote sensing image are obtained, including S500~S505:

[0109] S500: Perform local max pooling on the vertex confidence map of each extended target bounding box, obtain the pixel with the maximum confidence score in each local neighborhood, and select the pixel with the maximum confidence score greater than a preset threshold as the candidate vertex of the extended target bounding box.

[0110] For example, the preset threshold can be 0.1.

[0111] S501: Based on the vertex offset of each extended target bounding box, the pixel-level coordinates of each candidate vertex of the extended target bounding box are corrected, and the vertex set of the extended target bounding box is obtained based on the corrected candidate vertices.

[0112] S502: Extract the contour point sequence of ground features based on the ground feature mask map of each extended target bounding box, and search for the contour point with the smallest distance to each vertex in the vertex set of the extended target bounding box in the contour point sequence to obtain the contour point corresponding to each vertex.

[0113] S503: Connect each vertex in the vertex set of each extended target bounding box to its adjacent vertices to obtain the vertex vector of each vertex; connect the contour point corresponding to each vertex to the contour point corresponding to the adjacent vertex of that vertex to obtain the contour point vector of each vertex.

[0114] S504: If the angle difference between the vertex vector and the contour point vector of a vertex in the expanded target bounding box is less than the preset angle difference, then the vertex is taken as a candidate vector contour point of the expanded target bounding box.

[0115] S505: Based on all candidate vector contour points of each extended target bounding box, obtain the ground feature vector contour of that extended target bounding box; based on the ground feature vector contours of all extended target bounding boxes, obtain the ground feature vector contour of the remote sensing image.

[0116] In some embodiments, the training process of the feature extraction model includes S11~S15:

[0117] S11: Based on the land cover features in all remote sensing image samples in the training set, calculate the proportion of remote sensing image samples containing each type of land cover feature to the total number of remote sensing image samples, and obtain the occurrence frequency of each type of land cover feature.

[0118] Specifically, the formula for calculating the frequency of occurrence of each type of land feature is as follows:

[0119] ,

[0120] in, This indicates the frequency of occurrence of the c-th type of land feature; This represents the number of remote sensing image samples containing the c-th type of land cover features; This represents the total number of remote sensing image samples in the training set.

[0121] S12: Calculate the category repetition factor of each category of land cover based on the preset frequency threshold and the occurrence frequency of each category of land cover; calculate the repetition factor of each remote sensing image sample based on the land cover category contained in each remote sensing image sample and the category repetition factor of each category of land cover.

[0122] Specifically, the formula for calculating the category repetition factor for each category of land cover features is as follows:

[0123] ,

[0124] in, The category repetition factor represents the category of the land feature in category c; This indicates the frequency of occurrence of the c-th category of land feature; This represents a preset frequency threshold, used to control the degree of repetition of low-frequency categories during training. The larger the threshold t, the larger the repetition factor corresponding to the low-frequency category, and the higher the sampling probability obtained during training.

[0125] The formula for calculating the repetition factor of each remote sensing image sample is:

[0126] ,

[0127] in, Represents remote sensing image samples The repeating factor; Represents remote sensing image samples It is a collection of all categories of geographic features.

[0128] S13: Based on the repetition factor of each remote sensing image sample, sample the remote sensing image samples in the training set to obtain the target sample.

[0129] S14: Input the target sample into the SAM encoder, RPN network, cue generator and SAM decoder, output the land feature vector contour prediction result of the target sample, and then calculate the value of the land feature extraction loss function.

[0130] S15: Return to execute S13 to iteratively train the SAM encoder, RPN network, cue generator, and SAM decoder until the value of the feature extraction loss function converges, thus obtaining the trained SAM encoder, RPN network, cue generator, and SAM decoder, and thus obtaining the trained feature extraction model.

[0131] Specifically, due to the significant differences in the number of samples among different categories of land features, such as the overwhelming number of building instances, the model is more likely to learn the features of the building category during training, while it is less able to learn the features of other categories such as bridges. To address this, this application designs a dynamic sampling strategy based on image category frequency. By adjusting the participation frequency of samples of different categories during training, the minority category samples receive more attention during model training, thereby improving the model's ability to learn the minority categories.

[0132] Furthermore, such as Figure 8The diagram illustrates the principle of the efficient parameter fine-tuning strategy provided in this application. Since the Transformer encoder in the SAM encoder has many parameters, during iterative training, multiple low-rank adaptation modules are set up in parallel with the original linear mapping branch on the self-attention layer of the Transformer encoder in the SAM encoder. The output of the original linear mapping branch is added to the outputs of the multiple low-rank adaptation modules to obtain the output of the self-attention layer. During each iteration, the parameters of the original linear mapping branch are frozen, and only the parameters of the multiple low-rank adaptation modules are updated. After iterative training, the parameters of the multiple low-rank adaptation modules and the parameters of the original linear mapping branch are fused using matrix addition to obtain the final weights of the linear mapping in the self-attention layer.

[0133] Furthermore, the calculation process of the feature extraction loss function includes:

[0134] The RPN loss is calculated based on the classification loss and regression loss of each target bounding box in the target sample output by the RPN network.

[0135] The sum of the regression loss and the feature vector contour prediction loss of each target bounding box is conditionally calculated using the indicator function to obtain the geometric constraint loss of each target bounding box after constraint.

[0136] The geometric constraint loss and classification loss for each target bounding box are summed to obtain the multi-task loss for each target bounding box;

[0137] The average of the multi-task loss of all target bounding boxes in the target sample is calculated, and the value of the feature extraction loss function is obtained based on the sum of the average value and the RPN loss.

[0138] Specifically, the loss function for extracting ground features Represented as:

[0139] ,

[0140] in, Indicates RPN loss; This indicates the number of bounding boxes of targets in a remotely sensed image sample. Let represent the classification loss for the i-th target bounding box; This represents the geometric constraint loss for the i-th target bounding box; Let represent the regression loss for the i-th target bounding box; The feature vector profile prediction loss represents the loss for the i-th target bounding box. This represents an indicator function used to distinguish between positive and negative samples. The i-th target bounding box is considered a positive sample when the intersection-union ratio (IoU) between it and the ground truth bounding box is greater than 0.5. When the intersection-union ratio (IoU) between the i-th target bounding box and the ground truth bounding box is less than or equal to 0.5, the i-th target bounding box is considered a negative sample. .

[0141] The formula for calculating the loss in ground feature vector profile prediction is:

[0142] ,

[0143] in, The ground feature mask prediction loss represents the bounding box of the i-th target. The weights representing the prediction loss of the ground feature mask map; Let represent the vertex confidence graph prediction loss for the i-th target bounding box; The weights represent the prediction loss of the vertex confidence graph; This represents the vertex offset prediction loss for the i-th target bounding box; The weights represent the vertex offset prediction loss; The boundary mask prediction loss represents the bounding box prediction loss for the i-th target bounding box. The weights represent the prediction loss of the boundary mask image; The mask intersection-union score prediction loss represents the loss of the i-th target bounding box. This represents the weight of the mask intersection ratio score prediction loss.

[0144] It should be noted that this application uses weighted binary cross-entropy loss. With Dice loss Combinatorial calculation , and The L1 distance between the predicted and actual offsets is obtained by calculating the distance between them. The mean square error between the predicted mask cross-union score and the actual mask cross-union score is obtained by calculating the mean square error. .

[0145] Furthermore, QGIS, as a widely used open-source geographic information system platform, possesses mature layer management, spatial analysis, and visualization capabilities, playing a crucial role in remote sensing data processing and urban feature analysis. However, existing QGIS-based feature extraction tools primarily rely on traditional image processing methods or offline deep learning inference, making it difficult to fully leverage the interactive advantages of cue-driven segmentation models such as SAM. Especially in practical applications, users often desire to trigger model inference directly within the QGIS interface through simple interactive methods such as point or box prompts, obtaining detailed and continuous feature vector contour results in real time, while also intuitively perceiving the model's inference efficiency and response speed. Therefore, this application provides a QGIS plugin system, CitySAM, developed based on the aforementioned feature extraction model, such as... Figure 9The diagram shown is the architecture diagram of the CitySAM plugin system provided in this application.

[0146] This application embodiment also uses the plugin to extract building vector contours from a remote sensing image, which includes steps 1 to 12:

[0147] Step 1: Copy the plugin code folder to the QGIS plugin directory, and install the corresponding dependency libraries according to the plugin's running requirements to complete the plugin's deployment and initialization configuration;

[0148] Step 2: Drag the raster file to be processed into the QGIS canvas and display it;

[0149] Step 3: Open the image feature extraction tool (CitySAM Image Encoder), select the raster file, and configure various parameters according to the prompts. The weight file can be directly used from the model's weight file.

[0150] Step 4: Divide the raster file into blocks, and use the model's efficient fine-tuning SAM encoder to extract features from each block;

[0151] Step 5: Select the output location; the image patch features and corresponding geographic location indexes are saved.

[0152] Step 6: Open the vector contour extraction tool (CitySAM Polygon Extraction) and import the saved image patch feature addresses into it;

[0153] Step 7: The user adds hints on the QGIS canvas. CitySAM can obtain the largest bounding rectangle based on all the hints added by the user, which is the user's region of interest.

[0154] Step 8: CitySAM finds all image patch features that intersect with the region of interest, calculates the center distance between the region of interest and the image patch features in turn, and finds the closest image patch feature;

[0155] Step 9: CitySAM inputs the user-added hints and image patch features into the vector contour decoder, i.e., the multi-task decoder, to obtain the mask map, vertex confidence map, and vertex offset of the region of interest;

[0156] Step 10: Using the mask-guided vertex connection algorithm, obtain the vector contour of the features in the region of interest based on the mask image, vertex confidence image, and vertex offset;

[0157] Step 11: CitySAM maps the vector outline to the coordinate system of the raster file and displays it in the QGIS canvas. Alternatively, you can save it locally.

[0158] Step 12: Users can fine-tune the vector outline using QGIS's built-in vector file processing tools.

[0159] like Figure 10 The diagram shown illustrates the urban feature extraction architecture provided in this application. This application introduces SAM as the basic model. To meet the automated extraction requirements of urban features from the entire remote sensing image, an automatic hybrid cue generation mechanism based on ROI is designed. This mechanism can automatically generate high-quality bounding box cueing, foreground points, and background points without requiring manual interaction on a per-target basis. This not only conforms to the cueing input method of SAM but also significantly reduces manual intervention, improving overall processing efficiency while maintaining segmentation accuracy. Addressing the issue that existing SAM models can only output pixel-level mask results and are unable to directly obtain fine vector contours, a multi-task decoder is introduced based on the original SAM decoder structure to realize the processing of instance masks, vertex confidence information, and... This paper proposes a joint prediction method using vertex offset information and boundary masks, and further proposes a mask-guided vertex connection strategy. Under the geometric constraints of instance masks, disordered vertices are sorted and connected, thereby enabling stable and accurate output of the vector contour results of target features. Furthermore, the above model and method are integrated into the QGIS platform to develop the CitySAM plugin system. By performing image block segmentation and feature pre-encoding storage on large-scale remote sensing images, and quickly querying and decoding the most matching image block features after user-provided interactive prompts, the system achieves rapid generation and real-time visualization of feature vector contours. Simultaneously, the output vector contour results can be directly mapped to the original geographic coordinate system and further edited and analyzed in the QGIS environment, balancing computational efficiency and engineering practicality.

[0160] To verify the effectiveness of the above-mentioned urban feature extraction method, this application embodiment also selects the extended Typical Cities Building Dataset (TCBD) and uses remote sensing images of a certain area within it as test data for experimental verification.

[0161] 1. Experimental Data

[0162] The TCBD dataset contains 7260 image region samples, with a total of 63886 building instances annotated. All images are derived from high-resolution remote sensing imagery, with a spatial resolution of 0.3m and a single image size of 500×500 pixels. This resolution can clearly depict the geometric outlines and detailed structures of buildings, making it suitable for refined building extraction tasks.

[0163] Building upon the aforementioned dataset, this embodiment further utilizes the LabelMe tool to manually annotate bridge and tunnel entrance targets in the imagery, thereby constructing an expanded TCBD dataset. The expanded dataset retains the original building instance annotations while adding two new land features: bridges and tunnel entrances. The dataset contains 63,886 building instances, 649 bridge instances, and 49 tunnel entrance instances, providing data support for multi-category urban land feature extraction and model research in imbalanced scenarios.

[0164] 2. Experimental Setup

[0165] This embodiment uses the MMDetection and PyTorch Lightning frameworks to build the experimental environment, and performs model training and validation on an NVIDIA RTX 3090 GPU. The experiment employs a two-stage instance segmentation network based on the Segment Anything Model (SAM) as the basic architecture, with the ViT-B structure used as the backbone network and pre-trained weights loaded for initialization. During model training, the AdamW optimizer is used, with an initial learning rate of 0.0005 and a weight decay coefficient of 1×10⁻⁶. -4 Different learning rates were set for different sub-modules. The instance detection and segmentation head used the base learning rate, the SAM mask decoder used a learning rate of 0.1 times the base learning rate, and the image encoder used a learning rate that was further reduced to 0.01 times when it participated in training.

[0166] The training process consisted of 150 epochs, employing a learning rate scheduling strategy combining linear warm-up and cosine annealing. The first epoch used linear warm-up, followed by cosine annealing to gradually decrease the learning rate. Model training was performed on four GPUs in a distributed, data-parallel manner. Performance evaluation was conducted on the validation set every five epochs, and the mask_IoU (valmask_iou) on the validation set was used as the model selection metric, retaining the weights of the best-performing model.

[0167] In terms of data processing, all input remote sensing images were uniformly scaled to a resolution of 1024×1024 and subjected to data augmentation operations such as random flipping. To alleviate the class imbalance problem, a dynamic sampling strategy based on class frequency was introduced in both the training and validation phases, so that images containing targets of a minority class would receive a higher sampling probability during training.

[0168] During the model evaluation phase, a multi-index evaluation method based on the COCO protocol was adopted to evaluate the bounding box, mask, and polygon results separately. The category prediction threshold was set to 0.1, and performance indicators were statistically analyzed for different categories to comprehensively evaluate the model's detection and segmentation performance in multi-category urban land feature extraction tasks.

[0169] 3. Evaluation Indicators

[0170] This embodiment uses mAP 50 As an evaluation metric, with the Intersection over Union (IoU) threshold between the predicted results and the ground truth annotations set to 0.5, the average accuracy of the bounding box (bbox), pixel-level segmentation mask (seg), and vector polygon contour (poly) were calculated, and the average values ​​were taken for three target types: buildings, bridges, and tunnel entrances. Specifically, bbox_mAP... 50 Used to evaluate target detection and localization performance, seg_mAP 50 Poly_mAP is used to measure pixel-level segmentation accuracy. 50 It is used to evaluate the model's ability to represent the target vector contour, thereby comprehensively reflecting the model's overall performance under multiple categories and multiple representations.

[0171] 4. Experimental Results

[0172] Table 1 shows the experimental results of different methods and fine-tuning strategies in extended TCBD. Based on the experimental results in Table 1, the following conclusions can be drawn:

[0173] First, compared with traditional instance segmentation methods (Mask R-CNN, Cascade Mask R-CNN, and RTMDet), the method introducing a large model fine-tuning strategy has a significant advantage in segmentation performance. Specifically, traditional detection and segmentation methods are limited in overall performance on the mask's mAP50 metric and cannot directly output high-quality polygonal contours (poly_mAP50), making it difficult to meet the needs of refined building contour extraction.

[0174] Secondly, for the task of extracting ground feature vector contours, the FFL method has achieved some improvement in poly_mAP50 by introducing frame field constraints, but its performance in instance localization (bbox_mAP50) is poor. In contrast, the efficient fine-tuning method based on the SAM large model achieves better detection, segmentation and polygon extraction performance while maintaining a low number of trainable parameters, demonstrating the expressive advantages of large models in complex remote sensing scenarios.

[0175] Further comparison of different SAM fine-tuning strategies reveals that while the Adapter and Prefix-Tuning methods significantly reduce the number of trainable parameters, they still suffer some performance loss on the poly_mAP50 metric, indicating their limited ability to model the fine geometric structure of ground features. The LoRA fine-tuning method achieves a better balance between accuracy and efficiency; SAM_LoRA (r=32) introduces only 1.18M trainable parameters, yet achieves superior performance on both segmentation and polygon metrics.

[0176] Table 1

[0177]

[0178] When the rank of LoRA is further increased to r=96, the model's bbox_mAP50 and seg_mAP50 on both the training and test sets are improved, but its generalization ability on unseen images shows a certain downward trend, indicating a risk of overfitting. In contrast, the SAM_MLoRA method proposed in this application uses three LoRAs with r=32 instead of the r=96 LoRA. While maintaining the same parameter scale as SAM_LoRA (r=96), it effectively alleviates the overfitting problem through a multi-low-rank structure design. It achieves optimal or near-optimal comprehensive performance in the three metrics of bbox_mAP50, seg_mAP50, and poly_mAP50, verifying the advantages of the proposed method in terms of accuracy, generalization, and computational efficiency.

[0179] like Figure 11 The diagram shown illustrates the visual detection results of the method provided in this application and existing methods on test data; wherein, Figure 11 Image (a) shows the detection results of the FFL method on three remote sensing images. Figure 11 (b) in the figure is a schematic diagram of the detection results of the SAM_Adapter method on three remote sensing images. Figure 11 (c) in the figure is a schematic diagram of the detection results of the SAM_Prefix-Tuning method on three remote sensing images. Figure 11 (d) in the figure is a schematic diagram of the detection results of the SAM_LoRA (r=32) method on three remote sensing images. Figure 11 (e) in the figure is a schematic diagram of the detection results of the SAM_LoRA (r=96) method on three remote sensing images. Figure 11 (f) in the figure is a schematic diagram of the detection results of the method provided in this application on three remote sensing images. Figure 11 (g) in the figure is a schematic diagram of the detection results of the Ground Truth method on three remote sensing images.

[0180] from Figure 11As can be seen, the method provided in this application achieves the best contour extraction results in different remote sensing images. In contrast, the FFL method generates more redundant vertices during contour extraction and performs poorly in capturing some architectural details. Furthermore, when using the Adapter and Prefix-Tuning fine-tuning strategies, the method in this application still suffers from a small number of missed vertex detections. When using a single LoRA fine-tuning strategy, the model may misidentify shadows near the target as contours. Overall, the method provided in this application outperforms other comparative methods in terms of contour completeness and accuracy.

[0181] This application embodiment also provides a ground feature extraction device, which specifically includes:

[0182] The model acquisition module is used to acquire a trained ground feature extraction model, which includes a SAM encoder, an RPN network, a cue generator, and a SAM decoder.

[0183] The feature extraction and target detection module is used to input remote sensing images into the SAM encoder and output multi-scale feature maps; the multi-scale feature maps are then input into the RPN network to perform target detection using ground features as targets and generate target bounding boxes.

[0184] The prompt generation module is used to expand each target bounding box to obtain an expanded target bounding box, use the center point of the expanded target bounding box as the foreground point prompt, and obtain the background point prompt based on the center point of the expanded region along the positive y-axis direction, the center point of the expanded region along the negative y-axis direction, the center point of the expanded region along the positive x-axis direction, and the center point of the expanded region along the negative x-axis direction.

[0185] The mask generation module is used to input each extended target bounding box and its foreground and background point cues into the cue generator and output the cue token; it also inputs each extended target bounding box and its cue token into the SAM decoder and outputs the ground feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box.

[0186] The ground feature vector contour acquisition module is used to obtain the ground feature vector contour of the remote sensing image based on the ground feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box using a mask-guided vertex connection algorithm.

[0187] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0188] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0189] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0191] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for extracting urban feature elements, characterized in that, include: Obtain a trained feature extraction model, which includes a SAM encoder, an RPN network, a cue generator, and a SAM decoder; The remote sensing image is input into the SAM encoder, which outputs a multi-scale feature map. Multi-scale feature maps are input into the RPN network to perform target detection and generate target bounding boxes using ground features as targets. Each target bounding box is expanded to obtain an expanded target bounding box. The center point of the expanded target bounding box is used as the foreground point cue. The background point cue of the expanded target bounding box is obtained based on the center points of the expanded regions along the positive y-axis, negative y-axis, positive x-axis, and negative x-axis directions of the target bounding box. Input each extended target bounding box, along with its foreground and background point cues, into the cue generator, and output the cue token; Each extended target bounding box and its cue token are input into the SAM decoder, which outputs the ground feature mask, vertex confidence map, vertex offset, and boundary mask for each extended target bounding box. Using a mask-guided vertex connection algorithm, the vector contours of ground features in the remote sensing image are obtained based on the ground feature mask map, vertex confidence map, vertex offset, and boundary mask map of each extended target bounding box. The training process for the feature extraction model includes: S11: Based on the land cover features in all remote sensing image samples in the training set, calculate the proportion of remote sensing image samples containing each type of land cover feature to the total number of remote sensing image samples, and obtain the occurrence frequency of each type of land cover feature. S12: Based on a preset frequency threshold and the occurrence frequency of each category of land cover elements, calculate the category repetition factor for each category of land cover elements; based on the land cover element categories contained in each remote sensing image sample and the category repetition factor for each category of land cover elements, calculate the repetition factor for each remote sensing image sample; wherein, the formula for calculating the category repetition factor for each category of land cover elements is: ， in, The category repetition factor represents the category of the land feature in category c; This indicates the frequency of occurrence of the c-th category of land feature; Indicates the preset frequency threshold; The formula for calculating the repetition factor of each remote sensing image sample is: ， in, Represents remote sensing image samples The repeating factor; Represents remote sensing image samples The set consisting of all categories of geographic features; S13: Based on the repetition factor of each remote sensing image sample, sample the remote sensing image samples in the training set to obtain the target sample; S14: Input the target sample into the SAM encoder, RPN network, cue generator and SAM decoder, output the land feature vector contour prediction result of the target sample, and then calculate the value of the land feature extraction loss function; S15: Return to execute S13 to iteratively train the SAM encoder, RPN network, cue generator, and SAM decoder until the value of the feature extraction loss function converges, thus obtaining the trained SAM encoder, RPN network, cue generator, and SAM decoder, and thus obtaining the trained feature extraction model.

2. The method for extracting urban feature elements according to claim 1, characterized in that, The remote sensing image is input into the SAM encoder, which outputs a multi-scale feature map, including: The remote sensing image is input into the image block embedding layer and divided into blocks to obtain multiple non-overlapping image blocks; Convolution and position encoding are performed on each non-overlapping image patch to obtain the feature vector and position embedding vector of each non-overlapping image patch. The feature vectors and position embedding vectors of each non-overlapping image block are added together to obtain the position feature sequence of each non-overlapping image block. The position feature sequence of the non-overlapping image block is then input into the Transformer encoder. The intermediate features output by the m-th Transformer Block in the Transformer encoder are reduced in dimensionality using a feature aggregator to obtain the m-th dimensionality-reduced intermediate feature. The m-th dimensionality-reduced intermediate feature is then superimposed with the (m-2)-th fused feature by adding the residuals, and the superimposed feature is enhanced to obtain the m-th fused feature. Channel recovery is then performed on the last fused feature to obtain the target fused feature. Here, m is an odd number and m>1, and the first fused feature is the feature after dimensionality reduction of the intermediate features output by the first Transformer Block. The target fusion features are upsampled using transposed convolution in the feature decomposer to generate first and second mapping features with spatial scales magnified by one and two times, respectively; the target fusion features are downsampled using max pooling in the feature decomposer to generate third and fourth mapping features with spatial scales reduced by one and two times, respectively. Based on target fusion features, first mapping features, second mapping features, third mapping features and fourth mapping features, multi-scale feature maps with different spatial resolutions but consistent channel dimensions are obtained.

3. The method for extracting urban feature elements according to claim 1, characterized in that, Each extended target bounding box and its cue token are input into the SAM decoder, which outputs the feature mask map, vertex confidence map, vertex offset, and boundary mask map for each extended target bounding box, including: In the multi-scale feature map, perform region of interest alignment operation on the region where each extended target bounding box is located, and extract the extended region of interest features of each extended target bounding box; The mask token of the extended region of interest features, the mask intersection-union score token, and the preset polygon token used to fit the shape of the ground feature vector outline of each extended target bounding box are concatenated with the cue token of the extended target bounding box to obtain the concatenated token sequence of each extended target bounding box. The extended region of interest features of each target bounding box and their concatenated token sequence are input into the SAM decoder. The SAM decoder flattens the extended region of interest features and maps them to obtain a one-dimensional feature vector of each extended target bounding box. The one-dimensional feature vectors of each extended target bounding box and their concatenated token sequences are input into the bidirectional Transformer module in the multi-task decoder to perform cross-attention calculation, and the updated one-dimensional feature vectors and concatenated token sequences are output. Two-dimensional shape restoration is performed on the updated one-dimensional feature vectors of each extended target bounding box, and the restored two-dimensional feature vectors are upsampled to the target space resolution; the mask token, mask intersection-union score token, cue token and polygon token are separated from the stitched token sequence updated by each extended target bounding box. The mask token, mask intersection-union score token, cue token, and polygon token of each separated extended target bounding box are multiplied by matrix multiplication with the upsampled two-dimensional feature vector of each extended target bounding box to obtain the ground feature mask, mask intersection-union score, vertex confidence map, vertex offset, and boundary mask of each target bounding box.

4. The method for extracting urban feature elements according to claim 1, characterized in that, Using a mask-guided vertex connection algorithm, based on the feature mask map, vertex confidence map, vertex offset, and boundary mask map of each extended target bounding box, the feature vector contours of the remote sensing image are obtained, including: Local max pooling is performed on the vertex confidence map of each expanded target bounding box to obtain the pixel with the maximum confidence score in each local neighborhood. The pixels with a confidence score greater than a preset threshold among the pixels with the maximum confidence score are selected as candidate vertices of the expanded target bounding box. The pixel-level coordinates of each candidate vertex of the expanded target bounding box are corrected based on the vertex offset of each expanded target bounding box, and the vertex set of the expanded target bounding box is obtained based on the corrected candidate vertices. Based on the feature mask map of each extended target bounding box, extract the feature contour point sequence, and search in the feature contour point sequence for the contour point with the smallest distance to each vertex in the vertex set of the extended target bounding box to obtain the contour point corresponding to each vertex. Connect each vertex in the vertex set of each extended target bounding box to its adjacent vertices to obtain the vertex vector of each vertex; connect the contour point corresponding to each vertex to the contour point corresponding to the adjacent vertices of that vertex to obtain the contour point vector of each vertex. If the angle difference between the vertex vector and the contour point vector of a vertex in the expanded target bounding box is less than a preset angle difference, then the vertex is taken as a candidate vector contour point of the expanded target bounding box. Based on all candidate vector contour points of each extended target bounding box, the feature vector contour of that extended target bounding box is obtained; based on the feature vector contours of all extended target bounding boxes, the feature vector contour of the remote sensing image is obtained.

5. The method for extracting urban feature elements according to claim 1, characterized in that, During iterative training, multiple low-rank adaptation modules are set up in parallel with the original linear mapping branch on the self-attention layer of the Transformer encoder of the SAM encoder. The output of the original linear mapping branch is added to the output of the multiple low-rank adaptation modules as the output of the self-attention layer. In each iteration, the parameters of the original linear mapping branch are frozen, and only the parameters of the multiple low-rank adaptation modules are updated. After iterative training, the parameters of multiple low-rank adaptation modules are fused with the parameters of the original linear mapping branch by matrix addition as the final weights of the self-attention layer linear mapping.

6. The method for extracting urban feature elements according to claim 1, characterized in that, The calculation process of the loss function for extracting geographic features includes: The RPN loss is calculated based on the classification loss and regression loss of each target bounding box in the target sample output by the RPN network. The sum of the regression loss and the feature vector contour prediction loss of each target bounding box is conditionally calculated using the indicator function to obtain the geometric constraint loss of each target bounding box after constraint. The geometric constraint loss and classification loss for each target bounding box are summed to obtain the multi-task loss for each target bounding box; The average of the multi-task loss of all target bounding boxes in the target sample is calculated, and the value of the feature extraction loss function is obtained based on the sum of the average value and the RPN loss.

7. The method for extracting urban feature elements according to claim 6, characterized in that, Loss function for extracting ground features Represented as: ， in, Indicates RPN loss; This indicates the number of bounding boxes of targets in a remotely sensed image sample. Let represent the classification loss for the i-th target bounding box; This represents the geometric constraint loss for the i-th target bounding box; Let represent the regression loss for the i-th target bounding box; The feature vector profile prediction loss represents the loss for the i-th target bounding box. This represents an indicator function used to distinguish between positive and negative samples. The i-th target bounding box is considered a positive sample when the intersection-union ratio (IoU) between it and the ground truth bounding box is greater than 0.

5. When the intersection-union ratio (IoU) between the i-th target bounding box and the ground truth bounding box is less than or equal to 0.5, the i-th target bounding box is considered a negative sample. ; The formula for calculating the loss in ground feature vector profile prediction is: ， in, The ground feature mask prediction loss represents the bounding box of the i-th target. The weights representing the prediction loss of the ground feature mask map; Let represent the vertex confidence graph prediction loss for the i-th target bounding box; The weights represent the prediction loss of the vertex confidence graph; This represents the vertex offset prediction loss for the i-th target bounding box; The weights represent the vertex offset prediction loss; The boundary mask prediction loss represents the bounding box prediction loss for the i-th target bounding box. The weights represent the prediction loss of the boundary mask image; The mask intersection-union score prediction loss represents the loss of the i-th target bounding box. This represents the weight of the mask intersection ratio score prediction loss.

8. A device for extracting ground features, characterized in that, include: The model acquisition module is used to acquire a trained ground feature extraction model, which includes a SAM encoder, an RPN network, a cue generator, and a SAM decoder. The feature extraction and target detection module is used to input remote sensing images into the SAM encoder and output multi-scale feature maps; the multi-scale feature maps are then input into the RPN network to perform target detection using ground features as targets and generate target bounding boxes. The prompt generation module is used to expand each target bounding box to obtain an expanded target bounding box, use the center point of the expanded target bounding box as the foreground point prompt, and obtain the background point prompt based on the center point of the expanded region along the positive y-axis direction, the center point of the expanded region along the negative y-axis direction, the center point of the expanded region along the positive x-axis direction, and the center point of the expanded region along the negative x-axis direction. The mask generation module is used to input the bounding boxes of each extended target, along with their foreground and background point cues, into the cue generator and output a cue token. Each extended target bounding box and its cue token are input into the SAM decoder, which outputs the ground feature mask, vertex confidence map, vertex offset, and boundary mask for each extended target bounding box. The ground feature vector contour acquisition module is used to obtain the ground feature vector contour of the remote sensing image based on the ground feature mask map, vertex confidence map, vertex offset and boundary mask map of each extended target bounding box using a mask-guided vertex connection algorithm. The training process for the feature extraction model includes: S11: Based on the land cover features in all remote sensing image samples in the training set, calculate the proportion of remote sensing image samples containing each type of land cover feature to the total number of remote sensing image samples, and obtain the occurrence frequency of each type of land cover feature. S12: Based on a preset frequency threshold and the occurrence frequency of each category of land cover elements, calculate the category repetition factor for each category of land cover elements; based on the land cover element categories contained in each remote sensing image sample and the category repetition factor for each category of land cover elements, calculate the repetition factor for each remote sensing image sample; wherein, the formula for calculating the category repetition factor for each category of land cover elements is: ， in, The category repetition factor represents the category of the land feature in category c; This indicates the frequency of occurrence of the c-th category of land feature; Indicates the preset frequency threshold; The formula for calculating the repetition factor of each remote sensing image sample is: ， in, Represents remote sensing image samples The repeating factor; Represents remote sensing image samples The set consisting of all categories of geographic features; S13: Based on the repetition factor of each remote sensing image sample, sample the remote sensing image samples in the training set to obtain the target sample; S14: Input the target sample into the SAM encoder, RPN network, cue generator and SAM decoder, output the land feature vector contour prediction result of the target sample, and then calculate the value of the land feature extraction loss function; S15: Return to execute S13 to iteratively train the SAM encoder, RPN network, cue generator, and SAM decoder until the value of the feature extraction loss function converges, thus obtaining the trained SAM encoder, RPN network, cue generator, and SAM decoder, and thus obtaining the trained feature extraction model.

Images