Building edge segmentation method and system based on adaptive prompting of large segmentation model

Abstract

The present invention discloses a method and system for building edge segmentation based on adaptive prompting of a large segmentation model, belonging to the field of remote sensing image processing technology. The method of the present invention first inputs the image to be segmented into the SAM large model for full segmentation, and a classifier identifies the semantic information of the segmented patches; performs instance object recognition and spatial reorganization on the segmented patches, and automatically generates a set of positive prompt points and a set of negative prompt points through spatial clustering and similarity calculation; re-inputs the image to be segmented and the positive and negative prompt point sets of the instance objects into the SAM large model for fine-grained re-segmentation of the building edges, and finally obtains a complete segmentation result of the image to be segmented. Compared with the existing SAM fine-tuning method and prompt learning training method, the method proposed by the present invention greatly reduces the difficulty of task learning and can effectively achieve accurate positioning and fine segmentation of building edges.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and in particular relates to a building edge segmentation method and system based on adaptive prompting of a large segmentation model. Background Art

[0002] Buildings are the main components of cities, and their identification and extraction are of great significance for urban planning, urban governance, disaster emergency response, etc. Great progress has been made in the extraction of buildings from remote sensing images based on deep learning. Unlike traditional manually designed features, convolutional neural networks can automatically extract building features from remote sensing images. The U-Net encoder-decoder architecture provides a solution for the fusion of shallow features and deep features. Based on the U-Net architecture, network structures such as multi-level parallelism, multi-path fusion, and multi-task learning have been proposed to achieve more effective multi-scale fusion. The self-attention mechanism of the Transformer network structure can capture the long-range dependencies between global features and local features, and is used to replace the convolutional network structure to achieve multi-scale feature extraction.

[0003] However, the internal features of buildings in remote sensing imagery vary significantly, and the surrounding environments are complex and diverse. The application of deep learning technology requires a large amount of masked building data. Acquiring this labeled data is not only labor-intensive and time-consuming, but also increases the difficulty of labeling due to the specialized nature and complexity of remote sensing interpretation. To alleviate the burden of sample labeling, one solution is to transfer mature feature extraction networks pre-trained on large-scale natural image datasets to remote sensing tasks to achieve better feature representation. The Segment Anything Model (SAM) image segmentation model, trained on a large-scale dataset of 11 million images and 1.1 billion masked objects, demonstrates strong generalization capabilities across a wide range of objects and images. The model can leverage cue information to achieve zero-shot learning for any segmentation task. The SAM model's superior segmentation performance makes it a promising candidate for remote sensing extraction.

[0004] Because SAM segmentation lacks category information and relies heavily on pre-designed cues such as points, boxes, or coarse-grained mask layers, despite its superior performance in interactive segmentation tasks, it remains unsuitable for direct semantic segmentation. To adapt the SAM model for specific semantic segmentation tasks, researchers have made various attempts, including fine-tuning the SAM decoder and adding a cue branch network. However, such models are difficult to train and require a high training dataset. Other researchers have employed a simple combination of the SAM model with object detection models, language models, and other algorithms, using the points, boxes, masks, and other cue features generated by these models as cue inputs to SAM to achieve automatic semantic segmentation. These approaches, which crudely combine the two models, result in a cascade of SAM segmentation results that rely on the accuracy of the cues generated by the SAM model. This results in poor controllability and coarse object edge segmentation, making them suitable for tasks that do not require fine object edge segmentation, such as street scene segmentation. Consequently, they are not suitable for building segmentation. The current challenge lies in developing a lightweight SAM semantic segmentation framework that effectively addresses the semantic gap and cue dependency issues, enabling fine building segmentation. Summary of the Invention

[0005] The present invention aims to address the existing problems in the prior art and provide a method and system for building edge segmentation based on adaptive prompting of a large segmentation model. This paper constructs a lightweight automatic semantic segmentation framework for the SAM model, solving the problems of missing SAM segmentation categories and prompt generation, and achieving accurate positioning and fine segmentation of building edges.

[0006] In order to achieve the above-mentioned object of the invention, the present invention specifically adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for segmenting building edges based on adaptive prompting of a large segmentation model, comprising the following steps:

[0008] S1. Input the image to be segmented into the pre-trained SAM large model for full segmentation to obtain the original segmentation patch set of the image to be segmented. The trained classifier identifies the semantic information of each segmentation patch in the original segmentation patch set and obtains the category of each segmentation patch.

[0009] S2. Form a building segmentation patch set from the original segmentation patch set according to the segmentation patch category, initialize the instance object with the segmentation patch with the largest area in the building segmentation patch set, search all segmentation patches that intersect with the instance object space from the original segmentation patch set by the spatial neighborhood iterative search method and form a final intersecting segmentation patch set, split the final intersecting segmentation patch set into a positive patch set and a negative patch set according to the segmentation patch category, and use the maximum patch identification method to identify the potential maximum positive segmentation patch and the potential maximum negative segmentation patch from the positive patch set and the negative patch set respectively;

[0010] S3, using the superpixel segmentation algorithm and the abnormal superpixel identification method based on K-Means clustering to adaptively generate a positive prompt point set and a negative prompt point set from the potential maximum positive segmentation patch and the potential maximum negative segmentation patch;

[0011] S4. Input the image to be segmented, the positive cue point set, and the negative cue point set into the SAM large model, obtain three segmentation results of the instance object and corresponding original segmentation confidence scores, process the original segmentation confidence scores, and use the segmentation result with the largest final segmentation confidence score as the final segmentation result;

[0012] S5. Mark all segmentation patches of the building category that intersect with the final segmentation result space of the instance object, traverse the building segmentation patch set, and if a segmentation patch in the building segmentation patch set is marked, skip it and continue to judge the next segmentation patch. If it is not marked, initialize the instance object from the unmarked segmentation patch, and obtain the final segmentation result of the instance object where the unmarked segmentation patch is located, until all segmentation patches in the building segmentation patch set are marked, integrate all final segmentation results, and obtain the complete segmentation result of the image to be segmented.

[0013] Based on the above solution, each step can be implemented in the following preferred specific manner.

[0014] As a preferred embodiment of the first aspect, in step S1, the category of the segmented patch is non-building or building.

[0015] As a preferred embodiment of the above-mentioned first aspect, in step S2, the segmentation patches in the original segmentation patch set that are classified as buildings are sorted in descending order by area size to form a building segmentation patch set; when the category of the segmentation patches in the final intersecting segmentation patch set is non-building, the segmentation patch is added to the negative patch set; when the category of the segmentation patches in the final intersecting segmentation patch set is building, the segmentation patch is added to the positive patch set.

[0016] As a preference of the first aspect above, in step S2, the specific process of obtaining the final set of intersecting segmentation patches by the spatial neighborhood iterative search method is as follows: the segmentation patch used to initialize the instance object is used as the query segmentation patch of the first iterative search, and after the intersecting segmentation patch set is initialized by the query segmentation patch of the first iterative search, other segmentation patches that intersect with the query segmentation patch of the first iterative search in space and are not in the initialized intersecting segmentation patch set are obtained in the original segmentation patch set to form a set of other segmentation patches, and the segmentation patches in the other segmentation patch set are taken as the final set of intersecting segmentation patches. Add it to the initialized intersecting segmentation patch set to obtain the updated intersecting segmentation patch set. If the other segmentation patch set is an empty set, stop the iterative search and obtain the final intersecting segmentation patch set; if the other segmentation patch set is not an empty set, use the segmentation patch marked as a building in the other segmentation patch set as the query segmentation patch for the next iterative search, and re-iterate the search with the query segmentation patch of the next iterative search and the updated intersecting segmentation patch set until the other segmentation patch set is an empty set and the final intersecting segmentation patch set is obtained.

[0017] As a preferred embodiment of the first aspect, in step S2, the specific process of identifying the potential maximum positive segmentation patch and the potential maximum negative segmentation patch by the maximum patch identification method is as follows:

[0018] S21. For a pixel in the final set of intersecting segmented patches, the ratio of the frequency of the pixel appearing in the positive patch set to the sum of the frequency of the pixel appearing in the positive patch set and the negative patch set is used as the probability that the pixel belongs to the positive class, and the ratio of the frequency of the pixel appearing in the negative patch set to the sum of the frequency of the pixel appearing in the positive patch set and the negative patch set is used as the probability that the pixel belongs to the negative class;

[0019] S22, performing regional connectivity calculation on all pixels belonging to the positive class to obtain multiple separated regions, retaining the region where the segmentation patch involved in initializing the instance object is located in all separated regions, and obtaining a positive candidate region of the instance object;

[0020] S23, taking pixels that belong to both the positive patch set and the negative patch set as confused pixels, filtering out pixels that do not meet the preset criteria from the positive candidate region based on the probability of positive pixels and the confused pixels, and obtaining a potential maximum positive segmentation patch;

[0021] S24. A negative candidate region is formed by all pixels belonging to the negative class, and pixels overlapping with the potential maximum positive segmentation patch are filtered out in the negative candidate region to obtain the potential maximum negative segmentation patch.

[0022] As a preferred embodiment of the above-mentioned first aspect, in step S23, when a pixel in the positive candidate area is a confused pixel and meets one of the following three conditions, the pixel is removed from the positive candidate area: the first condition is that the positive class probability corresponding to the pixel is less than a preset first probability threshold; the second condition is that the positive class probability corresponding to the pixel is equal to the first probability threshold and the proportion of pixels in the positive candidate area with a positive class probability less than or equal to the first probability threshold does not exceed 80% of all pixels in the positive candidate area; the third condition is that the positive class probability corresponding to the pixel is greater than the preset first probability threshold and less than the preset second probability threshold and the proportion of pixels in the positive candidate area with a positive class probability greater than the first probability threshold and less than the second probability threshold does not exceed 20% of all pixels in the positive candidate area.

[0023] As a preferred embodiment of the first aspect, the specific process of step S3 is as follows:

[0024] S3, using the superpixel segmentation algorithm and the abnormal superpixel identification method based on K-Means clustering to adaptively generate a positive prompt point set and a negative prompt point set from the potential maximum positive segmentation patch and the potential maximum negative segmentation patch;

[0025] S31, using a superpixel segmentation algorithm to segment the potential maximum positive segmentation patch and the potential maximum negative segmentation patch into internally homogeneous superpixels, thereby obtaining an original positive segmentation superpixel set consisting of positive segmentation superpixels and an original negative segmentation superpixel set consisting of negative segmentation superpixels;

[0026] S32, filtering abnormal superpixels in the original positive segmentation superpixel set and the original negative segmentation superpixel set by the abnormal superpixel identification method, and obtaining a positive segmentation superpixel set and a negative segmentation superpixel set respectively;

[0027] S33, taking the positive segmentation superpixels in the positive segmentation superpixel set as the remaining positive superpixels, calculating the center of each remaining positive superpixel, and using the point in the remaining positive superpixel closest to its center as the key point representing the remaining positive superpixel; taking the negative segmentation superpixels in the negative segmentation superpixel set as the remaining negative superpixels, calculating the center of each remaining negative superpixel, and using the point in the remaining negative superpixel closest to its center as the key point representing the remaining negative superpixel;

[0028] S34, constructing a positive hint candidate point set from the key points of the remaining positive superpixels, calculating the minimum convex polygon deformation containing all points in the positive hint candidate point set using the convex hull algorithm and using it as the first convex polygon deformation, finding the longest side of the convex hull in the first convex polygon deformation and using it as the first reference side, adding the vertices of the first reference side to the positive hint candidate point set, selecting vertices adjacent to the vertices of the first reference side and adding them one by one to the positive hint candidate point set until the specified number of hint points is met, and finally obtaining a positive hint point set;

[0029] S35. Construct a negative hint candidate point set from the key points of the remaining negative superpixels, calculate the minimum convex polygon deformation containing all points in the negative hint candidate point set through the convex hull algorithm and use it as the second convex polygon deformation, find the longest side of the convex hull in the second convex polygon deformation and use it as the second reference side, add the vertices of the second reference side to the negative hint candidate point set, select the vertices adjacent to the vertices of the second reference side and add them one by one to the negative hint candidate point set until the specified number of hint points is met, and finally obtain the negative hint point set.

[0030] As a preferred embodiment of the first aspect, in step S32, the specific process of filtering abnormal superpixels by the abnormal superpixel identification method is as follows:

[0031] S321, calculating the color histogram of the area where each positive segmentation superpixel or negative segmentation superpixel in the original positive segmentation superpixel set and the original negative segmentation superpixel set is located, and using the obtained color histogram as the feature vector of each positive segmentation superpixel or negative segmentation superpixel;

[0032] S322, performing K-Means clustering on the feature vector of each positive segmentation superpixel, clustering the positive segmentation superpixels into multiple positive segmentation categories, taking the positive segmentation category with the largest number of positive segmentation superpixels as the positive reference category, calculating the ratio of the number of positive segmentation superpixels in each positive segmentation category to the number of positive segmentation superpixels belonging to the positive reference category, taking the smallest ratio result as a first reference ratio, and if the first reference ratio is less than a preset first ratio threshold, taking the positive segmentation superpixels contained in the positive reference category corresponding to the first reference ratio as abnormal superpixels;

[0033] S323, performing K-Means clustering on the feature vectors of each positive segmentation superpixel and negative segmentation superpixel, clustering the positive segmentation superpixels and negative segmentation superpixels into multiple segmentation categories, treating the cluster containing both positive segmentation superpixels and negative segmentation superpixels as a mixed category cluster, calculating the ratio of the number of positive segmentation superpixels contained in the mixed category cluster in the original positive segmentation superpixel set, and obtaining a second reference ratio. If the second reference ratio is less than a preset second ratio threshold, treating the positive segmentation superpixels contained in the mixed category cluster as abnormal superpixels. Calculating the ratio of the number of negative segmentation superpixels contained in the mixed category cluster in the original negative segmentation superpixel set, and obtaining a third reference ratio. If the third reference ratio is less than the preset second ratio threshold, treating the negative segmentation superpixels contained in the mixed category cluster as abnormal superpixels.

[0034] S324 , removing abnormal superpixels from the original positive segmentation superpixel set to obtain a positive segmentation superpixel set, and removing abnormal superpixels from the original negative segmentation superpixel set to obtain a negative segmentation superpixel set.

[0035] As a preferred embodiment of the first aspect, in step S4, for each instance object, the specific process of obtaining the final segmentation confidence score is as follows:

[0036] S41. Taking the potential maximum positive segmentation patch after removing abnormal superpixels as a high-confidence positive patch, calculating the compactness of the high-confidence positive patch and using it as a reference compactness, and calculating the compactness of each segmentation result;

[0037] S42. Calculate the intersection-and-union (IoU) of each segmentation result with the high-confidence positive patch. If the IoU of a segmentation result is greater than a preset first IoU threshold and less than a preset second IoU threshold, multiply the original segmentation confidence score of the segmentation result by its IoU as the final segmentation confidence score corresponding to the segmentation result. Otherwise, multiply the original segmentation confidence score of the segmentation result by its IoU to obtain a first calculation result. Divide the first calculation result by the compactness of the segmentation result to obtain a second calculation result. Perform a weighted sum of the first calculation result and the second calculation result to obtain the final segmentation confidence score corresponding to the segmentation result.

[0038] In a second aspect, the present invention provides a building edge segmentation system based on adaptive prompting of a large segmentation model, comprising:

[0039] The SAM full segmentation module is used to input the image to be segmented into the pre-trained SAM large model for full segmentation, obtaining a set of original segmentation patches of the image to be segmented. The trained classifier identifies the semantic information of each segmentation patch in the set of original segmentation patches and obtains the category of each segmentation patch.

[0040] The recognition and spatial reorganization module is used to form a building segmentation patch set from the original segmentation patch set according to the segmentation patch category, initialize the instance object from the segmentation patch with the largest area in the building segmentation patch set, search all segmentation patches that intersect with the instance object space from the original segmentation patch set using a spatial neighborhood iterative search method to form a final intersecting segmentation patch set, split the final intersecting segmentation patch set into a positive patch set and a negative patch set according to the segmentation patch category, and use a maximum patch recognition method to identify the potential maximum positive segmentation patch and the potential maximum negative segmentation patch from the positive patch set and the negative patch set respectively;

[0041] A cue point generation module is used to adaptively generate a positive cue point set and a negative cue point set from potential maximum positive segmentation patches and potential maximum negative segmentation patches using a superpixel segmentation algorithm and an abnormal superpixel identification method based on K-Means clustering;

[0042] A fine re-segmentation module is used to input the image to be segmented, the positive cue point set, and the negative cue point set into the SAM large model, obtain three segmentation results of the instance object and the corresponding original segmentation confidence scores, process the original segmentation confidence scores, and take the segmentation result with the largest final segmentation confidence score as the final segmentation result;

[0043] The global result acquisition module is used to mark all segmentation patches of the building category that intersect with the final segmentation result space of the instance object, traverse the building segmentation patch set, and if a segmentation patch in the building segmentation patch set is marked, it is skipped and the next segmentation patch is judged. If it is not marked, the instance object is initialized by the unmarked segmentation patch, and the final segmentation result of the instance object where the unmarked segmentation patch is located is obtained until all segmentation patches in the building segmentation patch set are marked, and all final segmentation results are integrated to obtain the complete segmentation result of the image to be segmented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This paper provides a building edge segmentation method based on adaptive cueing using a large segmentation model. This method employs a "SAM full segmentation - object-oriented segmentation - cue generation - re-segmentation" process for automatically generating segmentation points without training. Using the original segmented patches obtained from SAM segmentation as prior information, the method combines spatial constraints and similarity metrics to generate high-quality positive and negative cue points. This method addresses the semantic loss and cue dependency issues of the SAM model in a lightweight manner, achieving accurate building recognition and fine segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the steps of the method of the present invention;

[0047] Figure 2 Schematic diagram of the spatial reorganization and instance object recognition process in the method of the present invention;

[0048] Figure 3 Schematic diagram of the process of generating positive and negative prompt points in the method of the present invention;

[0049] Figure 4 Result diagrams of the refined re-segmentation process in the method of the present invention; wherein (a), (b) and (c) are segmentation results of the original segmentation patch of the image to be segmented;

[0050] Figure 5 Schematic diagram comparing the segmentation results of different methods in the embodiments of the present invention; wherein (a) is the original segmented patch obtained after full segmentation of the SAM large model; (b) is the final segmentation result obtained by the method of the present invention;

[0051] Figure 6 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in the various embodiments of the present invention can be combined accordingly without conflicting with each other.

[0053] In the description of the present invention, it should be understood that the terms "first" and "second" are used solely for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly specifying the number of technical features being described. Therefore, features defined as "first" or "second" may explicitly or implicitly include at least one of such features.

[0054] To address the cue-dependence and semantic loss issues of the Segment Anything Model (SAM) in real-world segmentation tasks, this paper proposes a lightweight, general framework for automatic semantic segmentation using SAM models. This framework involves adaptive cueing of building edges within the SAM model. This approach utilizes a semantic segmentation pipeline consisting of full SAM segmentation, object-oriented classification, cue point generation, and refined re-segmentation, enabling rapid adaptation and refined extraction of SAM data for real-world applications.

[0055] like Figure 1 As shown, in a preferred implementation of the present invention, the building edge segmentation method based on adaptive prompting of a large segmentation model includes the following steps S1 to S5. The specific implementation process is described in detail below.

[0056] 1. Image Full Segmentation and Classification

[0057] S1. Input the image to be segmented into the pre-trained SAM large model for full segmentation to obtain the original segmentation patch set of the image to be segmented , the trained classifier identifies the semantic information of each segmentation patch in the original segmentation patch set and obtains the category of each segmentation patch ;in, Represents the first a segmented patch; Indicates the The category of the segmented patch. When the category is 0, it means the category of the segmented patch is non-building, and when the category is 1, it means the category of the segmented patch is building; Indicates the index of the segmented patch; is the number of segmented patches in the original segmented patch set.

[0058] It should be noted that in step S1 of the present invention, for the building edge segmentation task, the SAM large model is first used to fully segment the image to be segmented to obtain an original segmentation patch set consisting of segmentation patches. The segmentation patches here have different scales and may be a certain component of a building, or the entire building or a collection of multiple building components. After obtaining the original segmentation patch set, it is necessary to obtain a classifier for SAM segmentation to identify the category information of the segmentation patches obtained after the image to be segmented is input into the SAM (Segment Anything Model) large model for full segmentation. The classifier here can be a pre-trained model, including a multimodal image classification model combined with a language large model, which can be directly used for the type recognition of the above-mentioned segmentation patches; it can also be an untrained model or a migration based on a pre-trained model. Therefore, the specific structure of the classifier is not limited in the present invention. Compared with SAM fine-tuning training and general image classification tasks, the above-mentioned segmentation patch type recognition task has relatively simple requirements for the classifier and training samples. Only a small amount of training samples is needed to obtain a classifier with higher accuracy. In this embodiment, a joint classification network combining object classification and multi-scale image classification features is used as the classifier, and trained using a small number of training samples. Training samples are labeled by spatially overlaying the segmented objects with building mask images. Building and non-building classifications are automatically labeled based on the proportion of building pixels within the segmented objects. The specific structure of this joint classification network is known in the art and will not be described in detail here.

[0059] 2. Spatial Reorganization and Instance Object Recognition

[0060] S2. According to the category of the segmentation patches, a building segmentation patch set is formed from the original segmentation patch set. The instance object is initialized by the segmentation patch with the largest area in the building segmentation patch set. All segmentation patches that intersect with the instance object space are searched from the original segmentation patch set by the spatial neighborhood iterative search method to form the final intersecting segmentation patch set. According to the category of the segmentation patches, the final intersecting segmentation patch set is split into a positive patch set and a negative patch set. The maximum patch identification method is used to identify the potential maximum positive segmentation patch from the positive patch set and the negative patch set. and the potential maximum negative segmentation patch .

[0061] In step S2 of this embodiment, the building-class segments in the original segmentation patch set are sorted in descending order by area to form a building segmentation patch set. An instance object is then initialized from the segmentation patch with the largest area in this set. For the building edge segmentation task, an instance object refers to each independent, distinguishable building entity in the task. After the instance object is initialized, the spatial neighborhood iterative search method designed by the present invention is used to obtain the final set of intersecting segmentation patches.

[0062] In step S2, the specific process of obtaining the final set of intersecting segmentation patches by the spatial neighborhood iterative search method is as follows: the segmentation patch used to initialize the instance object (i.e., the segmentation patch with the largest area in the building segmentation patch set) is used as the query segmentation patch for the first iterative search, and after the intersecting segmentation patch set is initialized by the query segmentation patch for the first iterative search, other segmentation patches that intersect with the query segmentation patch for the first iterative search in space and are not in the initialized intersecting segmentation patch set are obtained from the original segmentation patch set to form another set of segmentation patches, and the other segmentation patches in the other segmentation patch set are obtained. The segmentation patches are added to the initialized intersection segmentation patch set to obtain the updated intersection segmentation patch set. If the other segmentation patch sets are empty sets, the iterative search is stopped to obtain the final intersection segmentation patch set. If the other segmentation patch sets are not empty sets, the segmentation patches marked as buildings in the other segmentation patch sets are used as the query segmentation patches for the next iterative search, and the query segmentation patches of the next iterative search and the updated intersection segmentation patch set are re-iteratively searched together until the other segmentation patch sets are empty sets to obtain the final intersection segmentation patch set.

[0063] In step S2 of this embodiment, if the segmented patch in the final set of intersecting segmented patches is classified as non-building, the segmented patch is added to the negative patch set. If the segmented patch in the final set of intersecting segmented patches is classified as building, the segmented patch is added to the positive patch set. Similarly, the final set of intersecting segmented patches is split into a positive patch set and a negative patch set.

[0064] In step S2 of this embodiment, the maximum patch identification method is used to identify the potential maximum positive segmentation patch from the positive patch set and the potential maximum negative segmentation patch from the negative patch set. The specific process is as follows:

[0065] S21. For a pixel in the final set of intersecting segmented patches, the probability that the pixel belongs to the positive class is calculated by the frequency of the pixel appearing in the positive patch set, and the probability that the pixel belongs to the negative class is calculated by the frequency of the pixel appearing in the negative patch set.

[0066] In this embodiment, the building category is taken as the positive category and the non-building category is taken as the negative category. The probability that the pixel is a positive category is taken as the ratio of the frequency of the pixel appearing in the positive patch set to the sum of the frequencies of the pixel appearing in the positive patch set and the negative patch set. The probability that the pixel is a negative category is taken as the ratio of the frequency of the pixel appearing in the negative patch set to the sum of the frequencies of the pixel appearing in the positive patch set and the negative patch set.

[0067] Furthermore, the specific calculation process of the probability of a pixel being positive or negative is as follows:

[0068]

[0069]

[0070] Where: 、 They represent the probability that the pixel is positive or negative, respectively.

[0071] Identification of potential maximum positive segmentation patches:

[0072] S22, for all the positive classes (i.e. ) pixels to calculate the regional connectivity, and obtain multiple separated regions. In all separated regions, the region where the segmentation patch participating in initializing the instance object is located is retained, and the positive candidate region of the instance object is obtained.

[0073] S23. Pixels belonging to both the positive patch set and the negative patch set are regarded as confusing pixels, and pixels that do not meet the preset criteria are filtered out from the positive candidate region to obtain the potential maximum positive segmentation patch.

[0074] Identification of potential maximum negative segmentation patches:

[0075] S24, by all the negative class (i.e. ) constitutes a negative candidate region, and the pixels overlapping with the potential maximum positive segmentation patch are filtered out in the negative candidate region to obtain the potential maximum negative segmentation patch.

[0076] Through the above analysis, we know that the segmented patches obtained in step S1 are part of the instance object, some contain the instance object, and some are adjacent patches of the instance object. The segmented patches may overlap with each other. Therefore, it is necessary to spatially reorganize and identify the segmented patches obtained. The specific process of spatial reorganization and identification is as follows: Figure 2 shown.

[0077] In step S23 of the present invention, when a pixel in the positive candidate region is a confused pixel and satisfies one of the following three conditions, the pixel is removed from the positive candidate region:

[0078] The first condition is that the positive class probability corresponding to the pixel is less than a preset first probability threshold;

[0079] The second condition is that the positive probability corresponding to the pixel is equal to the first probability threshold and the proportion of pixels in the positive candidate area whose positive probability is less than or equal to the first probability threshold does not exceed 80% of all pixels in the positive candidate area;

[0080] The third condition is that the positive class probability corresponding to the pixel is greater than the preset first probability threshold and less than the preset second probability threshold, and the proportion of pixels in the positive candidate area whose positive class probability is greater than the first probability threshold and less than the second probability threshold does not exceed 20% of all pixels in the positive candidate area.

[0081] In step S23 of this embodiment, the first probability threshold and the second probability threshold are recommended to be 0.5 and 0.66. The threshold values ​​given here are only a preferred method and can of course be adjusted by those skilled in the art according to actual needs. The pixel ratio determination in the second and third conditions is to avoid the situation where the pixels to be filtered substantially cover the positive candidate area.

[0082] 3. Automatic generation of prompt points

[0083] S3. Use the superpixel segmentation algorithm and the abnormal superpixel identification method based on K-Means clustering to adaptively generate positive prompt point sets and negative prompt point sets from the potential maximum positive segmentation patches and the potential maximum negative segmentation patches.

[0084] It should be noted that the specific process of step S3 of the present invention is as follows:

[0085] S31. Use the superpixel segmentation algorithm to segment the potential maximum positive segmentation patch and the potential maximum negative segmentation patch into internally homogeneous superpixels, and obtain the original positive segmentation superpixel set composed of positive segmentation superpixels and the original negative segmentation superpixel set composed of negative segmentation superpixels.

[0086] In step S31 of this embodiment, the superpixel segmentation algorithm adopts the SLIC (Simple Linear Iterative Clustering) algorithm, and the number of positive and negative cue points is specified as 4. After segmentation, the superpixels in the potential maximum positive segmentation patch can be regarded as positive segmentation superpixels, and the superpixels in the potential maximum negative segmentation patch can be regarded as negative segmentation superpixels, thereby generating the original positive and negative segmentation superpixel sets.

[0087] S32. Filter abnormal superpixels in the original positive segmentation superpixel set and the original negative segmentation superpixel set using an abnormal superpixel identification method based on K-Means clustering, and obtain a positive segmentation superpixel set and a negative segmentation superpixel set respectively.

[0088] In step S32 of this embodiment, the specific process of filtering abnormal superpixels using the above abnormal superpixel identification method is as follows:

[0089] S321. Calculate the color histogram of the area where each positive segmentation superpixel or negative segmentation superpixel is located in the original positive segmentation superpixel set and the original negative segmentation superpixel set, and use the obtained color histogram as the feature vector of each positive segmentation superpixel or negative segmentation superpixel.

[0090] S322. Perform K-Means clustering on the feature vector of each positive segmentation superpixel, cluster the positive segmentation superpixels into multiple positive segmentation categories, take the positive segmentation category with the largest number of positive segmentation superpixels as the positive reference category, calculate the ratio of the number of positive segmentation superpixels in each positive segmentation category to the number of positive segmentation superpixels belonging to the positive reference category (the purpose is to find the positive segmentation category with a relatively small number of superpixels), take the smallest ratio result as the first reference ratio, and if the first reference ratio is less than the preset first ratio threshold, then the positive segmentation superpixels contained in the positive reference category corresponding to the first reference ratio are regarded as abnormal superpixels.

[0091] In this embodiment, the above S322 process is briefly described by taking five positive segmentation categories as an example. Assume that the first positive segmentation category contains 10 positive segmentation superpixels, the second positive segmentation category contains 30 positive segmentation superpixels, the third positive segmentation category contains 50 positive segmentation superpixels, the fourth positive segmentation category contains 40 positive segmentation superpixels, and the fifth positive segmentation category contains 20 positive segmentation superpixels. The positive segmentation category with the largest number of positive segmentation superpixels is the third positive segmentation category, which is then used as the positive reference category. Then, for each positive segmentation category, the corresponding ratio is calculated: the ratio of the first positive segmentation category is 10 / 50, the ratio of the second positive segmentation category is 30 / 50, the ratio of the third positive segmentation category is 5050, the ratio of the fourth positive segmentation category is 40 / 50, and the ratio of the fifth positive segmentation category is 20 / 50. It can be seen that the ratio of the first positive segmentation category is the smallest, so it is used as the first reference ratio. Then compare the first reference ratio with the above-mentioned first ratio threshold. If the first reference ratio is less than the first ratio threshold, it is considered that the number of positive segmentation superpixels in the first positive segmentation category is relatively small, so these positive segmentation superpixels are regarded as abnormal superpixels, and in the subsequent process, the abnormal superpixels are removed from the original positive segmentation superpixel set.

[0092] S323. Perform K-Means clustering on the feature vectors of each positive segmentation superpixel and negative segmentation superpixel, cluster the positive segmentation superpixels and negative segmentation superpixels into multiple segmentation categories, and regard the cluster containing both positive segmentation superpixels and negative segmentation superpixels as a mixed category cluster. Calculate the ratio of the number of positive segmentation superpixels contained in the mixed category cluster in the original positive segmentation superpixel set to obtain a second reference ratio. If the second reference ratio is less than a preset second ratio threshold, regard the positive segmentation superpixels contained in the mixed category cluster as abnormal superpixels. Calculate the ratio of the number of negative segmentation superpixels contained in the mixed category cluster in the original negative segmentation superpixel set to obtain a third reference ratio. If the third reference ratio is less than the preset second ratio threshold, regard the negative segmentation superpixels contained in the mixed category cluster as abnormal superpixels.

[0093] S324 , removing abnormal superpixels from the original positive segmentation superpixel set to obtain a positive segmentation superpixel set, and removing abnormal superpixels from the original negative segmentation superpixel set to obtain a negative segmentation superpixel set.

[0094] In steps S322 and S323 of the present invention, the first ratio threshold is typically less than 0.5, and the second ratio threshold is greater than 0.7. The threshold values ​​provided here are merely a preferred approach and can, of course, be adjusted by those skilled in the art based on actual needs. Therefore, the present invention does not impose any limitations thereon. In steps S322 and S323 of this embodiment, the first ratio threshold is 0.3, and the second ratio threshold is 0.85.

[0095] In steps S322 and S323 of this embodiment, when performing K-Means clustering on the feature vectors, the number of clusters is first set to 2 to 10, and clustering is performed separately. The silhouette coefficient under each number of clusters is calculated, and the clustering model with the highest silhouette coefficient is selected to obtain the optimal number of clusters. For each feature vector of a positive segmentation superpixel, the corresponding optimal number of clusters is N, and the positive segmentation superpixels are clustered into N positive segmentation categories in total. For each feature vector of a positive and negative segmentation superpixel, the corresponding optimal number of clusters is M, and the positive and negative segmentation superpixels are clustered into M segmentation categories in total.

[0096] S33. The positive segmentation superpixels in the positive segmentation superpixel set are taken as the remaining positive superpixels, the center of each remaining positive superpixel is calculated, and the point inside the remaining positive superpixel that is closest to its center is used as the key point representing the remaining positive superpixel. The negative segmentation superpixels in the negative segmentation superpixel set are taken as the remaining negative superpixels, the center of each remaining negative superpixel is calculated, and the point inside the remaining negative superpixel that is closest to its center is used as the key point representing the remaining negative superpixel.

[0097] S34. Construct a positive hint candidate point set from the key points of the remaining positive superpixels, calculate the minimum convex polygon deformation containing all points in the positive hint candidate point set through the convex hull algorithm and use it as the first convex polygon deformation, find the longest side of the convex hull in the first convex polygon deformation and use it as the first reference side, add the vertices of the first reference side to the positive hint candidate point set, select the vertices adjacent to the vertices of the first reference side and add them to the positive hint candidate point set one by one until the specified number of hint points is met, and finally obtain the positive hint point set.

[0098] S35. Construct a negative hint candidate point set from the key points of the remaining negative superpixels, calculate the minimum convex polygon deformation containing all points in the negative hint candidate point set through the convex hull algorithm and use it as the second convex polygon deformation, find the longest side of the convex hull in the second convex polygon deformation and use it as the second reference side, add the vertices of the second reference side to the negative hint candidate point set, select the vertices adjacent to the vertices of the second reference side and add them one by one to the negative hint candidate point set until the specified number of hint points is met, and finally obtain the negative hint point set.

[0099] It should be noted that in steps S34~S35 of the present invention, the positive prompt points and negative prompt points are not obtained by deep network training, nor are they obtained by relying on human interaction. Instead, they are obtained by combining a classifier with the positioning segmentation area and using similarity measurement to extract the corresponding prompt information, thereby reducing the difficulty of prompt learning and realizing automatic extraction of prompt points based on the image distribution characteristics of the target instance.

[0100] In the embodiment of the present invention, Figure 3 As shown in the figure, the centers of the positive segmentation superpixels are extracted from the positive segmentation superpixel set as the positive prompt candidate point set, and the centers of the negative segmentation superpixels are extracted from the negative segmentation superpixel set as the negative prompt candidate point set. The convex hull algorithm is used to obtain the positive and negative prompt point sets from the positive and negative prompt candidate point sets, which meet the specified number and can cover the spatial range of the candidate point sets of the positive and negative prompt points to the greatest extent.

[0101] 4. Refined Re-segmentation

[0102] S4. Input the image to be segmented, the positive cue point set and the negative cue point set into the SAM large model together to obtain three segmentation results of the instance object and the corresponding original segmentation confidence scores, process the original segmentation confidence scores, and take the segmentation result with the largest final segmentation confidence score as the final segmentation result.

[0103] It should be noted that in the refined re-segmentation process of step S4 of the present invention, by providing positive and negative prompt information for the instance object, a more clearly positioned segmentation is achieved. The SAM large model segmentation obtains three segmentation results of the instance object and their corresponding original segmentation confidence scores, which are respectively for the region, component, and whole of the instance object. Each segmentation result is presented in the form of a mask. Taking into account the segmentation confidence, segmentation morphology, and regularity, the original segmentation confidence scores are processed and recalculated to obtain the final segmentation confidence scores corresponding to each segmentation result. Finally, the segmentation result with the highest final segmentation confidence score is selected as the final segmentation result of the instance object.

[0104] In the present invention, for each instance object, the specific process of obtaining the above-mentioned final segmentation confidence score is as follows:

[0105] S41. The potential maximum positive segmentation patch after removing abnormal superpixels is used as a high-confidence positive patch, the compactness of the high-confidence positive patch is calculated and used as a reference compactness, and the compactness of each segmentation result is calculated.

[0106] In step S41 of the present invention, for the instance object, the pixels in the high confidence positive patch are highly credible and can be used to constrain the segmentation result. The compactness calculation method belongs to the prior art. In this embodiment, taking the high confidence positive patch as an example, its compactness The specific calculation method is as follows:

[0107]

[0108] Where, represents a connected region in a high-confidence positive patch; represents the perimeter of a connected region in a high-confidence positive patch; Represents the area of ​​a connected region in a high-confidence positive patch; Represents the number of connected regions in a high-confidence positive patch.

[0109] S42. Calculate the intersection-and-union (IoU) of each segmentation result with the high-confidence positive patch. If the IoU of a segmentation result is greater than a preset first IoU threshold and less than a preset second IoU threshold, multiply the original segmentation confidence score of the segmentation result by its IoU as the final segmentation confidence score corresponding to the segmentation result. Otherwise, multiply the original segmentation confidence score of the segmentation result by its IoU to obtain a first calculation result. Divide the first calculation result by the compactness of the segmentation result to obtain a second calculation result. Perform a weighted sum of the first calculation result and the second calculation result to obtain the final segmentation confidence score corresponding to the segmentation result.

[0110] In step S42 of the present invention, the process of calculating the intersection-over-union ratio belongs to the prior art. Taking the segmentation result as an example, the calculation method of the intersection-over-union ratio with the high-confidence positive patch is expressed as:

[0111]

[0112] Where, Indicates the The intersection-over-union ratio of the segmentation results and the high-confidence positive patches; Indicates high confidence positive patches; Indicates taking the intersection; Indicates taking the union of sets.

[0113] In step S42 of the present invention, after completing the above-mentioned IoU calculation, the IoU of each segmentation result is determined to be equal to the preset first IoU threshold. and the second intersection-over-union threshold The size relationship between: If If the intersection-over-union ratio of the segmentation results is greater than the first intersection-over-union ratio threshold and less than the second intersection-over-union ratio threshold, the original segmentation confidence score of the segmentation result is Multiply it by its intersection and union ratio as the final segmentation confidence score corresponding to the segmentation result Otherwise, the After multiplying the original segmentation confidence score of the segmentation result by its intersection and union ratio, the first calculation result (i.e. ), the first calculation result and the The compactness of the segmentation results After division, the second calculation result is obtained (i.e. ), the first calculation result and the second calculation result are weighted and summed as the final segmentation confidence score corresponding to the segmentation result. The final segmentation confidence score of the segmentation results can be expressed as:

[0114]

[0115] Where, Represents the weight hyperparameter. This parameter, like the first IoU threshold and the second IoU threshold, can be selected by those skilled in the art according to actual needs, so it is not limited in the present invention. In this embodiment, the first IoU threshold is , the second intersection-over-union threshold is , the value of the weight hyperparameter is .

[0116] like Figure 4 , which shows the result of the refined re-segmentation process in an embodiment of the present invention. Figure 4(a) in Figure 4 (b) and Figure 4 (c) in the figure are all segmentation results of the original segmentation patches of the image to be segmented. Figure 4 (a) in the figure is the first segmentation result. Figure 4 (b) and Figure 4 (c) in the figure serves as the second and third segmentation results, respectively. The original segmentation confidence scores of the first, second, and third segmentation results are 0.9474, 0.9600, and 0.9413, respectively. After the above processing, the final segmentation confidence scores of the first, second, and third segmentation results are 0.8921, 0.7985, and 0.4923, respectively. For traditional edge segmentation methods based on the large SAM model, the second segmentation result has the highest original segmentation confidence score and is directly used as the final segmentation result. The method of the present invention obtains a new segmentation confidence score calculation method based on the original segmentation confidence score output by the SAM large model and some morphological calculations. After the final segmentation confidence score is recalculated by the method of the present invention, the final segmentation confidence score of the first segmentation result is significantly higher than that of the second segmentation result. It can be seen that when the edge segmentation of the building is directly performed using the general SAM large model, the result is not necessarily accurate. After the improvement of the present invention, it is more suitable for building extraction. Therefore, the method of the present invention can screen out the most suitable segmentation result by the final segmentation confidence score in the subsequent processing process to achieve more accurate edge segmentation.

[0117] 5. Full-image fine semantic segmentation

[0118] S5. Mark all segmentation patches of the building category that intersect with the final segmentation result space of the instance object, traverse the building segmentation patch set, and if a segmentation patch in the building segmentation patch set is marked, skip it and continue to judge the next segmentation patch. If it is not marked, initialize the instance object from the unmarked segmentation patch, and obtain the final segmentation result of the instance object where the unmarked segmentation patch is located, until all segmentation patches in the building segmentation patch set are marked, integrate all final segmentation results, and obtain the complete segmentation result of the image to be segmented.

[0119] like Figure 5 The following are the segmentation results obtained by using different methods in the embodiments of the present invention. Figure 5 (a) is the original segmentation patch obtained by directly using the SAM large model for full segmentation. Figure 5 (b) in FIG. 5 is the final segmentation result obtained by the method of the present invention. Figure 5 In (a), there are over-mentioned cases in the selected areas 1 and 2, and there are over-mentioned cases in the selected area 3. Figure 5In (b), the number of over-detections in areas 1 and 2, as well as the number of under-detections in area 3, has been reduced. This demonstrates that the proposed method of hinting and re-segmentation significantly improves segmentation accuracy. Over-detection refers to the inclusion of non-building pixels as buildings, while under-detection refers to the presence of some building pixels that remain undetected.

[0120] It should also be noted that the building edge segmentation method based on adaptive prompting of the segmentation large model in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a building edge segmentation system based on adaptive prompting of the segmentation large model corresponding to the building edge segmentation method based on adaptive prompting of the segmentation large model provided in the above embodiment, such as Figure 6 As shown, it includes:

[0121] The SAM full segmentation module is used to input the image to be segmented into the pre-trained SAM large model for full segmentation, obtaining a set of original segmentation patches of the image to be segmented. The trained classifier identifies the semantic information of each segmentation patch in the set of original segmentation patches and obtains the category of each segmentation patch.

[0122] The recognition and spatial reorganization module is used to form a building segmentation patch set from the original segmentation patch set according to the segmentation patch category, initialize the instance object from the segmentation patch with the largest area in the building segmentation patch set, search all segmentation patches that intersect with the instance object space from the original segmentation patch set using a spatial neighborhood iterative search method to form a final intersecting segmentation patch set, split the final intersecting segmentation patch set into a positive patch set and a negative patch set according to the segmentation patch category, and use a maximum patch recognition method to identify the potential maximum positive segmentation patch and the potential maximum negative segmentation patch from the positive patch set and the negative patch set respectively;

[0123] A cue point generation module is used to adaptively generate a positive cue point set and a negative cue point set from potential maximum positive segmentation patches and potential maximum negative segmentation patches using a superpixel segmentation algorithm and an abnormal superpixel identification method based on K-Means clustering;

[0124] A fine re-segmentation module is used to input the image to be segmented, the positive cue point set, and the negative cue point set into the SAM large model, obtain three segmentation results of the instance object and the corresponding original segmentation confidence scores, process the original segmentation confidence scores, and take the segmentation result with the largest final segmentation confidence score as the final segmentation result;

[0125] The global result acquisition module is used to mark all segmentation patches of the building category that intersect with the final segmentation result space of the instance object, traverse the building segmentation patch set, and if a segmentation patch in the building segmentation patch set is marked, it is skipped and the next segmentation patch is judged. If it is not marked, the instance object is initialized by the unmarked segmentation patch, and the final segmentation result of the instance object where the unmarked segmentation patch is located is obtained until all segmentation patches in the building segmentation patch set are marked, and all final segmentation results are integrated to obtain the complete segmentation result of the image to be segmented.

[0126] It should also be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division. In actual implementation, there may be other division methods, for example, multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0127] The embodiment described above is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Persons skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent substitution or equivalent transformation falls within the scope of protection of the present invention.

Claims

1. A building edge segmentation method based on adaptive prompting of a large segmentation model, characterized in that: The following steps are involved: S1. Input the image to be segmented into the pre-trained SAM large model for full segmentation to obtain the original segmentation patch set of the image to be segmented. The trained classifier identifies the semantic information of each segmentation patch in the original segmentation patch set and obtains the category of each segmentation patch. S2. Form a building segmentation patch set from the original segmentation patch set according to the segmentation patch category, initialize the instance object with the segmentation patch with the largest area in the building segmentation patch set, search all segmentation patches that intersect with the instance object space from the original segmentation patch set by the spatial neighborhood iterative search method and form a final intersecting segmentation patch set, split the final intersecting segmentation patch set into a positive patch set and a negative patch set according to the segmentation patch category, and use the maximum patch identification method to identify the potential maximum positive segmentation patch and the potential maximum negative segmentation patch from the positive patch set and the negative patch set respectively; S3, using the superpixel segmentation algorithm and the abnormal superpixel identification method based on K-Means clustering to adaptively generate a positive prompt point set and a negative prompt point set from the potential maximum positive segmentation patch and the potential maximum negative segmentation patch; S4. Input the image to be segmented, the positive cue point set, and the negative cue point set into the SAM large model, obtain three segmentation results of the instance object and corresponding original segmentation confidence scores, process the original segmentation confidence scores, and use the segmentation result with the largest final segmentation confidence score as the final segmentation result; S5. Mark all segmentation patches of the building category that intersect with the final segmentation result space of the instance object, traverse the building segmentation patch set, and if a segmentation patch in the building segmentation patch set is marked, skip it and continue to judge the next segmentation patch. If it is not marked, initialize the instance object from the unmarked segmentation patch, and obtain the final segmentation result of the instance object where the unmarked segmentation patch is located, until all segmentation patches in the building segmentation patch set are marked, integrate all final segmentation results, and obtain the complete segmentation result of the image to be segmented; In step S2, the specific process of identifying the potential maximum positive segmentation patch and the potential maximum negative segmentation patch by the maximum patch identification method is as follows: S21. For a pixel in the final set of intersecting segmented patches, the ratio of the frequency of the pixel appearing in the positive patch set to the sum of the frequency of the pixel appearing in the positive patch set and the negative patch set is used as the probability that the pixel belongs to the positive class, and the ratio of the frequency of the pixel appearing in the negative patch set to the sum of the frequency of the pixel appearing in the positive patch set and the negative patch set is used as the probability that the pixel belongs to the negative class; S22, performing regional connectivity calculation on all pixels belonging to the positive class to obtain multiple separated regions, retaining the region where the segmentation patch involved in initializing the instance object is located in all separated regions, and obtaining a positive candidate region of the instance object; S23, taking pixels that belong to both the positive patch set and the negative patch set as confused pixels, filtering out pixels that do not meet the preset criteria from the positive candidate region based on the probability of positive pixels and the confused pixels, and obtaining a potential maximum positive segmentation patch; S24, forming a negative candidate region from all pixels belonging to the negative class, filtering out pixels overlapping with the potential maximum positive segmentation patch in the negative candidate region, and obtaining the potential maximum negative segmentation patch; In step S4, for each instance object, the specific process of obtaining the final segmentation confidence score is as follows: S41. Taking the potential maximum positive segmentation patch after removing abnormal superpixels as a high-confidence positive patch, calculating the compactness of the high-confidence positive patch and using it as a reference compactness, and calculating the compactness of each segmentation result; S42. Calculate the intersection-and-union (IoU) of each segmentation result with the high-confidence positive patch. If the IoU of a segmentation result is greater than a preset first IoU threshold and less than a preset second IoU threshold, multiply the original segmentation confidence score of the segmentation result by its IoU as the final segmentation confidence score corresponding to the segmentation result. Otherwise, multiply the original segmentation confidence score of the segmentation result by its IoU to obtain a first calculation result. Divide the first calculation result by the compactness of the segmentation result to obtain a second calculation result. Perform a weighted sum of the first calculation result and the second calculation result to obtain the final segmentation confidence score corresponding to the segmentation result.

2. The building edge segmentation method based on adaptive prompting of a large segmentation model according to claim 1, characterized in that: In step S1, the patch is segmented into a non-building or a building category.

3. The building edge segmentation method based on adaptive prompting of a large segmentation model according to claim 1, characterized in that: In step S2, the segmented patches of the original segmented patch set that are classified as buildings are sorted in descending order by area size to form a building segmented patch set; when the category of the segmented patch in the final intersecting segmented patch set is non-building, the segmented patch is added to the negative patch set; when the category of the segmented patch in the final intersecting segmented patch set is building, the segmented patch is added to the positive patch set.

4. The building edge segmentation method based on adaptive prompting of a large segmentation model according to claim 1, characterized in that: In step S2, the specific process of obtaining the final set of intersecting segmentation patches by the spatial neighborhood iterative search method is as follows: the segmentation patch used to initialize the instance object is used as the query segmentation patch for the first iterative search, and after the intersecting segmentation patch set is initialized by the query segmentation patch for the first iterative search, other segmentation patches that intersect with the query segmentation patch for the first iterative search in the original segmentation patch set and are not in the initialized intersecting segmentation patch set are obtained and formed into other segmentation patch sets, and the segmentation patches in the other segmentation patch sets are added to the initialization The updated intersecting segmentation patch set is obtained from the intersecting segmentation patch set after the search. If the other segmentation patch set is an empty set, the iterative search is stopped to obtain the final intersecting segmentation patch set. If the other segmentation patch set is not an empty set, the segmentation patch marked as a building in the other segmentation patch set is used as the query segmentation patch for the next iterative search, and the query segmentation patch of the next iterative search and the updated intersecting segmentation patch set are re-iteratively searched together until the other segmentation patch set is an empty set to obtain the final intersecting segmentation patch set.

5. The building edge segmentation method based on adaptive prompting of a large segmentation model according to claim 1, characterized in that: In step S23, when a pixel in the positive candidate area is a confused pixel and meets one of the following three conditions, the pixel is removed from the positive candidate area: the first condition is that the positive class probability corresponding to the pixel is less than a preset first probability threshold; the second condition is that the positive class probability corresponding to the pixel is equal to the first probability threshold and the proportion of pixels in the positive candidate area with a positive class probability less than or equal to the first probability threshold does not exceed 80% of all pixels in the positive candidate area; the third condition is that the positive class probability corresponding to the pixel is greater than the preset first probability threshold and less than the preset second probability threshold and the proportion of pixels in the positive candidate area with a positive class probability greater than the first probability threshold and less than the second probability threshold does not exceed 20% of all pixels in the positive candidate area.

6. The building edge segmentation method based on adaptive prompting of a large segmentation model according to claim 1, characterized in that: The specific process of step S3 is as follows: S3, using the superpixel segmentation algorithm and the abnormal superpixel identification method based on K-Means clustering to adaptively generate a positive prompt point set and a negative prompt point set from the potential maximum positive segmentation patch and the potential maximum negative segmentation patch; S31, using a superpixel segmentation algorithm to segment the potential maximum positive segmentation patch and the potential maximum negative segmentation patch into internally homogeneous superpixels, thereby obtaining an original positive segmentation superpixel set consisting of positive segmentation superpixels and an original negative segmentation superpixel set consisting of negative segmentation superpixels; S32, filtering abnormal superpixels in the original positive segmentation superpixel set and the original negative segmentation superpixel set by the abnormal superpixel identification method, and obtaining a positive segmentation superpixel set and a negative segmentation superpixel set respectively; S33, taking the positive segmentation superpixels in the positive segmentation superpixel set as the remaining positive superpixels, calculating the center of each remaining positive superpixel, and using the point in the remaining positive superpixel closest to its center as the key point representing the remaining positive superpixel; taking the negative segmentation superpixels in the negative segmentation superpixel set as the remaining negative superpixels, calculating the center of each remaining negative superpixel, and using the point in the remaining negative superpixel closest to its center as the key point representing the remaining negative superpixel; S34, constructing a positive hint candidate point set from the key points of the remaining positive superpixels, calculating the minimum convex polygon deformation containing all points in the positive hint candidate point set using the convex hull algorithm and using it as the first convex polygon deformation, finding the longest side of the convex hull in the first convex polygon deformation and using it as the first reference side, adding the vertices of the first reference side to the positive hint candidate point set, selecting vertices adjacent to the vertices of the first reference side and adding them one by one to the positive hint candidate point set until the specified number of hint points is met, and finally obtaining a positive hint point set; S35. Construct a negative hint candidate point set from the key points of the remaining negative superpixels, calculate the minimum convex polygon deformation containing all points in the negative hint candidate point set through the convex hull algorithm and use it as the second convex polygon deformation, find the longest side of the convex hull in the second convex polygon deformation and use it as the second reference side, add the vertices of the second reference side to the negative hint candidate point set, select the vertices adjacent to the vertices of the second reference side and add them one by one to the negative hint candidate point set until the specified number of hint points is met, and finally obtain the negative hint point set.

7. The building edge segmentation method based on adaptive prompting of a large segmentation model according to claim 6, characterized in that: In step S32, the specific process of filtering abnormal superpixels by the abnormal superpixel identification method is as follows: S321, calculating the color histogram of the area where each positive segmentation superpixel or negative segmentation superpixel in the original positive segmentation superpixel set and the original negative segmentation superpixel set is located, and using the obtained color histogram as the feature vector of each positive segmentation superpixel or negative segmentation superpixel; S322, performing K-Means clustering on the feature vector of each positive segmentation superpixel, clustering the positive segmentation superpixels into multiple positive segmentation categories, taking the positive segmentation category with the largest number of positive segmentation superpixels as the positive reference category, calculating the ratio of the number of positive segmentation superpixels in each positive segmentation category to the number of positive segmentation superpixels belonging to the positive reference category, taking the smallest ratio result as a first reference ratio, and if the first reference ratio is less than a preset first ratio threshold, taking the positive segmentation superpixels contained in the positive reference category corresponding to the first reference ratio as abnormal superpixels; S323, performing K-Means clustering on the feature vectors of each positive segmentation superpixel and negative segmentation superpixel, clustering the positive segmentation superpixels and negative segmentation superpixels into multiple segmentation categories, treating the cluster containing both positive segmentation superpixels and negative segmentation superpixels as a mixed category cluster, calculating the ratio of the number of positive segmentation superpixels contained in the mixed category cluster in the original positive segmentation superpixel set, and obtaining a second reference ratio. If the second reference ratio is less than a preset second ratio threshold, treating the positive segmentation superpixels contained in the mixed category cluster as abnormal superpixels. Calculating the ratio of the number of negative segmentation superpixels contained in the mixed category cluster in the original negative segmentation superpixel set, and obtaining a third reference ratio. If the third reference ratio is less than the preset second ratio threshold, treating the negative segmentation superpixels contained in the mixed category cluster as abnormal superpixels. S324 , removing abnormal superpixels from the original positive segmentation superpixel set to obtain a positive segmentation superpixel set, and removing abnormal superpixels from the original negative segmentation superpixel set to obtain a negative segmentation superpixel set.

8. A building edge segmentation system based on adaptive prompting of a large segmentation model, characterized in that: include: The SAM full segmentation module is used to input the image to be segmented into the pre-trained SAM large model for full segmentation, obtaining a set of original segmentation patches of the image to be segmented. The trained classifier identifies the semantic information of each segmentation patch in the set of original segmentation patches and obtains the category of each segmentation patch. The recognition and spatial reorganization module is used to form a building segmentation patch set from the original segmentation patch set according to the segmentation patch category, initialize the instance object from the segmentation patch with the largest area in the building segmentation patch set, search all segmentation patches that intersect with the instance object space from the original segmentation patch set using a spatial neighborhood iterative search method to form a final intersecting segmentation patch set, split the final intersecting segmentation patch set into a positive patch set and a negative patch set according to the segmentation patch category, and use a maximum patch recognition method to identify the potential maximum positive segmentation patch and the potential maximum negative segmentation patch from the positive patch set and the negative patch set respectively; A cue point generation module is used to adaptively generate a positive cue point set and a negative cue point set from potential maximum positive segmentation patches and potential maximum negative segmentation patches using a superpixel segmentation algorithm and an abnormal superpixel identification method based on K-Means clustering; A fine re-segmentation module is used to input the image to be segmented, the positive cue point set, and the negative cue point set into the SAM large model, obtain three segmentation results of the instance object and the corresponding original segmentation confidence scores, process the original segmentation confidence scores, and take the segmentation result with the largest final segmentation confidence score as the final segmentation result; The global result acquisition module is used to mark all segmentation patches of the building category that intersect with the final segmentation result space of the instance object, traverse the building segmentation patch set, and if a segmentation patch in the building segmentation patch set is marked, it is skipped and the next segmentation patch is judged. If it is not marked, the instance object is initialized from the unmarked segmentation patch, and the final segmentation result of the instance object where the unmarked segmentation patch is located is obtained. This module continues until all segmentation patches in the building segmentation patch set are marked, and all final segmentation results are integrated to obtain the complete segmentation result of the image to be segmented. In the recognition and spatial reorganization module, the specific process of identifying the potential maximum positive segmentation patch and the potential maximum negative segmentation patch by the maximum patch recognition method is as follows: S21. For a pixel in the final set of intersecting segmented patches, the ratio of the frequency of the pixel appearing in the positive patch set to the sum of the frequency of the pixel appearing in the positive patch set and the negative patch set is used as the probability that the pixel belongs to the positive class, and the ratio of the frequency of the pixel appearing in the negative patch set to the sum of the frequency of the pixel appearing in the positive patch set and the negative patch set is used as the probability that the pixel belongs to the negative class; S22, performing regional connectivity calculation on all pixels belonging to the positive class to obtain multiple separated regions, retaining the region where the segmentation patch involved in initializing the instance object is located in all separated regions, and obtaining a positive candidate region of the instance object; S23, taking pixels that belong to both the positive patch set and the negative patch set as confused pixels, filtering out pixels that do not meet the preset criteria from the positive candidate region based on the probability of positive pixels and the confused pixels, and obtaining a potential maximum positive segmentation patch; S24, forming a negative candidate region from all pixels belonging to the negative class, filtering out pixels overlapping with the potential maximum positive segmentation patch in the negative candidate region, and obtaining the potential maximum negative segmentation patch; In the fine re-segmentation module, for each instance object, the specific process of obtaining the final segmentation confidence score is as follows: S41. Taking the potential maximum positive segmentation patch after removing abnormal superpixels as a high-confidence positive patch, calculating the compactness of the high-confidence positive patch and using it as a reference compactness, and calculating the compactness of each segmentation result; S42. Calculate the intersection-and-union (IoU) of each segmentation result with the high-confidence positive patch. If the IoU of a segmentation result is greater than a preset first IoU threshold and less than a preset second IoU threshold, multiply the original segmentation confidence score of the segmentation result by its IoU as the final segmentation confidence score corresponding to the segmentation result. Otherwise, multiply the original segmentation confidence score of the segmentation result by its IoU to obtain a first calculation result. Divide the first calculation result by the compactness of the segmentation result to obtain a second calculation result. Perform a weighted sum of the first calculation result and the second calculation result to obtain the final segmentation confidence score corresponding to the segmentation result.

Images