Man-machine cooperation remote sensing image labeling method based on SAM model

Abstract

The invention relates to the field of image processing, and particularly discloses a man-machine collaborative remote sensing image labeling method based on an SAM model, which does not directly input sparse prompts of a user into the SAM, but firstly utilizes a pre-trained remote sensing semantic segmentation model to pre-compute an image to generate a semantic graph rich in surface feature category information. On the basis, single-point interaction of the user is combined with the semantic graph, dynamic semantic prompt enhancement is carried out, namely, a user intention area is automatically recognized, a dense enhancement prompt set with definite positive and negative attributes is generated, the enhancement prompt set can convert the fuzzy intention of the user into a guide signal capable of being accurately understood by a machine, and the user intention is automatically recognized. According to the method, the segmentation process of the SAM model is constrained, the segmentation result which is consistent in semantics and accurate in boundary is generated, and the optimal mask is further optimized through semantic consistency, so that the segmentation ambiguity is fundamentally solved, and the efficiency and accuracy of remote sensing image labeling are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to a human-computer collaborative remote sensing image annotation method based on the SAM model. Background Art

[0002] Remote sensing technology, as a crucial means of Earth observation, produces image data that plays an indispensable role in numerous fields, including urban planning, resource exploration, environmental protection, and disaster monitoring. To efficiently extract valuable information from massive amounts of remote sensing imagery, precise semantic annotation of imaged objects (such as buildings, roads, water bodies, and vegetation) is required. This involves assigning a specific category label to each pixel in the image. However, traditional, purely manual annotation methods are not only time-consuming, labor-intensive, and costly, but also prone to inconsistent annotation results due to subjective factors, making them inadequate for large-scale, high-precision applications. Therefore, developing collaborative human-machine intelligent annotation methods for remote sensing images, combining the computational efficiency of artificial intelligence with the cognitive judgment of human experts, has become a key approach to improving the efficiency and quality of remote sensing image interpretation.

[0003] In recent years, the advancement of deep learning technology, particularly the emergence of segmentation-based models such as SAM, has brought revolutionary breakthroughs to the field of image segmentation. With its strong zero-shot generalization capabilities, SAM can segment arbitrary objects in an image based on simple user-provided cues (such as points, boxes, and text), demonstrating great potential in general image segmentation tasks. This makes it an ideal technical foundation for building a new generation of human-machine collaborative annotation tools. However, direct application of SAM to the highly specialized field of remote sensing image annotation still faces significant challenges. The core technical issue lies in the fact that, as a category-agnostic, general model, SAM lacks prior semantic knowledge of remote sensing objects. When a user provides a simple cue point, SAM struggles to accurately understand the user's true intention to annotate a complete object of a specific category, given the complex scenes, dense objects, and varying scales of remote sensing images. For example, in dense urban imagery, if a user clicks on the roof of a building, SAM may segment only a portion of the roof or mistakenly include adjacent trees or roads, resulting in multiple, ambiguous candidate results. The ambiguity of this segmentation stems from the model's inability to effectively associate simple user interactions with the semantic information of the remote sensing scene, which requires users to perform a large number of extra clicks or repeated corrections, greatly reducing the labeling efficiency and violating the original intention of human-computer collaboration to improve efficiency.

[0004] Therefore, an optimized human-machine collaborative remote sensing image annotation solution is needed to solve the above technical problems. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a human-machine collaborative remote sensing image annotation method based on the SAM model. By constructing a dual-path detection framework that can distinguish and collaboratively process acute anomalies and chronic baseline drift, it overcomes the shortcomings of the existing technology of the single threshold method with poor adaptability and the traditional model's inability to distinguish anomaly types.

[0006] According to one aspect of the present application, a method for human-machine collaborative remote sensing image annotation based on the SAM model is provided, which includes: Obtain original remote sensing images; Performing semantic priori pre-calculation on the original remote sensing image to obtain a semantic map; Performing dynamic semantic hint enhancement on the semantic graph to obtain an enhanced hint set; Based on the enhanced prompt set, the original remote sensing image is subjected to guided segmentation and multi-candidate generation based on the SAM model to obtain a candidate mask set and an original IOU score set; Based on the original IOU score set and the semantic graph, the candidate mask set is preferentially sorted by semantic consistency to obtain a sorted candidate mask set; On the user interface, the candidate mask ranked first in the sorted candidate mask set is displayed as a recommendation result.

[0007] Preferably, performing semantic a priori pre-calculation on the original remote sensing image to obtain a semantic map includes: inputting the original remote sensing image into a trained semantic segmentation model to obtain the semantic map.

[0008] Preferably, inputting the original remote sensing image into a trained semantic segmentation model to obtain the semantic map comprises: Inputting the original remote sensing image into the encoder of the trained semantic segmentation model to obtain a remote sensing image visual shallow feature map and a remote sensing image visual semantic feature map; Input the remote sensing image visual shallow feature map and remote sensing image visual semantic feature map into the visual feature cross-modal interaction layer of the trained semantic segmentation model to obtain the remote sensing image visual multi-scale fusion feature map; The remote sensing image visual multi-scale fusion feature map is input into the decoder of the trained semantic segmentation model to obtain the semantic map.

[0009] Preferably, the remote sensing image visual shallow feature map and the remote sensing image visual semantic feature map are input into the visual feature cross-modal interaction layer of the trained semantic segmentation model to obtain the remote sensing image visual multi-scale fusion feature map, including: Reshaping the remote sensing image visual shallow feature map to obtain a remote sensing image visual shallow feature vector; Reshaping the remote sensing image visual semantic feature map to obtain a remote sensing image visual semantic feature vector; Performing local feature perception and linear interpolation-based dimension unification on the remote sensing image visual shallow feature vector to obtain a sequence distribution of the remote sensing image visual shallow feature local granularity coding vector; After inserting the remote sensing image visual semantic feature vectors into different positions of the sequence distribution of the remote sensing image visual shallow feature local granularity encoding vector, the vectors are input into an interactive encoder based on a bidirectional LSTM model to obtain a set of remote sensing image visual multi-scale feature cross-position interactive encoding feature vectors; Dynamically aggregating a set of cross-position interactively encoded feature vectors of the remote sensing image visual multi-scale features to obtain a remote sensing image visual multi-scale fusion feature vector; The remote sensing image visual multi-scale fusion feature vector is reshaped to obtain the remote sensing image visual multi-scale fusion feature map.

[0010] Preferably, performing dynamic semantic hint enhancement on the semantic graph to obtain an enhanced hint set includes: Get the user's original prompt point; Based on the user's original prompt point, target semantic region identification and extraction are performed in the semantic graph to obtain a target region mask and a target category ID; Based on the target category ID and the semantic map, generating positive and negative sample points based on boundary perception for the target area mask to obtain a positive sample point set and a negative sample point set; The user's original prompt points, positive sample point set and negative sample point set are combined into an enhanced prompt set to obtain an enhanced prompt set.

[0011] Preferably, based on the user's original prompt point, target semantic region identification and extraction are performed in the semantic graph to obtain a target region mask and a target category ID, including: Extract the coordinates (x, y) from the user's original prompt point; Using the coordinates (x, y) as the index, query the pixel value of the corresponding pixel in the semantic map as the target category ID; The coordinates (x, y) of the original prompt point of the user are used as seed points, and a connected component analysis is performed on the semantic graph to obtain the target region mask.

[0012] Preferably, based on the target category ID and the semantic map, generating the target area mask based on boundary-aware positive and negative sample points to obtain a positive sample point set and a negative sample point set includes: Performing random sampling in the area defined by the target area mask to obtain the positive sample point set; performing morphological dilation on the target region mask to obtain a dilated mask; subtracting the target region mask from the dilated mask to obtain a boundary ring mask; The negative sample point set is extracted from the boundary ring mask based on the pixel value of each pixel position in the semantic map.

[0013] Preferably, based on the enhanced prompt set, the original remote sensing image is subjected to guided segmentation and multi-candidate generation based on the SAM model to obtain a candidate mask set and an original IOU score set, including: Inputting the original remote sensing image into the image encoder of the SAM model to obtain image feature embedding; Performing prompt injection based on the enhanced prompt set to obtain a prompt embedding feature; The hint embedding features and image features are embedded into the mask decoder of the input SAM model to obtain a candidate mask set and an original IOU score set.

[0014] Preferably, performing prompt injection based on the enhanced prompt set to obtain prompt embedding features includes: extracting a first cue point from the enhanced cue set; Map the coordinates of the first hint point (a, b) to the coordinates under the feature map coordinates ( , ); With the coordinates ( , ) is an index to search the hint embedding feature to obtain a first point feature vector; performing type-specific embedding encoding on the target category ID of the first cue point to obtain a first type embedding; The first point feature vector and the first type of embedding are fused to obtain a first hint embedding feature.

[0015] Beneficial effect: Compared with the existing technology, the human-machine collaborative remote sensing image annotation method based on the SAM model provided by this application does not directly input the user's sparse prompts into SAM, but first uses a pre-trained remote sensing semantic segmentation model to pre-calculate the image and generate a semantic map rich in ground feature category information. On this basis, the user's single-point interaction is combined with the semantic map to perform dynamic semantic prompt enhancement, that is, automatically identify the user's intended area and generate a dense enhanced prompt set with clear positive and negative attributes. The enhanced prompt set can convert the user's vague intention into a guidance signal that the machine can accurately understand, effectively constrain the segmentation process of the SAM model, and generate a semantically consistent and boundary-accurate segmentation result. It further selects the best mask through semantic consistency, thereby fundamentally solving the segmentation ambiguity and improving the efficiency and accuracy of remote sensing image annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 This is a flowchart of an embodiment of the present application; Figure 2 A schematic diagram of data flow in an embodiment of the present application; Figure 3 This is a flowchart of inputting the original remote sensing image into the trained semantic segmentation model to obtain the semantic map in the embodiment of the present application; Figure 4 This is a flowchart of an embodiment of the present application in which a remote sensing image visual shallow feature map and a remote sensing image visual semantic feature map are input into a visual feature cross-modal interaction layer of a trained semantic segmentation model to obtain a remote sensing image visual multi-scale fusion feature map; Figure 5 This is a flowchart of performing dynamic semantic hint enhancement on the semantic graph to obtain an enhanced hint set in an embodiment of the present application. DETAILED DESCRIPTION

[0018] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0019] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0020] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0021] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0022] It should be noted that the acquisition and processing of all information or data in this application are carried out in compliance with the relevant national data protection laws and policies and with the authorization given by the owner of the corresponding device.

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0024] Existing technologies that apply the SAM model to remote sensing image annotation face a core technical problem: as a general segmentation model, SAM lacks specialized semantic knowledge of remote sensing objects, resulting in its inability to accurately understand the user's true annotation intent. When a user provides a simple cue point on a complex remote sensing image, the model often generates multiple possible, ambiguous segmentation results due to the dense objects and blurred boundaries in the scene, significantly reducing the efficiency and accuracy of human-computer collaboration. To address this technical issue, the present invention proposes a method for human-computer collaborative remote sensing image annotation based on the SAM model. Before user interaction, this method first preprocesses the original image using a trained remote sensing semantic segmentation model to generate a semantic map containing pixel-level object category information, providing critical prior knowledge for subsequent operations. After the user provides the initial cue point, the system does not directly use it for segmentation. Instead, it uses it as an index to query the semantic map for the target category and identify the complete connected region corresponding to that category. The system then performs dynamic semantic cue enhancement around this region: automatically sampling dense positive sample points within the region and simultaneously sampling negative sample points in the surrounding area immediately adjacent to its boundary. These rich, informative positive and negative sample points intelligently generated by the semantic graph are combined with the user's original click to form a powerful set of enhanced prompts. Finally, this enhanced prompt set is injected into the SAM model, guiding it to perform segmentation based on a full understanding of user intent and feature context, and to prioritize multiple candidate results based on the semantic graph. In this way, the user's ambiguous intent is converted into semantic guidance signals that can be precisely understood by the machine, fundamentally resolving segmentation ambiguity and facilitating efficient and accurate human-machine collaborative remote sensing image annotation.

[0025] like Figure 1 and Figure 2As shown, the human-machine collaborative remote sensing image annotation method based on the SAM model according to the embodiment of the present application includes: S100, obtaining the original remote sensing image; S200, performing semantic prior pre-calculation on the original remote sensing image to obtain a semantic graph; S300, performing dynamic semantic prompt enhancement on the semantic graph to obtain an enhanced prompt set; S400, based on the enhanced prompt set, performing guided segmentation and multi-candidate generation based on the SAM model on the original remote sensing image to obtain a candidate mask set and an original IOU score set; S500, based on the original IOU score set and the semantic graph, performing semantic consistency priority sorting on the candidate mask set to obtain a sorted candidate mask set; S600, displaying the candidate mask ranked first in the sorted candidate mask set as a recommendation result on the user interface.

[0026] Specifically, in steps S100 and S200, the original remote sensing image is acquired and semantics are pre-computed a priori on the original remote sensing image to generate a semantic map. It should be understood that the SAM model itself, as a general segmentation foundation model, lacks domain knowledge specific to specific objects in remote sensing images. This results in its inability to accurately discern the user's true intent when faced with simple interactive prompts, leading to segmentation ambiguity. Therefore, in the human-machine collaborative remote sensing image annotation method based on the SAM model, the original remote sensing image must first be acquired and semantics pre-computed a priori on this image to generate a semantic map. In other words, pixel-level semantic context information is pre-injected into the system before performing human-machine interactive segmentation. By generating a semantic map, the system can pre-understand the object category, such as building, road, or water, to which each pixel in the image belongs. Ultimately, this semantic map becomes the cornerstone of all subsequent intelligent processing steps. It can transform the user's ambiguous and sparse interactions into semantically rich guidance signals that the machine can accurately understand, thereby laying a solid foundation for subsequent accurate segmentation and efficient annotation.

[0027] More specifically, in an embodiment of the present application, semantic prior pre-computation is performed on the original remote sensing image to obtain a semantic map, including: inputting the original remote sensing image into a trained semantic segmentation model to obtain the semantic map. Specifically, the semantic prior pre-computation process is implemented by a trained deep learning semantic segmentation model optimized for remote sensing images. First, the original remote sensing image to be labeled is used as input and sent to the encoder module of the semantic segmentation model. The encoder extracts two key features from the original remote sensing image in parallel through multi-layer convolution and downsampling operations: on the one hand, the visual shallow feature map of the remote sensing image that retains rich spatial details and texture information, and on the other hand, the visual semantic feature map of the remote sensing image that contains high-level, abstract category concepts. Subsequently, in order to comprehensively utilize the advantages of these two features at different scales, the visual shallow feature map of the remote sensing image and the visual semantic feature map of the remote sensing image are combined. Figure 1 This information is then fed into the cross-modal interaction layer of visual features. This interaction layer effectively fuses and aligns shallow detail information with deep semantic information, generating a multi-scale fused visual feature map of the remote sensing image that combines both detailed accuracy and global semantics. Finally, this fused feature map is fed into the decoder module of the semantic segmentation model. Through a series of upsampling and convolution operations, the decoder gradually restores the highly concentrated feature information to the resolution of the original remote sensing image. The final output is a semantic map, in which the value of each pixel corresponds to a predefined feature category identifier.

[0028] like Figure 3 As shown, the original remote sensing image is input into the trained semantic segmentation model to obtain the semantic map, including: S210, inputting the original remote sensing image into the encoder of the trained semantic segmentation model to obtain the remote sensing image visual shallow feature map and the remote sensing image visual semantic feature map; S220, inputting the remote sensing image visual shallow feature map and the remote sensing image visual semantic feature map into the visual feature cross-modal interaction layer of the trained semantic segmentation model to obtain the remote sensing image visual multi-scale fusion feature map; S230, inputting the remote sensing image visual multi-scale fusion feature map into the decoder of the trained semantic segmentation model to obtain the semantic map.

[0029] Specifically, in step S210, the original remote sensing image is input into the encoder of the trained semantic segmentation model to obtain a remote sensing image visual shallow feature map and a remote sensing image visual semantic feature map. It should be understood that accurate recognition of ground objects in remote sensing images requires both high-resolution boundary detail information to accurately outline the contours and a wide range of global context information to correctly determine the category. Layers of different depths in the encoder network are precisely able to capture these two types of information. Therefore, during the semantic prior pre-computation process, the original remote sensing image is input into the encoder of the trained semantic segmentation model to obtain a remote sensing image visual shallow feature map and a remote sensing image visual semantic feature map, so as to decouple and extract these two different but functionally complementary feature information from the original remote sensing image in parallel. The system can simultaneously obtain a remote sensing image visual shallow feature map rich in spatial texture and edge details, as well as a remote sensing image visual semantic feature map containing high-level, abstract category concepts. Together, these two provide a solid data foundation for subsequent feature fusion and the generation of a high-quality final semantic map.

[0030] More specifically, in a specific example of the present application, the encoder adopts a hierarchical backbone network architecture, such as a structure based on a residual network (ResNet). When the original remote sensing image is fed into the input of the encoder, the remote sensing image data sequentially passes through multiple residual modules or feature extraction stages consisting of convolutional layers, normalization layers, and nonlinear activation functions. During this process, the feature maps output from one or more earlier stages of the network (such as the first or second stage) are used as shallow visual feature maps of the remote sensing image because they have undergone fewer downsampling operations and retain higher spatial resolution and rich low-level visual features. At the same time, the data flow continues to propagate forward in the network. After deeper feature extraction and abstraction, the feature map output from the last stage of the encoder has a lower spatial resolution but a larger receptive field. Each feature vector encodes highly concentrated semantic information about the global content of the remote sensing image. This feature map is then used as the visual semantic feature map of the remote sensing image.

[0031] Specifically, in step S220, the remote sensing image visual shallow feature map and the remote sensing image visual semantic feature map are input into the visual feature cross-modal interaction layer of the trained semantic segmentation model to obtain the remote sensing image visual multi-scale fusion feature map. It should be understood that traditional fusion methods, such as simple element-level addition or channel splicing, often cannot deeply reveal the intrinsic structural dependency between the two heterogeneous remote sensing image visual features, that is, they cannot answer how high-level semantic concepts should accurately guide the interpretation of underlying details in space. Therefore, in the semantic segmentation model, the remote sensing image visual shallow feature map and the remote sensing image visual semantic feature map are cross-modally fused to construct an asymmetric deep interaction paradigm based on systematic permutation and sequence modeling to exhaustively explore the structural and positional dependencies between shallow detail features and deep semantic features.

[0032] Specifically, this approach abandons the traditional approach of treating the visual features of two remote sensing images as homogeneous information for fusion. Instead, it deconstructs the detail-rich shallow visual features of remote sensing images from a static feature map into a dynamic sequence of local features with internal positional relationships. Subsequently, the deep semantic features of the remote sensing images, viewed as atomic semantic probes, are systematically embedded into every possible position of this shallow feature sequence, creating a complete set of scenarios of cross-scale feature interactions. By deeply encoding each interaction scenario using a bidirectional sequence model, this layer accurately captures the contextual semantic changes triggered by the injection of remote sensing image visual semantic information at different spatial locations. Finally, through a dynamic aggregation mechanism, the most critical multi-scale interaction patterns of remote sensing image visual images are intelligently identified and amplified. The resulting multi-scale fusion feature map output by this interaction layer is no longer a simple superposition of the two information types, but a highly optimized representation of remote sensing image visual features that deeply understands the core question of how the two interact in their structural locations. This deeply fused feature map provides high-quality input for the subsequent decoder to generate a semantic map with clear boundaries and accurate categories, thereby achieving the ultimate effect of improving the accuracy of semantic segmentation.

[0033] like Figure 4 As shown, step S220 includes: S221, reshaping the feature shape of the remote sensing image visual shallow feature map to obtain the remote sensing image visual shallow feature vector; S222, reshaping the feature shape of the remote sensing image visual semantic feature map to obtain the remote sensing image visual semantic feature vector; S223, performing local feature perception and linear interpolation-based dimension unification on the remote sensing image visual shallow feature vector to obtain the sequence distribution of the remote sensing image visual shallow feature local granularity encoding vector; S224, inserting the remote sensing image visual semantic feature vector into the After reaching different positions of the sequence distribution of the local granularity encoding vector of the remote sensing image visual shallow feature, it is input into the interactive encoder based on the bidirectional LSTM model to obtain a set of cross-position interactive encoding feature vectors of the remote sensing image visual multi-scale feature; S225, the set of cross-position interactive encoding feature vectors of the remote sensing image visual multi-scale feature is dynamically aggregated to obtain the remote sensing image visual multi-scale fusion feature vector; S226, the remote sensing image visual multi-scale fusion feature vector is feature-reshaped to obtain the remote sensing image visual multi-scale fusion feature map.

[0034] In step S221 and step S222, the remote sensing image visual shallow feature map is reshaped to obtain a remote sensing image visual shallow feature vector, and the remote sensing image visual semantic feature map is reshaped to obtain a remote sensing image visual semantic feature vector, which is expressed by the following formula: ; ; in, and They are respectively the visual shallow feature map of remote sensing images and the visual semantic feature map of remote sensing images. Reshape the feature shape, represents the visual shallow feature vector of the remote sensing image, Represents the visual semantic feature vector of remote sensing image.

[0035] It should be understood that the deep interaction paradigm employed by this visual feature cross-modal interaction layer is not based on traditional spatial convolution operations, but rather builds upon a sequence model, aiming to explore positional and structural dependencies between features. The original two-dimensional feature map morphology cannot be directly processed by the sequence model, so format conversion is necessary. Specifically, in the visual feature cross-modal interaction layer, reshaping the remote sensing image visual shallow feature map and the remote sensing image visual semantic feature map to obtain corresponding feature vectors is a fundamental step in initiating the deep interaction paradigm. This step is not simply a data format conversion; rather, it pre-processes the two heterogeneous feature maps into specific forms that suit their different roles in the subsequent interaction. Specifically, reshaping the remote sensing image visual shallow feature map prepares it as a macroscopic, deconstructable entity. This completes the necessary data format pre-processing for the subsequent deep interaction process, which involves deconstructing the remote sensing image visual shallow features into a dynamic sequence with internal positional relationships and systematically embedding the remote sensing image visual semantic features as atomic probes.

[0036] Step S223, performing local feature extraction and dimension alignment on the remote sensing image visual shallow feature vector to obtain a sequence distribution of the remote sensing image visual shallow feature local granularity coding vector, which is expressed by the following formula: ; ; in, represents the visual shallow feature vector of the remote sensing image, is a one-dimensional convolutional code, is the sliding window size, express The location of the eigenvalues ​​in , Represents the local shallow feature vector of remote sensing image vision, is a linear interpolation operation, is the target dimension, for The number of eigenvalues ​​in , Represents the sequence distribution of local granularity encoding vectors of shallow visual features of remote sensing images, It is A local granularity encoding vector of visual shallow features of remote sensing images.

[0037] It's understandable that a single, macroscopic shallow feature vector of a remote sensing image obscures the rich, spatially related details within it. To achieve deep interaction with the visual semantic features of remote sensing images—that is, to explore the spatial structure under which semantic information influences detailed features—this static shallow feature vector must first be made dynamic and serialized. Therefore, the purpose of this step is to dissect the macroscopic shallow feature vector into its inherent, more fine-grained components. Specifically, local feature perception techniques (such as sliding window convolution) extract local features from different regions within the shallow feature vector of a remote sensing image, generating a series of subvectors representing details at different spatial locations. Subsequently, because these subvectors may have inconsistent dimensions and cannot be directly sequenced, a dimensional unification operation based on linear interpolation ensures that the local shallow feature vectors of the remote sensing image at all local granularities have the same length, allowing them to be organized into an ordered sequence distribution suitable for processing by time series models.

[0038] Step S224: After inserting the remote sensing image visual semantic feature vectors into different positions of the sequence distribution of the remote sensing image visual shallow feature local granularity encoding vector, the vectors are input into an interactive encoder based on a bidirectional LSTM model to obtain a set of remote sensing image visual multi-scale feature cross-position interactive encoding feature vectors, which is expressed by the following formula: ; ; in, They represent the first and the second in the sequence distribution of the local granularity coding vector of the remote sensing image visual shallow feature respectively. , and A local granularity encoding vector of the visual shallow features of remote sensing images, Represents the visual semantic feature vector of remote sensing image, Represents the sequence distribution of local granularity insertion coding vectors of remote sensing image visual multi-scale features, represents a bidirectional LSTM model, A collection of feature vectors representing the cross-location interaction encoding of multi-scale visual features of remote sensing images.

[0039] It should be understood that simple feature concatenation or element-wise operations cannot answer a key question: how should high-level remote sensing image visual semantic concepts (e.g., "this is a building area") guide the interpretation of low-level detail features within a specific spatial structure (e.g., at the edge, center, or corner of a building)? Therefore, in the technical solution of this application, the remote sensing image visual semantic feature vectors are inserted into different positions of the sequence distribution of the local granularity encoding vectors of the remote sensing image visual shallow features. These vectors are then input into an interactive encoder based on a bidirectional LSTM model to obtain a set of cross-position interactive encoding feature vectors of the remote sensing image visual multi-scale features. In other words, to exhaustively explore this structural and positional dependency, the remote sensing image visual semantic feature vectors, viewed as atomic probes, must be systematically embedded into every possible position in the shallow feature sequence representing spatial details. This creates a complete set of scenarios regarding interactions. Each insertion generates a unique mixed sequence, representing a unique contextual configuration, and proposes a hypothesis about the semantic flow that occurs when semantic information appears at this specific position within the shallow feature structure. Each scene is then deeply encoded using a powerful sequence modeling tool called a bidirectional LSTM model. This aims to accurately capture the long-range dependencies and global contextual changes triggered by the injection of semantic information, both before and after the insertion point. This collection of feature vectors encoding cross-location interactions of multi-scale visual features from remote sensing imagery comprehensively captures the full range of possible interactions between semantic features and shallow, detailed visual features in remote sensing imagery at all possible spatial locations. This provides rich and comprehensive raw material for the subsequent intelligent identification and amplification of the most critical interaction patterns through a dynamic aggregation mechanism, and is crucial for achieving the cognitive leap from simple feature fusion to deep interaction.

[0040] Step S225, dynamically aggregate the set of cross-position interactive coding feature vectors of the remote sensing image visual multi-scale features to obtain a remote sensing image visual multi-scale fusion feature vector, which is expressed by the following formula: ; ; ; in, is the mean vector of the set of cross-position interactive encoding feature vectors of the remote sensing image visual multi-scale features, is the transposed vector of the mean vector, , represents the contribution of multi-scale visual features of remote sensing images, Represents the visual multi-scale fusion feature vector of remote sensing imagery.

[0041] It should be understood that in the interaction scenario of the previous step, the cross-position interaction encoding features of the remote sensing image visual multi-scale features obtained are not equally important at all locations. For example, injecting semantic information about a building into shallow features at its boundary significantly contributes to accurate segmentation compared to injecting it into the homogeneous region within the building. A simple averaging or concatenation operation would indiscriminately treat these interactions, diluting the most critical interaction signals and causing the final fused features to fail to highlight key structural relationships. Therefore, in the visual feature cross-modal interaction layer, the set of feature vectors encoding cross-position interactions of the remote sensing image visual multi-scale features is further dynamically aggregated to obtain a remote sensing image visual multi-scale fused feature vector. An intelligent information selection and fusion mechanism is introduced to achieve final information convergence. By employing dynamic aggregation, this layer adaptively learns to evaluate the contribution of each remote sensing image visual multi-scale interaction configuration—that is, the contribution of semantic features injected at different spatial locations—and assigns higher weights to those interactions that best reveal the relationship between shallow details and deep semantic cores. Its purpose is to intelligently identify and amplify the most effective interaction patterns from a multitude of possibilities, rather than simply averaging information. More importantly, the multi-scale fusion feature vector of remote sensing imagery contains profound insights into the interaction patterns between shallow and deep visual features in all possible structural relationships. It is no longer a simple blend of information, but an optimized representation with a deep understanding of the complex relationships between features. This provides the highest-quality input for the subsequent decoder to generate a semantic map with clear boundaries and accurate categories.

[0042] Step S226, reshape the remote sensing image visual multi-scale fusion feature vector to obtain the remote sensing image visual multi-scale fusion feature map, which is expressed by the following formula: ; in, is the visual multi-scale fusion feature map of remote sensing images, Feature reshape operation.

[0043] It should be understood that in the cross-modal interaction layer, the feature reshaping of the resulting multi-scale fused remote sensing image feature vector to produce a multi-scale fused remote sensing image feature map is a necessary bridge between the deep interaction module and the standard decoder module. The subsequent decoder module is designed to restore feature information to the spatial dimensions of the original remote sensing image through operations such as upsampling and convolution to generate a pixel-level semantic map. Its architecture inherently requires input feature maps with spatial structure (i.e., height and width), while the single fused feature vector produced by the preceding dynamic aggregation has lost this spatial form. Therefore, the fundamental purpose of this step is to reproject the profound insights into the complex relationships between features, condensed by the preceding deep interaction and dynamic aggregation, from an abstract one-dimensional vector space back into a two-dimensional feature space. This effort aims to re-attribute this highly concentrated interaction knowledge to spatial attributes, enabling it to be understood and utilized by spatially sensitive operations (such as convolution) in the decoder, thereby transforming abstract semantic relationships into concrete pixel-level classification guidance and ensuring the compatibility and integrity of the entire network process.

[0044] Specifically, in step S230, the multi-scale fused visual feature map of the remote sensing image is input into the decoder of the trained semantic segmentation model to obtain the semantic map. It should be understood that the multi-scale fused visual feature map of the remote sensing image produced in the previous step, while containing profound insights into the complex relationships between features, is essentially still a highly condensed, low-spatial-resolution abstract representation. To generate a pixel-level semantic map consistent with the original remote sensing image size that can be directly used by the subsequent human-machine collaborative system, the technical solution of this application further inputs the multi-scale fused visual feature map of the remote sensing image obtained after deep interaction into the decoder of the trained semantic segmentation model. This is the final step in the transition from abstract feature understanding to concrete pixel-level annotation and ultimately embodies the value of the entire process. The decoder's task is to systematically and layer-by-layer restore the spatial resolution of the feature map, while utilizing its internal convolutional layers to further refine and integrate the high-quality information provided by the fused feature map. Its core goal is to accurately and losslessly translate the fused features, which have already resolved the complex question of how they are related in which structural position, into a final category judgment for each pixel.

[0045] More specifically, in one specific example of this application, the decoder implementation follows a symmetrical architecture that progressively upscales feature maps and restores detail, similar to the encoder. First, the decoder receives as its initial input a visual multi-scale fused feature map of the remote sensing image output by the visual feature cross-modal interaction layer. This feature map then enters the decoder's first upsampling module, which upscales its spatial resolution (i.e., height and width) via a transposed convolutional layer. To compensate for the spatial detail lost during encoding, the upsampled feature map is channel-wise concatenated with a shallow feature map from the corresponding encoder layer, which retains rich detail. This allows for direct fusion of high-level semantic information with low-level detail information. The concatenated feature map is then passed through one or more standard convolutional layers for further feature fusion and refinement, smoothing out artifacts that may be introduced by upsampling and learning more accurate feature representations. This process of upsampling, feature concatenation, and convolutional refinement is repeated multiple times, with the spatial resolution of the feature map increasing at each stage until its size matches that of the original input remote sensing image. Finally, a final classification head, usually a 1x1 convolutional layer, is used to map the multi-channel feature map into a score map with the number of channels equal to the number of predefined categories, and the corresponding classification function is applied to each pixel position to determine the final category of each pixel point and obtain the final semantic map.

[0046] Specifically, in step S300, the semantic graph is dynamically enhanced with semantic hints to obtain an enhanced hint set. It should be understood that the individual original hint points provided by the user are inherently sparse and ambiguous, and segmentation models such as SAM lack domain priors. Therefore, it is difficult to accurately infer the user's complete segmentation intent based solely on single-point input, and this can easily lead to erroneous segmentation results. Therefore, in the human-machine collaborative annotation process, further dynamic semantic hint enhancement of the semantic graph generated in the previous step to obtain an enhanced hint set is a key link in connecting machine semantic understanding with human interaction intent. By leveraging the rich prior knowledge provided by high-quality semantic graphs, the user's single, ambiguous interactive input is automatically and intelligently expanded into a dense set of hints with extensive information and clear directionality, thereby eliminating ambiguity. This generates an enhanced hint set that includes the original intent, positive samples of the target region, and negative samples of the background region. This hint set can inject complex semantic constraints and boundary information into the subsequent segmentation model in a model-friendly manner, significantly guiding the model to generate highly accurate segmentation masks that meet user expectations.

[0047] like Figure 5As shown, step S300 includes: S310, obtaining the user's original prompt point; S320, based on the user's original prompt point, identifying and extracting the target semantic area in the semantic map to obtain a target area mask and a target category ID; S330, based on the target category ID and the semantic map, generating positive and negative sample points based on boundary perception for the target area mask to obtain a positive sample point set and a negative sample point set; S340, synthesizing the user's original prompt point, the positive sample point set and the negative sample point set into an enhanced prompt set to obtain an enhanced prompt set.

[0048] In steps S310 and S320, the user's original cue point is obtained. Based on this, the target semantic region is identified and extracted from the semantic graph to obtain a target region mask and a target category ID. It should be understood that the user's interaction intent is expressed through a sparse click action, and the system must first accurately translate this physical action into a machine-understandable semantic concept. In the dynamic semantic cue enhancement process, obtaining the user's original cue point and using it to identify and extract the target semantic region in the semantic graph is the starting point and foundation of the entire enhancement process. The fundamental purpose of this step is to leverage the deterministic prior knowledge of a high-quality semantic graph to accurately resolve the single, specific feature instance targeted by the user's click, specifically clarifying the specific feature category and object the user wishes to segment. In this way, the system successfully transforms a fuzzy coordinate point input into two clear, structured data entities: a target category ID representing the feature category and a target region mask that accurately depicts the spatial extent of the feature instance. This provides an indispensable basis for subsequently generating a dense set of positive and negative sample points based on this region.

[0049] More specifically, in this embodiment of the present application, based on the user's original cue point, target semantic regions are identified and extracted from the semantic graph to obtain a target region mask and target category ID. This process includes: extracting the coordinates (x, y) from the user's original cue point; using the coordinates (x, y) as an index, querying the semantic graph for the corresponding pixel value as the target category ID; and performing connected component analysis on the semantic graph using the coordinates (x, y) of the user's original cue point as a seed point to obtain the target region mask. Specifically, the identification and extraction process first obtains the original cue point from the user interface and extracts its two-dimensional coordinates (x, y) in the remote sensing image coordinate system. The system then uses the coordinates (x, y) as an index to directly query the pre-generated semantic graph, reading and recording the pixel value corresponding to the pixel location. This pixel value is then defined as the target category ID. Next, using the coordinates (x, y) of the user's original cue point as a seed point, and subject to the determined target category ID, the system performs a connected component analysis algorithm on the semantic graph. The algorithm traverses the pixels adjacent to the seed point with the same category ID and recursively expands to all connected pixels of the same type until no further expansion is possible, thereby identifying this complete, independent area and generating the corresponding binary target area mask.

[0050] In step S330, based on the target category ID and the semantic map, boundary-aware positive and negative sample points are generated for the target region mask to obtain positive and negative sample point sets. It should be understood that providing only the overall target region mask or a few internal positive sample points to the subsequent segmentation model is insufficient to accurately constrain its behavior at the boundary, and the model may still leak or shrink. Therefore, based on the target category ID and the semantic map, boundary-aware positive and negative sample points are generated for the target region mask to establish a strong and clear boundary constraint by strategically and densely placing positive and negative sample points near the inner and outer boundaries of the target region, thereby reducing the difficulty of the segmentation task from finding a general region to accurately outlining the contour defined by the dense point set. This step ultimately generates two point sets with high spatial correlation: a positive sample point set that confirms the target interior region, and a negative sample point set that explicitly excludes the adjacent background region. Together, these two sets form a strong clamp on the target boundary, providing the most direct and effective guidance signal for the subsequent model to generate clear and accurate segmentation results.

[0051] More specifically, in an embodiment of the present application, based on the target category ID and the semantic map, the target region mask is generated based on boundary-aware positive and negative sample points to obtain a positive sample point set and a negative sample point set, including: performing random sampling within the area defined by the target region mask to obtain the positive sample point set; performing morphological dilation on the target region mask to obtain a dilated mask; subtracting the target region mask from the dilated mask to obtain a boundary ring mask; and extracting the negative sample point set from the boundary ring mask based on the pixel values ​​of each pixel position in the semantic map. More specifically, first, to generate a positive sample point set, the system performs a random sampling strategy within the binary area defined by the target region mask obtained in the previous step to generate a set of coordinate points. These points together constitute the positive sample point set, which serves to confirm to the model the ownership of the area inside the mask. Subsequently, to generate a negative sample point set, the system first performs a morphological dilation operation on the target region mask to obtain an dilated mask that is slightly larger in spatial range than the original mask. Then, by subtracting the original target area mask from the dilated mask, the system obtains a morphologically ring-shaped boundary ring mask that contains only pixels outside the boundary. Finally, the system samples within the area covered by the boundary ring mask to extract a set of negative sample points. However, this process is not indiscriminate sampling, but rather screening in combination with the semantic map: only when the category ID corresponding to a pixel position in the boundary ring mask on the semantic map is not equal to the determined target category ID, the position is considered a valid negative sample candidate point. The negative sample point set extracted in this way can accurately exclude the background area adjacent to the target boundary, thereby completing the precise constraint on the target boundary.

[0052] In step S340, the user's original prompt point, the positive sample point set, and the negative sample point set are enhanced and synthesized into a prompt set to obtain an enhanced prompt set. It should be understood that the three independent data entities generated in the previous step (an original point, a set of positive sample points, and a set of negative sample points) logically serve a unified segmentation goal together, but they are separated in data structure. The subsequent SAM model requires a single, structured prompt input to drive its segmentation process. In the final stage of the dynamic semantic prompt enhancement process, the user's original prompt point, the positive sample point set, and the negative sample point set are enhanced and synthesized into a prompt set, which is a key step in completing the transition from scattered guidance signals to unified model input. The fundamental purpose of performing this step is to structurally encapsulate these scattered but logically related point information and integrate them into a unified data entity that complies with the subsequent model interface specifications. The generated single and comprehensive set of enhanced prompts not only fully preserves the user's original intention, but also encodes complex semantics and boundary constraints into a format that can be directly understood and utilized by the model through dense auxiliary points with clear positive and negative labels, laying the ultimate data foundation for achieving high-precision and unambiguous guided segmentation.

[0053] More specifically, in a specific example of the present application, the implementation process of the enhanced prompt set synthesis is intended to create a unified data structure containing coordinates and corresponding labels. First, an empty list or array is initialized as a container for the enhanced prompt set. Subsequently, the user's original prompt point is processed, and its coordinates (x, y) and a label representing the foreground or positive sample (for example, the label value is 1) are added to the container as a data pair. Next, the positive sample point set generated in the previous step is traversed, and for each point in it, its coordinates are also paired with the positive sample label (the label value is 1), and these data pairs are added to the container one by one. Finally, the negative sample point set is traversed, and for each point in it, its coordinates are paired with a label representing the background or negative sample (for example, the label value is 0), and all these data pairs are also added to the container. After this process is completed, the container contains all relevant prompt points and their clear category affiliation, thereby forming the final enhanced prompt set that can be directly input into subsequent modules.

[0054] Specifically, in step S400, based on the enhanced hint set, the original remote sensing image undergoes guided segmentation and multi-candidate generation using the SAM model to obtain a set of candidate masks and a set of original IOU scores. It should be understood that the previous steps have transformed the user's vague intent into an information-dense set of enhanced hints. A powerful segmentation engine is now required to precisely guide this information on the original high-resolution image to generate pixel-level segmentation results. In the entire human-machine collaborative annotation process, performing guided segmentation and multi-candidate generation based on the SAM model on the original remote sensing image based on the enhanced hint set is the core step in utilizing a general segmentation model to solve specific annotation tasks. The fundamental purpose of this step is to efficiently combine semantic constraints (the enhanced hint set) with visual details (the original remote sensing image), driving the SAM model to complete a highly controlled, target-specific segmentation operation. Leveraging its multi-candidate generation capabilities, it then provides a set of high-quality segmentation hypotheses for subsequent optimization. Accordingly, a candidate mask set and a corresponding raw IOU score set are produced. The former contains multiple possible segmentation schemes for the target object, and the latter is the model's preliminary evaluation of the quality of these schemes, which together provide rich and reliable input for subsequent semantic consistency ranking.

[0055] More specifically, in an embodiment of the present application, based on the enhanced prompt set, the original remote sensing image is subjected to guided segmentation and multi-candidate generation based on the SAM model to obtain a candidate mask set and an original IOU score set, including: inputting the original remote sensing image into the image encoder of the SAM model to obtain image feature embedding; performing prompt injection based on the enhanced prompt set to obtain prompt embedding features; and inputting the prompt embedding features and image feature embedding into the mask decoder of the SAM model to obtain a candidate mask set and an original IOU score set. Wherein, performing prompt injection based on the enhanced prompt set to obtain prompt embedding features includes: extracting a first prompt point from the enhanced prompt set; mapping the coordinates (a, b) of the first prompt point to the coordinates ( , ); with the coordinates ( , ) is an index to search in the prompt embedding feature to obtain a first point feature vector; type-specific embedding encoding is performed on the target category ID of the first prompt point to obtain a first type embedding; and the first point feature vector and the first type embedding are fused to obtain a first prompt embedding feature.

[0056] Accordingly, the above-mentioned guided segmentation and multi-candidate generation process strictly follows the standard workflow of the SAM model. First, the original remote sensing image is input into the SAM model's image encoder. This encoder (a Vision Transformer architecture) performs deep feature extraction on the image, generating a high-dimensional image feature embedding that contains rich spatial and textural information. Simultaneously, the system performs cue injection based on the enhanced cue set generated in the previous step to obtain a cue embedding that encodes information about all guide points. This cue injection process processes each point in the enhanced cue set (here, the first cue point is used as an example): its coordinates (a, b) are extracted and mapped to the coordinate system of the feature map output by the image encoder, and its class label (i.e., positive or negative) is extracted. A point feature vector is generated for the point's coordinates, and a type-specific embedding is generated for its class label. These two are then fused to form the cue embedding for that point. The cue embeddings for all points together constitute the final cue embedding. Finally, this cue embedding, along with the aforementioned image feature embedding, is input into the SAM model's mask decoder. The decoder analyzes the relationship between the prompt and the image features through the attention mechanism and other methods, and finally outputs multiple different candidate masks related to the prompt in parallel to form a candidate mask set, and predicts a quality score for each candidate mask to form the original IOU score set.

[0057] Specifically, in step S500, based on the original IOU score set and the semantic map, the candidate mask set is preferentially sorted by semantic consistency to obtain a sorted candidate mask set. It should be understood that the original IOU score generated by the SAM model is essentially an assessment of geometric fidelity. It does not have semantic knowledge in the remote sensing field and may give a high score to a mask that is highly consistent in shape but mistakenly contains different adjacent features. This has a significant semantic blind spot. In the entire human-machine collaborative labeling process, the semantic consistency preferential sorting of the candidate mask set based on the original IOU score set and the semantic map is a key correction step to ensure that the final recommendation result conforms to both the geometric shape and the feature category definition. A second review mechanism based on domain knowledge is introduced to use the high-quality semantic map, a deterministic feature category prior, to correct and optimize the original ranking of the SAM model to ensure that the segmentation result finally recommended to the user is semantically correct. The mask ranked first in the sorted candidate mask set is not only geometrically preferred, but also highly consistent with the user's true intention in semantics, thereby improving the accuracy of annotation and reducing the burden of subsequent corrections on the user.

[0058] More specifically, in a specific example of the present application, the semantic consistency preference sorting process is systematically implemented. First, for each candidate mask in the candidate mask set, its exclusive semantic consistency score is calculated. This calculation process is completed by performing a pixel-level alignment comparison between the current candidate mask and the pre-generated semantic map. Count the number of pixels in the area covered by the current candidate mask whose corresponding category on the semantic map is consistent with the target category ID determined by the user's initial click, and then divide this number by the total number of pixels covered by the candidate mask to obtain a ratio between 0 and 1, which is the semantic consistency score of the candidate mask. Subsequently, the semantic consistency score is weightedly fused with the original IOU score corresponding to the candidate mask obtained from the original IOU score set to generate a final comprehensive ranking score. Finally, the entire candidate mask set is sorted in descending order based on the comprehensive ranking scores of all candidate masks to obtain the final, sorted candidate mask set, and the top candidate mask is the dual-optimal recommendation result in terms of geometry and semantics.

[0059] Specifically, in step S600, the top-ranked candidate mask in the sorted candidate mask set is displayed on the user interface as a recommended result. This allows the system's optimal judgment, resulting from multiple screening and intelligent optimization processes, to be delivered directly to the user in a zero-burden, highly efficient manner, maximizing the efficiency of human-computer collaboration. This approach addresses the issue of users being simply prompted with segmentation ambiguity, requiring extensive subsequent manual corrections. By providing a single, highly confident recommendation, the previously complex drawing-modification process is transformed into an efficient click-and-confirm process. After completing a simple click interaction, the user instantly obtains a highly accurate annotation result in terms of both geometric form and feature classification. Because this result is dual-secured by the SAM model's geometric fidelity assessment and the solution's unique semantic consistency check, its accuracy far exceeds that of traditional methods, reducing the need for users to perform secondary edits or sift through multiple options. This not only significantly improves the efficiency and quality of single-pass annotation but also fundamentally optimizes the overall workflow for remote sensing image annotation, achieving the technical solution's goal of efficient, accurate, and intelligent human-computer collaboration.

[0060] In summary, the human-machine collaborative remote sensing image annotation method based on the SAM model according to the embodiment of the present application is explained. It overcomes the defects of the poor adaptability of the single threshold method in the prior art and the inability of the traditional model to distinguish the types of abnormalities by constructing a dual-path detection framework that can distinguish and collaboratively process acute anomalies and chronic baseline drift. Specifically, instead of directly inputting the user's sparse prompts into SAM, a pre-trained remote sensing semantic segmentation model is first used to pre-calculate the image to generate a semantic map rich in ground feature category information. On this basis, the user's single-point interaction is combined with the semantic map to perform dynamic semantic prompt enhancement, that is, automatically identify the user's intended area and generate a dense enhanced prompt set with clear positive and negative attributes. The enhanced prompt set can convert the user's vague intention into a guidance signal that the machine can accurately understand, effectively constrain the segmentation process of the SAM model, and generate a semantically consistent and boundary-accurate segmentation result, and further select the best mask through semantic consistency, thereby fundamentally solving the segmentation ambiguity and improving the efficiency and accuracy of remote sensing image annotation.

[0061] As described above, the data collaborative management platform according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an electric vehicle transmission downshift control algorithm. In one possible implementation, the data collaborative management platform according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the data collaborative management platform can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the data collaborative management platform can also be one of the many hardware modules of the wireless terminal.

[0062] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A human-machine collaborative remote sensing image annotation method based on the SAM model, characterized in that: include: Obtain original remote sensing images; Performing semantic priori pre-calculation on the original remote sensing image to obtain a semantic map; Performing dynamic semantic hint enhancement on the semantic graph to obtain an enhanced hint set; Based on the enhanced prompt set, the original remote sensing image is subjected to guided segmentation and multi-candidate generation based on the SAM model to obtain a candidate mask set and an original IOU score set; Based on the original IOU score set and the semantic graph, the candidate mask set is preferentially sorted by semantic consistency to obtain a sorted candidate mask set; On the user interface, the candidate mask ranked first in the sorted candidate mask set is displayed as a recommendation result.

2. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 1 is characterized in that: Performing semantic priori pre-calculation on the original remote sensing image to obtain a semantic map includes: inputting the original remote sensing image into a trained semantic segmentation model to obtain the semantic map.

3. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 2 is characterized in that: Inputting the original remote sensing image into a trained semantic segmentation model to obtain the semantic map includes: Inputting the original remote sensing image into the encoder of the trained semantic segmentation model to obtain a remote sensing image visual shallow feature map and a remote sensing image visual semantic feature map; Input the remote sensing image visual shallow feature map and remote sensing image visual semantic feature map into the visual feature cross-modal interaction layer of the trained semantic segmentation model to obtain the remote sensing image visual multi-scale fusion feature map; The remote sensing image visual multi-scale fusion feature map is input into the decoder of the trained semantic segmentation model to obtain the semantic map.

4. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 3 is characterized in that: The remote sensing image visual shallow feature map and the remote sensing image visual semantic feature map are input into the visual feature cross-modal interaction layer of the trained semantic segmentation model to obtain the remote sensing image visual multi-scale fusion feature map, including: Reshaping the remote sensing image visual shallow feature map to obtain a remote sensing image visual shallow feature vector; Reshaping the remote sensing image visual semantic feature map to obtain a remote sensing image visual semantic feature vector; Performing local feature perception and linear interpolation-based dimension unification on the remote sensing image visual shallow feature vector to obtain a sequence distribution of the remote sensing image visual shallow feature local granularity coding vector; After inserting the remote sensing image visual semantic feature vectors into different positions of the sequence distribution of the remote sensing image visual shallow feature local granularity encoding vector, the vectors are input into an interactive encoder based on a bidirectional LSTM model to obtain a set of remote sensing image visual multi-scale feature cross-position interactive encoding feature vectors; Dynamically aggregating a set of cross-position interactively encoded feature vectors of the remote sensing image visual multi-scale features to obtain a remote sensing image visual multi-scale fusion feature vector; The remote sensing image visual multi-scale fusion feature vector is reshaped to obtain the remote sensing image visual multi-scale fusion feature map.

5. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 1 is characterized in that: Performing dynamic semantic hint enhancement on the semantic graph to obtain an enhanced hint set, including: Get the user's original prompt point; Based on the user's original prompt point, target semantic region identification and extraction are performed in the semantic graph to obtain a target region mask and a target category ID; Based on the target category ID and the semantic map, generating positive and negative sample points based on boundary perception for the target area mask to obtain a positive sample point set and a negative sample point set; The user's original prompt points, positive sample point set and negative sample point set are combined into an enhanced prompt set to obtain an enhanced prompt set.

6. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 5 is characterized in that: Based on the user's original prompt point, target semantic region identification and extraction are performed in the semantic graph to obtain a target region mask and a target category ID, including: Extract the coordinates (x, y) from the user's original prompt point; Using the coordinates (x, y) as the index, query the pixel value of the corresponding pixel in the semantic map as the target category ID; The coordinates (x, y) of the original prompt point of the user are used as seed points, and a connected component analysis is performed on the semantic graph to obtain the target region mask.

7. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 5 is characterized in that: Based on the target category ID and the semantic map, generating boundary-aware positive and negative sample points for the target region mask to obtain a positive sample point set and a negative sample point set, including: Performing random sampling in the area defined by the target area mask to obtain the positive sample point set; performing morphological dilation on the target region mask to obtain a dilated mask; subtracting the target region mask from the dilated mask to obtain a boundary ring mask; The negative sample point set is extracted from the boundary ring mask based on the pixel value of each pixel position in the semantic map.

8. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 1 is characterized in that: Based on the enhanced prompt set, the original remote sensing image is subjected to guided segmentation and multi-candidate generation based on the SAM model to obtain a candidate mask set and an original IOU score set, including: Inputting the original remote sensing image into the image encoder of the SAM model to obtain image feature embedding; Performing prompt injection based on the enhanced prompt set to obtain a prompt embedding feature; The hint embedding features and image features are embedded into the mask decoder of the input SAM model to obtain a candidate mask set and an original IOU score set.

9. The human-machine collaborative remote sensing image annotation method based on the SAM model according to claim 8 is characterized in that: Prompt injection is performed based on the enhanced prompt set to obtain prompt embedding features, including: extracting a first cue point from the enhanced cue set; Map the coordinates of the first hint point (a, b) to the coordinates under the feature map coordinates ( , ); With the coordinates ( , ) is an index to search the hint embedding feature to obtain a first point feature vector; performing type-specific embedding encoding on the target category ID of the first cue point to obtain a first type embedding; The first point feature vector and the first type of embedding are fused to obtain a first hint embedding feature.

Images