Cross-block sensing remote sensing image interactive segmentation method and device
By propagating local semantic features in remote sensing images and fusing them with basic features, the problem of ignoring spatial relationships caused by segmenting remote sensing images into small, independent blocks is solved, achieving global consistency and high-precision segmentation of remote sensing images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2025-12-18
- Publication Date
- 2026-04-21
AI Technical Summary
Existing technologies divide remote sensing images into small, independent segments, which leads to the neglect of the overall spatial relationships of the image. Different segments cannot share information, affecting the consistency and accuracy of remote sensing image segmentation.
By propagating the local semantic features of user interaction click locations to remote sensing images, cross-tile cue embeddings containing global interaction information are generated and fused with basic features to obtain fused features of image tiles. Finally, the complete segmentation result of the remote sensing image is obtained by stitching them together.
It achieves consistent user prompts across the entire map, improves the coherence and integrity of segmentation results between different map tiles, enhances the interactive segmentation accuracy and stability of large-format remote sensing images, and can cope with typical challenges of remote sensing images such as high information density scenes, sparse or small target distribution, complex background interference, and large-scale changes.
Smart Images
Figure CN121904068A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for interactive segmentation of remote sensing images with cross-tile perception. Background Technology
[0002] In related technologies, remote sensing images typically have high resolution and large format characteristics. Directly performing interactive segmentation on the entire remote sensing image requires a large amount of video memory and computing resources, making real-time inference difficult for high-resolution inputs. Therefore, related technologies generally adopt the approach of dividing large-format images into multiple smaller blocks and segmenting them independently to reduce the processing difficulty of the model.
[0003] However, since related technologies typically segment images into small, independent blocks, they can easily lead to the neglect of the overall spatial relationships of the image during block processing. As a result, user interaction prompts often only apply to the currently processed block, and different blocks cannot share these prompts. This makes it impossible to provide consistent guidance for the entire image, resulting in inconsistent segmentation results across the entire image. This makes it difficult to meet the requirements of remote sensing ground feature extraction tasks with high integrity requirements, and urgently needs to be addressed. Summary of the Invention
[0004] This application provides a remote sensing image interactive segmentation method and apparatus with cross-tile perception to solve the problem in related technologies that, due to the independent segmentation of images into small blocks, the spatial relationship of the overall image is easily ignored, and different small blocks cannot share prompt information, thereby affecting the segmentation consistency and accuracy of remote sensing images.
[0005] The first aspect of this application provides a cross-tile-aware remote sensing image interactive segmentation method, comprising the following steps: segmenting a remote sensing image into at least one image tile and extracting local visual features of the at least one image tile; extracting local semantic features of user interaction click locations based on the local visual features, and propagating the local semantic features to the remote sensing image to generate a cross-tile cue embedding containing global interaction information; generating basic features of the at least one image tile based on the at least one image tile and at least one auxiliary tile containing user interaction click information, and injecting the cross-tile cue embedding into the basic features to obtain a fusion feature of the at least one image tile; decoding the fusion feature to generate a local prediction mask of the at least one image tile, and stitching the local prediction mask to obtain a complete segmentation result of the remote sensing image.
[0006] Through the above technical means, the embodiments of this application can inject user interaction click information into all image tiles to obtain the complete segmentation result of the remote sensing image. This allows the interaction click information to no longer be limited to a single area, but to play a consistent and coherent guiding role across the entire image. Each tile can obtain semantic guidance from the same source, thereby effectively avoiding the problems of boundary breaks, regional discontinuities, and overall structural distortion caused by independent block segmentation, and ensuring that the segmentation results of large-format remote sensing images maintain consistency and integrity at the global level.
[0007] Optionally, in one embodiment of this application, after obtaining the complete segmentation result of the remote sensing image, the method further includes: applying multiple different segmentation configurations to the remote sensing image to obtain multiple complete segmentation results; averaging the weights of the multiple complete segmentation results; and performing weighted fusion of the multiple complete segmentation results according to the weights to output a final binary segmentation mask that meets preset high-precision conditions.
[0008] Through the above technical means, the embodiments of this application can perform weighted fusion by averaging the weights of multiple complete segmentation results. This can comprehensively improve the robustness and accuracy of the final mask by combining the advantages of different segmentation results. It can effectively reduce the impact of local errors, interactive noise or local blurred boundaries in a single prediction, and enhance the final segmentation results in terms of detail completeness, regional consistency and boundary coherence, thereby improving the reliability and stability of interactive segmentation of remote sensing images.
[0009] Optionally, in one embodiment of this application, the step of extracting local semantic features of the user interaction click location based on the local visual features includes: stitching the local visual features according to the original spatial order of the remote sensing image to obtain a full-resolution feature map of the remote sensing image; performing average pooling on the mask activation region of the user interaction click location based on the full-resolution feature map to obtain the pooling result; and calculating the local semantic features based on the pooling result.
[0010] Through the above technical means, the embodiments of this application can calculate the local semantic features of the user's interactive click position based on the result of average pooling. While maintaining the semantic representativeness of the local area, the feature dimension can be effectively compressed and noise interference can be reduced. A more stable and more aggregated local semantic expression can be obtained, thereby accurately reflecting the foreground or background intent of the user's marked position.
[0011] Optionally, in one embodiment of this application, the step of propagating the local semantic features to the remote sensing image includes: calculating the distance between the pixel position of the remote sensing image and the user interaction click position; obtaining at least one index of the user interaction click position, and assigning the pixels of the remote sensing image to the at least one index with the smallest distance according to the distance, so as to construct a recent click index map of the remote sensing image; constructing a reindexed feature map of the remote sensing image according to the recent click index map; and propagating the local semantic features to the remote sensing image according to the reindexed feature map.
[0012] Through the above technical means, the embodiments of this application can construct a reindexed feature map based on the most recently clicked index map, and then propagate local semantic features to remote sensing images. This allows each image region to be associated with the user click closest to it in terms of spatial distance or semantic relevance, thereby introducing consistent interactive guidance at the feature level. It can establish a unified prompt response mechanism between different image patches, and also enhance the semantic perception capability of regions far from the click location, ensuring stable propagation of interactive information in high-resolution, large-format scenes, and improving the coherence and accuracy of the overall segmentation results.
[0013] Optionally, in one embodiment of this application, the formula for the fusion feature is: , in, Indicates fusion characteristics, This indicates the image embedding module. Indicates an auxiliary embedded module. Represents an image patch. Indicates auxiliary blocks, This indicates that the cross-tile hint is embedded. Through the above technical means, the embodiments of this application can embed cross-tile prompts and fuse basic features, so that each tile has both local visual information and global interactive semantic information at the encoding stage, thereby achieving a unified response to user intent. This can further avoid problems such as boundary misalignment and regional fragmentation caused by independent segmentation, and help improve the overall segmentation accuracy and stability of large-format remote sensing images.
[0014] A second aspect of this application provides a remote sensing image interactive segmentation apparatus with cross-tile perception, comprising: an extraction module for segmenting a remote sensing image into at least one image tile and extracting local visual features of the at least one image tile; a propagation module for extracting local semantic features of user interaction click locations based on the local visual features and propagating the local semantic features to the remote sensing image to generate a cross-tile cue embedding containing global interaction information; an injection module for generating basic features of the at least one image tile based on the at least one image tile and at least one auxiliary tile containing user interaction click information, and injecting the cross-tile cue embedding into the basic features to obtain a fused feature of the at least one image tile; and a segmentation module for decoding the fused feature to generate a local prediction mask of the at least one image tile and stitching the local prediction mask to obtain a complete segmentation result of the remote sensing image.
[0015] Optionally, in one embodiment of this application, it further includes: a segmentation module, used to apply multiple different segmentation configurations to the remote sensing image to obtain the multiple complete segmentation results; a setting module, used to set the weights of the multiple complete segmentation results on an average basis; and a fusion module, used to perform weighted fusion of the multiple complete segmentation results according to the weights to output a final binary segmentation mask that meets the preset high-precision conditions.
[0016] Optionally, in one embodiment of this application, the propagation module includes: a stitching unit, configured to stitch the local visual features according to the original spatial order of the remote sensing image to obtain a full-resolution feature map of the remote sensing image; a pooling unit, configured to perform average pooling on the mask activation region of the user interaction click position according to the full-resolution feature map to obtain a pooling result; and a first calculation unit, configured to calculate the local semantic features according to the pooling result.
[0017] Optionally, in one embodiment of this application, the propagation module includes: a second calculation unit, configured to calculate the distance between the pixel position of the remote sensing image and the user interaction click position; a first construction unit, configured to obtain at least one index of the user interaction click position, and assign the pixels of the remote sensing image to the at least one index with the smallest distance according to the distance, so as to construct a recent click index map of the remote sensing image; a second construction unit, configured to construct a reindexed feature map of the remote sensing image according to the recent click index map; and a propagation unit, configured to propagate the local semantic features to the remote sensing image according to the reindexed feature map.
[0018] Optionally, in one embodiment of this application, the formula for the fusion feature is: , in, Indicates fusion characteristics, This indicates the image embedding module. Indicates an auxiliary embedded module. Represents an image patch. Indicates auxiliary blocks, This indicates that the cross-tile hint is embedded.
[0019] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cross-tile sensing remote sensing image interactive segmentation method as described in the above embodiments.
[0020] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described cross-tile-aware remote sensing image interactive segmentation method.
[0021] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the cross-tile sensing remote sensing image interactive segmentation method described above.
[0022] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a cross-tile sensing remote sensing image interactive segmentation method provided according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the principle of cross-tile sensing remote sensing image interactive segmentation according to an embodiment of this application; Figure 3 A flowchart of a cross-tile-aware remote sensing image interactive segmentation method according to an embodiment of this application; Figure 4 This is a block diagram of a remote sensing image interactive segmentation device with cross-tile perception provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.
[0024] Figure label: 10-Interactive segmentation device for remote sensing images with cross-tile perception; 100-Extraction module, 200-Propagation module, 300-Injection module, 400-Segmentation module; 501-Memory, 502-Processor, 503-Communication interface. Detailed Implementation
[0025] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0026] The following describes a cross-tile-aware interactive remote sensing image segmentation method and apparatus according to embodiments of this application, with reference to the accompanying drawings. Addressing the technical problem mentioned in the background art, where related techniques divide images into small, independent blocks, easily leading to the neglect of the overall spatial relationships of the image and the inability to share prompt information between different blocks, thus affecting the consistency and accuracy of remote sensing image segmentation, this application provides a cross-tile-aware interactive remote sensing image segmentation method. In this method, local semantic features of the user's interactive click location are propagated to the remote sensing image to generate a cross-tile prompt embedding containing global interactive information. This embedding is then fused with basic features to obtain fused features of the image blocks. Finally, the fused features are stitched together to obtain the complete segmentation result of the remote sensing image. This method achieves consistent user prompts across the entire image, improves the coherence and integrity of segmentation results between different blocks, and effectively improves the interactive segmentation accuracy and stability of large-format remote sensing images. It can cope with typical challenges of remote sensing images such as high information density scenes, sparse or small target distributions, complex background interference, and large-scale changes, supporting efficient and high-precision interactive remote sensing image segmentation. This solves the problem that when related technologies segment images into small, independent blocks, the spatial relationships of the overall image are easily overlooked, and different blocks cannot share information, thus affecting the consistency and accuracy of remote sensing image segmentation.
[0027] Specifically, Figure 1 This is a flowchart illustrating a cross-tile sensing remote sensing image interactive segmentation method provided in an embodiment of this application.
[0028] like Figure 1 As shown, this cross-tile-aware remote sensing image interactive segmentation method includes the following steps: In step S101, the remote sensing image is divided into at least one image patch, and local visual features of at least one image patch are extracted.
[0029] It can be noted that, to reduce the computational cost of large-format remote sensing imagery, embodiments of this application can divide the original image into several image patches of fixed size. For example, embodiments of this application can divide the entire remote sensing image along both the horizontal and vertical directions with fixed step sizes according to a set sliding window method or grid partitioning method, obtaining multiple patches of the same size. Extraction methods can utilize convolutional neural network encoders, visual Transformer encoders based on self-attention mechanisms, or hybrid coding networks, etc., through multi-layer convolution, attention computation, or feature aggregation operations to extract high-dimensional feature representations reflecting local texture, edge structure, and semantic information.
[0030] The segmentation and extraction methods can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.
[0031] As one possible approach, firstly, embodiments of this application can acquire a high information density remote sensing image to be segmented. Next, in order to process the large-scale features of remote sensing images, embodiments of this application can convert the input image... Spatially divided into Non-overlapping image patches The image tiles are then input into the target branch, and the local visual features of each tile are extracted using the image encoder. These features are then used for subsequent interactive cue fusion, feature reconstruction, or full-image segmentation prediction, so as to obtain a more refined representation of land features at the local scale.
[0032] In step S102, local semantic features of the user interaction click location are extracted based on local visual features, and the local semantic features are propagated to the remote sensing image to generate cross-tile cue embedding containing global interaction information.
[0033] User interaction clicks can include, but are not limited to, positive and negative clicks. Positive clicks indicate pixel locations that the user believes belong to the target area, while negative clicks indicate pixel locations that the user believes do not belong to the target area. Different types of clicks can be distinguished by different labeling methods and mapped to corresponding feature spaces to generate guidance information reflecting the user's intent. This application embodiment can enhance or suppress the semantic category of local areas based on the distribution of positive and negative clicks in the feature space, thereby more accurately correcting the segmentation boundary and improving the precision and reliability of the interactive segmentation results. Local semantic features include, but are not limited to, the texture of features near the click location, boundary shape, and semantic category information, providing a valid basis for subsequent interactive guidance, feature fusion, and segmentation region correction.
[0034] In the embodiments of this application, the propagation method may include, but is not limited to, global semantic association propagation based on multi-head self-attention mechanism, neighborhood feature diffusion based on graph structure, etc. Through the above propagation, the embodiments of this application can form consistent semantic guidance for the interactive prompts provided by the user in the local area across the entire map, generate cross-tile prompt embedding containing global interactive information, thereby achieving unified segmentation prediction of the entire remote sensing image and improving the consistency and accuracy of the global segmentation results.
[0035] Optionally, in one embodiment of this application, extracting local semantic features of the user interaction click location based on local visual features includes: stitching the local visual features according to the original spatial order of the remote sensing image to obtain a full-resolution feature map of the remote sensing image; performing average pooling on the mask activation region of the user interaction click location based on the full-resolution feature map to obtain the pooling result; and calculating the local semantic features based on the pooling result.
[0036] As a specific example, firstly, embodiments of this application can utilize an image embedding module that shares weights with the target branch to extract features from all patches. The features of these patches are then stitched together according to the original spatial order of the remote sensing images to reconstruct a full-resolution feature map. .
[0037] Furthermore, for a given A set of coordinates for a positive click can be represented as: ,in In this embodiment of the application, a value can be generated for each positive click. Circular binary mask centered .
[0038] To obtain semantic information about the click location, embodiments of this application can utilize full-resolution feature maps. Above, for each mask The activated region is subjected to average pooling to calculate the local feature vector corresponding to the click. The calculation formula can be shown below: , In this application, the embodiments can utilize soft space selection to make feature extraction more robust to slight click biases.
[0039] Optionally, in one embodiment of this application, propagating local semantic features to a remote sensing image includes: calculating the distance between a pixel location in the remote sensing image and a user-interacted click location; obtaining at least one index of the user-interacted click location, and assigning pixels in the remote sensing image to the at least one index with the smallest distance based on the distance, to construct a recently clicked index map of the remote sensing image; constructing a reindexed feature map of the remote sensing image based on the recently clicked index map; and propagating local semantic features to the remote sensing image based on the reindexed feature map.
[0040] To propagate sparse click information across the entire image, embodiments of this application construct a recently clicked index graph. .
[0041] Specifically, for each pixel position in the image In this application embodiment, spoken language calculation is performed and compared with all positive clicks. The Euclidean distance is calculated, and the pixel is assigned to the nearest click index. The formula is expressed colloquially as follows: , Furthermore, based on this index graph The embodiment of this application constructs a re-indexed feature map of spoken language. That is, assigning the feature of each pixel to the positive click feature vector corresponding to its region. Thus, in this embodiment of the application, the image is divided into Voronoi-like regions, each of which is filled with the semantic features of a center click.
[0042] Next, embodiments of this application can reindex the feature map. The input is encoded into a multilayer perceptron to generate cross-tile cue embeddings that contain global interaction information. Finally, the embodiments of this application can... Re-divided in space A grid is used to obtain the hint embedding for each tile. .
[0043] In step S103, basic features of at least one image patch are generated based on at least one image patch and at least one auxiliary patch containing user interaction click information, and cross-patch hints are embedded into the basic features to obtain fused features of at least one image patch.
[0044] As a specific example, firstly, embodiments of this application can convert a user's positive and negative clicks into a two-channel disk-shaped image, and then stitch the prediction mask generated from the previous interaction with this click image to form an auxiliary input. Similarly, auxiliary input Divided into images Input of auxiliary blocks .
[0045] Next, in this embodiment of the application, the divided image tiles and auxiliary tiles can be input to the target branch; for the first... Each image tile utilizes an image embedding module. Image features are extracted, and click and mask features are extracted using an auxiliary embedding module, thus forming the basic features. The formula for calculating can be shown below: , in, and It consists of convolutional layers.
[0046] Furthermore, embodiments of this application can embed the generated cross-tile hints. The features are injected into the target branch and fused with the basic features element by element to obtain the fused features of the image patch.
[0047] Optionally, in one embodiment of this application, the formula for the fusion feature can be expressed as: , in, Indicates fusion characteristics, This indicates the image embedding module. Indicates an auxiliary embedded module. Represents an image patch. Indicates auxiliary blocks, This indicates a cross-tile hint embedding.
[0048] The embodiments of this application can explicitly introduce semantic guidance information carried by user interaction prompts into the local feature representation through feature fusion, thereby enhancing the response of the target area and suppressing interference from non-target areas within the tile.
[0049] In step S104, the fusion features are decoded to generate a local prediction mask for at least one image patch, and the local prediction masks are stitched together to obtain a complete segmentation result of the remote sensing image. It can be explained that the embodiments of this application can input the fused features of cross-tile cue embedding and basic visual features into the decoding network. Through multi-scale upsampling, convolutional decoding or attention decoding and other operations, a binary or probabilistic prediction mask corresponding to the spatial size of the tile is obtained. This mask can reflect the distribution of the target area and boundary details within the tile, providing a basis for subsequent full-image mask stitching, cross-tile boundary consistency correction or full-image reconstruction and segmentation.
[0050] Specifically, embodiments of this application can incorporate fusion features. The input is fed into a backbone network (e.g., ViT (VisionTransformer)) for deep feature extraction, and then processed by a decoder (e.g., containing a feature pyramid and a segmentation head) to generate a prediction mask for each patch. : , in, Indicates decoder, This refers to the backbone network.
[0051] Finally, the embodiments of this application can predict the results of all tiles. The images are stitched together according to their original spatial layout to obtain the final complete remote sensing image segmentation result. Through the above process, the embodiments of this application can achieve synchronous segmentation on different tiles using interactive information, ensuring the global consistency of the segmentation results.
[0052] Optionally, in one embodiment of this application, after obtaining the complete segmentation result of the remote sensing image, the method further includes: applying multiple different segmentation configurations to the remote sensing image to obtain multiple complete segmentation results; averaging the weights of the multiple complete segmentation results; and performing weighted fusion of the multiple complete segmentation results according to the weights to output a final binary segmentation mask that meets the preset high-precision conditions.
[0053] The preset high-precision conditions can be conditions that the segmentation mask reaches the threshold in terms of cross-union ratio, Dice coefficient, etc., or conditions that the output pixel-level confidence meets the lower confidence limit or multi-level consistency constraints. These conditions can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.
[0054] In the embodiments of this application, in order to further improve the robustness of segmentation and solve the problems of boundary misalignment or missing contextual information that may be caused by single-scale tile partitioning, the embodiments of this application may further adopt a multi-scale segmentation fusion strategy during the inference stage, which may specifically include the following steps: (1) Multi-configuration parallel inference: For the same remote sensing image to be segmented The embodiments of this application can be set Different tile partitioning configurations (e.g., 2×2, 3×3, and 4×4 grids); for each configuration Independently execute cross-tile sensing remote sensing image interactive segmentation methods to obtain segmentation prediction results at different granularities. .
[0055] It can be explained that different partitioning configurations can capture target features and contextual ranges at different resolutions, forming complementary information.
[0056] (2) Weighted fusion: The embodiments of this application can be adapted from the above. The segmentation results under different configurations are weighted and summed to obtain the final fused segmentation result. The fusion calculation formula can be shown below: , in, Indicates the first The fusion weights corresponding to the configurations, and satisfying the normalization constraints. as well as .
[0057] In one embodiment of this application, an average fusion strategy can be adopted, that is, all configurations have equal weights (e.g., when...). hour, This fusion mechanism effectively suppresses prediction noise caused by single partitioning, resulting in a segmentation mask with smoother boundaries and higher accuracy.
[0058] like Figure 2 As shown below, a specific example will be used to illustrate the principle of cross-tile sensing remote sensing image interactive segmentation in this application embodiment.
[0059] In the embodiments of this application, the upper part of the graph is a cross-tile branch: it can be used to generate cross-tile cue embeddings containing global interaction information. This branch first inputs the segmented tile image into a shared backbone network to extract features; then, using the recently clicked index map, it constructs reindexed features based on the click position, thereby generating cross-tile cue embeddings.
[0060] The lower half of the graph represents the target branch, which can be used to perform specific segmentation predictions. This branch receives user interaction click information and image features, and embeds cross-tile cues from the upper half for fusion. The fused features are then processed by a shared backbone network to extract deep features, and finally input into a decoder containing a feature pyramid and a segmentation head, outputting the final tile segmentation result.
[0061] Furthermore, such as Figure 3 As shown, the cross-tile sensing remote sensing image interactive segmentation method of this application embodiment is further explained. The embodiments of this application may include the following steps: In step S301, patching and basic feature extraction are performed: First, embodiments of this application can acquire the remote sensing image to be segmented and the user's interactive click information (positive and negative clicks), and divide the image into... The image is divided into several patches, and the image patches are input into the target branch. The local visual features of each patch are extracted using an image encoder.
[0062] In step S302, cross-tile hint embedding is generated: The embodiments of this application can extract local semantics around the positive click using the reconstructed full-resolution features, and propagate sparse click features to the entire map through the recent click index map to generate cross-tile hint embeddings containing global interaction information, thereby solving the problem of information isolation between tiles.
[0063] In step S303, feature fusion and mask prediction: In this embodiment, the generated cross-tile hints can be embedded into the corresponding tile basic features for fusion. The decoder processes the fused features to generate a local prediction mask for each tile, and then stitches them together in spatial order to obtain a complete segmentation result.
[0064] In step S304, multi-scale fusion and output: The embodiments of this application can employ a multi-scale segmentation and fusion strategy, applying multiple different segmentation configurations to the same image and repeating the above process, and weighting and fusing the segmentation results of different granularities to output the final high-precision binary segmentation mask.
[0065] The cross-tile-aware remote sensing image interactive segmentation method proposed in this application can propagate the local semantic features of the user's interactive click location to the remote sensing image to generate a cross-tile prompt embedding containing global interactive information. This embedding is then fused with basic features to obtain the fused features of the image tiles. Finally, the fused features are stitched together to obtain the complete segmentation result of the remote sensing image. This method can achieve consistent user prompts across the entire image, improve the coherence and integrity of segmentation results between different tiles, and effectively improve the interactive segmentation accuracy and stability of large-format remote sensing images. It can address typical challenges of remote sensing images such as high information density scenes, sparse or small target distribution, complex background interference, and large-scale changes, and supports efficient and high-precision interactive remote sensing image segmentation.
[0066] Next, referring to the accompanying drawings, a remote sensing image interactive segmentation apparatus for cross-tile perception according to an embodiment of this application is described.
[0067] Figure 4 This is a block diagram of a remote sensing image interactive segmentation device with cross-tile perception according to an embodiment of this application.
[0068] like Figure 4 As shown, the remote sensing image interactive segmentation device 10 with cross-plot perception includes: an extraction module 100, a propagation module 200, an injection module 300, and a segmentation module 400.
[0069] The extraction module 100 is used to divide the remote sensing image into at least one image patch and extract the local visual features of at least one image patch.
[0070] The propagation module 200 is used to extract local semantic features of the user interaction click position based on local visual features, and propagate the local semantic features to the remote sensing image to generate cross-tile cue embedding containing global interaction information.
[0071] The injection module 300 is used to generate basic features of at least one image tile based on at least one image tile and at least one auxiliary tile containing user interaction click information, and to embed cross-tile prompts into the basic features to obtain fused features of at least one image tile.
[0072] The segmentation module 400 is used to decode the fusion features to generate a local prediction mask for at least one image patch, and to stitch the local prediction masks together to obtain a complete segmentation result of the remote sensing image.
[0073] Optionally, in one embodiment of this application, it further includes: a segmentation module, a setting module, and a fusion module.
[0074] The segmentation module is used to apply various segmentation configurations to remote sensing images to obtain a variety of complete segmentation results.
[0075] The setting module is used to average the weights of multiple complete segmentation results.
[0076] The fusion module is used to perform weighted fusion of multiple complete segmentation results according to weights, so as to output a final binary segmentation mask that meets the preset high-precision conditions.
[0077] Optionally, in one embodiment of this application, the propagation module 200 includes: a splicing unit, a pooling unit, and a first computing unit.
[0078] The stitching unit is used to stitch together local visual features according to the original spatial order of the remote sensing image to obtain a full-resolution feature map of the remote sensing image.
[0079] The pooling unit is used to perform average pooling on the mask activation region of the user interaction click position based on the full-resolution feature map to obtain the pooling result.
[0080] The first computational unit is used to calculate local semantic features based on the pooling results.
[0081] Optionally, in one embodiment of this application, the propagation module 200 includes: a second computing unit, a first building unit, a second building unit, and a propagation unit.
[0082] The second calculation unit is used to calculate the distance between the pixel position of the remote sensing image and the user's interactive click position.
[0083] The first building unit is used to obtain at least one index of the user interaction click position and assign the pixels of the remote sensing image to the at least one index with the smallest distance according to the distance, so as to build a recent click index map of the remote sensing image.
[0084] The second building unit is used to construct a reindexed feature map of the remote sensing image based on the most recently clicked index map.
[0085] The propagation unit is used to propagate local semantic features to the remote sensing image based on the reindexed feature map.
[0086] Optionally, in one embodiment of this application, the formula for fusing features is: , in, Indicates fusion characteristics, This indicates the image embedding module. Indicates an auxiliary embedded module. Represents an image patch. Indicates auxiliary blocks, This indicates a cross-tile hint embedding.
[0087] It should be noted that the foregoing explanation of the cross-tile sensing remote sensing image interactive segmentation method embodiment also applies to the cross-tile sensing remote sensing image interactive segmentation device of this embodiment, and will not be repeated here.
[0088] The cross-tile-aware remote sensing image interactive segmentation device proposed in this application can propagate the local semantic features of the user's interactive click location to the remote sensing image to generate a cross-tile prompt embedding containing global interactive information. This embedding is then fused with basic features to obtain the fused features of the image tiles. Finally, the fused features are stitched together to obtain the complete segmentation result of the remote sensing image. This device can achieve consistent user prompts across the entire image, improve the coherence and integrity of segmentation results between different tiles, and effectively improve the interactive segmentation accuracy and stability of large-format remote sensing images. It can address typical challenges of remote sensing images such as high information density scenes, sparse or small target distribution, complex background interference, and large-scale changes, and supports efficient and high-precision interactive remote sensing image segmentation.
[0089] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.
[0090] When the processor 502 executes the program, it implements the remote sensing image interactive segmentation method with cross-tile perception provided in the above embodiments.
[0091] Furthermore, electronic devices also include: Communication interface 503 is used for communication between memory 501 and processor 502.
[0092] The memory 501 is used to store computer programs that can run on the processor 502.
[0093] Memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0094] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0095] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.
[0096] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.
[0097] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described cross-tile sensing remote sensing image interactive segmentation method.
[0098] This application also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the cross-tile sensing remote sensing image interactive segmentation method provided in this application.
[0099] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0100] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0101] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0102] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0103] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0104] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0106] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.
Claims
1. A cross-tile sensing remote sensing image interactive segmentation method, characterized in that, Includes the following steps: The remote sensing image is divided into at least one image patch, and the local visual features of the at least one image patch are extracted; Based on the local visual features, local semantic features of the user interaction click location are extracted, and the local semantic features are propagated to the remote sensing image to generate cross-tile prompt embedding containing global interaction information; Based on the at least one image tile and at least one auxiliary tile containing user interaction click information, the basic features of the at least one image tile are generated, and the cross-tile prompt is embedded and injected into the basic features to obtain the fused features of the at least one image tile; The fusion features are decoded to generate local prediction masks for the at least one image patch, and the local prediction masks are stitched together to obtain the complete segmentation result of the remote sensing image.
2. The method according to claim 1, characterized in that, After obtaining the complete segmentation result of the remote sensing image, the process also includes: Multiple different segmentation configurations are applied to the remote sensing image to obtain the various complete segmentation results; The weights of the various complete segmentation results are set on an average basis; The multiple complete segmentation results are weighted and fused according to the weights to output a final binary segmentation mask that meets the preset high-precision conditions.
3. The method according to claim 1, characterized in that, The step of extracting local semantic features of the user interaction click location based on the local visual features includes: The local visual features are stitched together according to the original spatial order of the remote sensing image to obtain the full-resolution feature map of the remote sensing image; The mask activation region at the user interaction click location is averaged based on the full-resolution feature map to obtain the pooling result. The local semantic features are calculated based on the pooling results.
4. The method according to claim 1, characterized in that, The step of propagating the local semantic features to the remote sensing image includes: Calculate the distance between the pixel position of the remote sensing image and the user's interactive click position; Obtain at least one index of the user interaction click location, and assign the pixels of the remote sensing image to the at least one index with the smallest distance according to the distance, so as to construct the recent click index map of the remote sensing image; Construct a reindexed feature map of the remote sensing image based on the recently clicked index map; The local semantic features are propagated to the remote sensing image based on the reindexed feature map.
5. The method according to claim 1, characterized in that, The formula for the fusion feature is: ， in, Indicates fusion features, This indicates the image embedding module. Indicates an auxiliary embedded module, Represents an image patch. Indicates auxiliary blocks, This indicates that the cross-tile hint is embedded.
6. A remote sensing image interactive segmentation device with cross-tile sensing, characterized in that, include: An extraction module is used to segment a remote sensing image into at least one image patch and extract local visual features of the at least one image patch; The propagation module is used to extract local semantic features of the user interaction click position based on the local visual features, and propagate the local semantic features to the remote sensing image to generate cross-tile prompt embedding containing global interaction information; An injection module is configured to generate basic features of the at least one image patch based on the at least one image patch and at least one auxiliary patch containing user interaction click information, and to embed the cross-patch prompt into the basic features to obtain the fused features of the at least one image patch; The segmentation module is used to decode the fusion features to generate a local prediction mask for the at least one image patch, and to stitch the local prediction mask together to obtain a complete segmentation result of the remote sensing image.
7. The apparatus according to claim 6, characterized in that, Also includes: The segmentation module is used to apply various segmentation configurations to the remote sensing image to obtain the various complete segmentation results; A setting module is used to set the weights of the various complete segmentation results on an average basis; The fusion module is used to perform weighted fusion of the multiple complete segmentation results according to the weights, so as to output a final binary segmentation mask that meets the preset high-precision conditions.
8. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the cross-tile-aware remote sensing image interactive segmentation method as described in any one of claims 1-5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the cross-tile-aware remote sensing image interactive segmentation method as described in any one of claims 1-5.
10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the remote sensing image interactive segmentation method with cross-tile perception as described in any one of claims 1-5.