A remote sensing image segmentation method based on wavelet band enhancement and related equipment

Abstract

The application relates to the technical field of remote sensing image segmentation, and provides a remote sensing image segmentation method based on wavelet band enhancement and related equipment, which comprises the following steps: performing wavelet transform decomposition on a target remote sensing image to obtain a plurality of band components of the target remote sensing image; performing feature projection on each band component to obtain a corresponding band feature of each band component; splicing all the band features to obtain frequency volume information, and performing deep feature interaction and fusion on the frequency volume information to obtain final features of the target remote sensing image; and performing mask segmentation based on the final features to obtain a mask segmentation result of the target remote sensing image. The method can improve the precision of interactive segmentation of remote sensing images.

Description

Technical Field

[0001] This application relates to the field of remote sensing image segmentation technology, and in particular to a remote sensing image segmentation method and related equipment based on wavelet band enhancement. Background Technology

[0002] With the rapid development of deep learning technology, interactive image masking segmentation technology is playing an increasingly important role in the field of remote sensing image analysis. The development of this technology has mainly gone through the following stages: traditional interactive segmentation, deep learning-driven interactive segmentation, and fine-tuning of the visual base model.

[0003] Traditional interactive image masking segmentation methods primarily rely on user-provided interactive cues, such as bounding boxes, click points, and contours, combined with low-level features of the image itself, such as color, texture, and edges, for masking segmentation. Representative methods include the graph cut algorithm GraphCut, the Boykov-Kolmogorov algorithm, and the grab-and-segmentation algorithm GrabCut. These methods transform the segmentation problem into an energy minimization problem by constructing an energy function, and then solve it using the minimum cut algorithm in graph theory. However, traditional methods heavily rely on manually designed features, making them difficult to adapt to complex scenes and diverse targets, and performing particularly poorly when processing remote sensing images. With the rise of deep learning technology, interactive segmentation methods based on convolutional neural networks (CNNs) have gradually become mainstream. Network architectures such as DeepLab semantic segmentation network, U-Net (U-Shaped Convolutional Neural Network), and Mask R-CNN (Mask Region-based Convolutional Neural Network) have achieved significant results in image segmentation tasks. These methods automatically extract image features through end-to-end learning, overcoming the limitations of traditional methods that rely on manual features. Some studies combine interactive cues (such as click points and bounding boxes) with deep networks to develop various interactive segmentation frameworks, such as PointRend (Point-based Rendering), Segmenter network, and SimpleClick interactive segmentation algorithm. These methods have shown excellent performance on natural image segmentation tasks, but still face challenges when applied to mask segmentation of remote sensing images.

[0004] In recent years, large-scale pre-trained visual foundational models have become a research hotspot. Models such as Contrastive Language-Image Pre-training (CLIP), Unsupervised Visual Distillation (DINO), Masked Autoencoder (MAE), and Segment Anything Model (SAM) have learned rich, general visual representations through pre-training on massive amounts of data. To adapt these foundational models to specific tasks, researchers have proposed various fine-tuning strategies, including full-parameter fine-tuning, Low-Rank Adaptation (LoRA), Adapter modules, and Prompt Learning. SAM, as a general image segmentation foundational model proposed by Meta AI, has achieved breakthrough results on multiple segmentation tasks thanks to its powerful zero-shot generalization ability and flexible prompting mechanism. For applications in specific domains, researchers have attempted to fine-tune SAM to improve its segmentation performance in specific scenarios.

[0005] However, interactive mask segmentation of remote sensing images has its own unique characteristics. Remote sensing images differ significantly from natural images, mainly in the following aspects: (1) large imaging scale and wide coverage, with dramatic changes in the size of ground objects; (2) rich spectral information but relatively low spatial resolution; (3) blurred boundaries and complex texture features of ground objects; (4) diverse features of the same type of ground objects in different regions. The model is difficult to adapt to the mask segmentation task, resulting in low accuracy of mask segmentation of remote sensing images. Summary of the Invention

[0006] This application provides a remote sensing image segmentation method and related equipment based on wavelet band enhancement, which can solve the problem of low accuracy in remote sensing image mask segmentation.

[0007] In a first aspect, embodiments of this application provide a remote sensing image segmentation method based on wavelet band enhancement, the remote sensing image segmentation method comprising: Wavelet transform decomposition is performed on the target remote sensing image to obtain multiple frequency band components of the target remote sensing image; the frequency band components are the components of the target remote sensing image under different frequency bands. For each frequency band component, feature projection is performed to obtain the frequency band features corresponding to each frequency band component; All frequency band features are stitched together to obtain frequency volume information, and deep feature interaction and fusion are performed on the frequency volume information to obtain the final features of the target remote sensing image; the frequency volume information is used to describe the frequency domain attributes of remotely sensed land objects in the target remote sensing image. Mask segmentation is performed based on the final features to obtain the mask segmentation result of the target remote sensing image.

[0008] Optionally, feature projection is performed on each frequency band component to obtain the frequency band features corresponding to each frequency band component, including: For each frequency band component, perform the following steps: The initial frequency band features are obtained by performing feature projection on the frequency band components; Add the corresponding frequency band embedding to the initial frequency band features to obtain the frequency band features.

[0009] Optionally, feature projection is performed on the frequency band components to obtain initial frequency band features, including: Through the formula:

[0010] Perform feature projection to obtain the initial frequency band features. ; in, Indicates frequency band components. This represents pointwise convolution. This represents the activation function. Represents depthwise convolution. The dimension of the initial frequency band features. For training batch size, Image length, Image width, For depth dimension.

[0011] Optionally, the corresponding frequency band embedding is added to the initial frequency band features to obtain frequency band features, including: Through the formula:

[0012] Calculate frequency band characteristics ; in, This indicates frequency band embedding.

[0013] Optionally, deep feature interaction and fusion are performed on the frequency volume information to obtain the final features of the target remote sensing image, including: Based on frequency volume information, depth feature extraction and dimensional projection are performed to obtain frequency domain features; Spatial domain features are defined, and frequency domain features are fused with spatial domain features to obtain the final features of the target remote sensing image.

[0014] Optionally, depth feature extraction and dimensional projection are performed based on frequency volume information to obtain frequency domain features, including: The frequency volume information is downsampled to obtain the downsampled features; The downsampled features are augmented and self-attention operations are performed to obtain the enhanced features. The enhanced features are projected dimensionally to obtain the frequency domain features.

[0015] Optionally, frequency domain features and spatial domain features are fused to obtain the final features of the target remote sensing image, including: The frequency domain features are upsampled and flattened to obtain the upsampled frequency domain features. Based on the upsampled frequency domain and spatial domain features, the gating weights are calculated. Based on the gating weights, the upsampled frequency domain features and spatial domain features are fused to obtain the final features of the target remote sensing image.

[0016] Optionally, based on the upsampled frequency domain features and spatial domain features, the gating weights are calculated, including: Through the formula:

[0017]

[0018] Calculate the gating weight ; in, Indicates splicing characteristics, Representing spatial domain characteristics, This represents the frequency domain characteristics after upsampling. This indicates concatenation along the channel dimension. Indicates a fully connected layer. Indicates the gated activation function; Based on the gating weights, the upsampled frequency domain features and spatial domain features are fused to obtain the final features of the target remote sensing image, including: Through the formula:

[0019]

[0020] Calculate the final features ; in, This represents the frequency domain characteristics after projection. Indicates the projection layer.

[0021] Secondly, embodiments of this application provide a remote sensing image segmentation device based on wavelet band enhancement, comprising: The decomposition module is used to perform wavelet transform decomposition on the target remote sensing image to obtain multiple frequency band components of the target remote sensing image; the frequency band components are the components of the target remote sensing image under different frequency bands. The projection module is used to perform feature projection on each frequency band component to obtain the frequency band features corresponding to each frequency band component. The fusion module is used to stitch together all frequency band features to obtain frequency volume information, and to perform deep-dimensional feature interaction and fusion on the frequency volume information to obtain the final features of the target remote sensing image; the frequency domain volume information is used to describe the frequency domain attributes of remote sensing objects in the target remote sensing image. The segmentation module is used to perform mask segmentation based on the final features to obtain the mask segmentation result of the target remote sensing image.

[0022] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned remote sensing image segmentation method based on wavelet band enhancement.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned remote sensing image segmentation method based on wavelet band enhancement.

[0024] The above-mentioned solution in this application has the following beneficial effects: In the embodiments of this application, wavelet transform decomposition is performed on the target remote sensing image to obtain multiple frequency band components. Then, feature projection is performed on each frequency band component to obtain the corresponding frequency band features. All frequency band features are then stitched together to obtain frequency volume information. Deep feature interaction and fusion are then performed on the frequency volume information to obtain the final features of the target remote sensing image. Finally, mask segmentation is performed based on the final features to obtain the mask segmentation result of the target remote sensing image. Specifically, wavelet transform decomposition of the target remote sensing image decomposes it into multiple vectors, enabling a full and comprehensive analysis of information in different frequency bands. This increases the comprehensiveness and effectiveness of the frequency volume information obtained based on the frequency band components. Mask segmentation of the remote sensing image based on the comprehensive fusion features effectively improves the accuracy of remote sensing image mask segmentation.

[0025] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating a remote sensing image segmentation method based on wavelet band enhancement provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a remote sensing image segmentation device based on wavelet band enhancement provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0029] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0030] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0031] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0032] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0033] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0034] To address the low accuracy of existing remote sensing image masking segmentation methods, this application provides a remote sensing image segmentation method based on wavelet band enhancement. This method decomposes the target remote sensing image using wavelet transform to obtain multiple frequency band components. Then, it performs feature projection on each frequency band component to obtain its corresponding frequency band features. All frequency band features are then stitched together to obtain frequency volume information. This frequency volume information is then subjected to deep feature interaction and fusion to obtain the final features of the target remote sensing image. Finally, masking segmentation is performed based on these final features to obtain the masked segmentation result of the target remote sensing image. The wavelet transform decomposition of the target remote sensing image decomposes it into multiple frequency components, enabling a full and comprehensive analysis of information in different frequency bands. This increases the comprehensiveness and effectiveness of the frequency volume information obtained from the frequency band components. Performing masking segmentation on the remote sensing image based on this comprehensive frequency volume information effectively improves the accuracy of remote sensing image masking segmentation.

[0035] The following is an illustrative example of the remote sensing image segmentation method based on wavelet band enhancement provided in this application.

[0036] like Figure 1 As shown, the remote sensing image segmentation method based on wavelet band enhancement provided in this application includes the following steps: Step 11: Perform wavelet transform decomposition on the target remote sensing image to obtain multiple frequency band components of the target remote sensing image.

[0037] The aforementioned target remote sensing image is a remote sensing image that needs to be masked and segmented, such as a remote sensing image of a certain region. The aforementioned frequency band components are the frequency domain components of the target remote sensing image under different frequency bands, including low-frequency approximation components, horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components.

[0038] For example, multiple wavelet basis functions are simultaneously applied to the target remote sensing image for stationary wavelet transform decomposition, resulting in multi-level stationary wavelet transform results. Each level has the four frequency band components mentioned above. Then, according to the type of frequency band components, all frequency band components of each type are integrated into a vector to obtain the frequency band components. For example: Let the set of wavelet bases used be W = {w1, w2, ..., wk}, where k is the number of wavelet types (in this embodiment, k=3, including Haar, db2, and sym2). For each wavelet base wi, an L-level stationary wavelet transform is performed to obtain four frequency band components for each level: low-frequency approximation component: LLj = DWTLL(Xj), horizontal high-frequency component: LHj = DWTLH(Xj), vertical high-frequency component: HLj = DWTHL(Xj), and diagonal high-frequency component: HHj = DWTHH(Xj). The stationary wavelet transform employs an undecimated decomposition strategy to maintain the output resolution of each level consistent with the input, i.e.: Size(LLj) = 1. j ) = Size(LH j ) = Size(HL j ) = Size(HH j ) = Size(X j ) j ∈ {0, 1, ..., L-1}.

[0039] Taking the Haar wavelet basis as an example, the decomposition filter is defined as: low-pass filter: h = [ , High-pass filter: g = [- , Row direction decomposition, for any position (a, b), where a = 1, 2, ..., w, b = 1, 2, ..., h / 2:

[0040]

[0041] Column-direction decomposition, for any position (a, b), where a = 1, 2, ..., w / 2, b = 1, 2, ..., h / 2:

[0042]

[0043]

[0044]

[0045] in, and These represent the low-frequency and high-frequency components after row direction decomposition, respectively. , , , These represent the four frequency band components after column direction decomposition.

[0046] Specifically, multi-wavelet decomposition: Let the input image be... For the wavelet type set Wavelets = {Haar, db2, sym2}, the decomposition level L = 2, and stationary wavelet transform decomposition is performed on the input image using three wavelet bases:

[0047] in, Indicates the first The first wavelet of the small wavelet basis Layer-stationary wavelet transform. Remote sensing imagery of the target.

[0048] Multi-scale frequency information preservation: Due to the use of stationary wavelet transform (SWT), the spatial resolution of the output of each decomposition layer remains unchanged, that is:

[0049] This feature ensures the consistency of frequency domain features and spatial domain features in terms of resolution, facilitating subsequent cross-domain fusion operations.

[0050] It should be noted that before performing this step, the target remote sensing image can be preprocessed, such as normalization and size filling.

[0051] Step 12: Perform feature projection on each frequency band component to obtain the frequency band features corresponding to each frequency band component.

[0052] In some embodiments of this application, the step of performing feature projection on each frequency band component to obtain the frequency band features corresponding to each frequency band component includes: For each frequency band component, perform the following steps: The first step is to perform feature projection on the frequency band components to obtain the initial frequency band features.

[0053] Specifically, through the formula:

[0054] Perform feature projection to obtain the initial frequency band features. .

[0055] in, Indicates frequency band components. This represents pointwise convolution. This represents the activation function. Represents depthwise convolution. The dimension of the initial frequency band features. For training batch size, Image length, Image width, The depth dimension represents different wavelets / bands / levels.

[0056] For example, the above formula is applied to all four frequency band components recorded in step 11. The specific expression is as follows:

[0057]

[0058]

[0059]

[0060] in, The initial frequency band characteristics of the low-frequency approximation component. This represents the initial frequency band characteristics of the horizontal high-frequency components. This represents the initial frequency band characteristics of the vertical high-frequency components. This represents the initial frequency band characteristics of the diagonal high-frequency components.

[0061] The second step is to add the corresponding frequency band embedding to the initial frequency band features to obtain the frequency band features.

[0062] Specifically, through the formula:

[0063] Calculate frequency band characteristics .

[0064] in, This indicates frequency band embedding.

[0065] For example, the above frequency band embedding is a pre-defined embedding representation, specifically: Embedding table initialization: Create a three-dimensional learnable embedding table with dimensions [num_wavelets × num_levels × num_bands × Db], where num_wavelets is the number of wavelet bases and num_levels is the number of wavelet decomposition levels. Embedding values ​​are initialized using a zero-mean normal distribution.

[0066] Where k is the number of wavelet types, L is the number of decomposition layers, 4 is the number of frequency bands, and Db is the embedding dimension per frequency band.

[0067] Frequency band index mapping establishes a mapping relationship between frequency band names and indices:

[0068]

[0069] Embedding lookup and addition: For each initial frequency band feature, the corresponding embedding vector is found based on its wavelet type, decomposition level and frequency band name, and then added to the projected feature using the above expression.

[0070] Step 13: stitch together all frequency band features to obtain frequency volume information, and perform deep feature interaction and fusion on the frequency volume information to obtain the final features of the target remote sensing image.

[0071] The aforementioned frequency volume information is used to describe the frequency domain attributes of remotely sensed ground features (such as buildings and public facilities) in the target remote sensing image. This step is to perform deep feature interaction and fusion at different wavelet / band / level dimensions on the frequency volume information.

[0072] In some embodiments of this application, the steps of performing deep feature extraction and fusion of frequency volume information to obtain the final features of the target remote sensing image include: The first step is to perform deep feature interaction and dimensional projection based on frequency volume information to obtain frequency domain features.

[0073] Specifically, the frequency volume information is downsampled to obtain downsampled features; feature enhancement and self-attention operations are performed on the downsampled features to obtain enhanced features; and dimensional projection is performed on the enhanced features to obtain frequency domain features.

[0074] For example, the downsampling process includes: Spatial downsampling factor selection: Based on the input image size and computational resource constraints, the spatial downsampling factor s is set to 16. For a 1024×1024 input image, the downsampled size is 64×64.

[0075] Adaptive average pooling: Use adaptive average pooling to reduce the spatial dimension of the frequency volume to [H / s, W / s]:

[0076] Bilinear interpolation restoration: Use bilinear interpolation to restore the pooled features to the target size.

[0077] Depthwise separable convolution: Applying depthwise separable convolution for intra-band feature extraction:

[0078]

[0079]

[0080] Cross-band attention computation: Applying a multi-head self-attention mechanism to achieve cross-band feature interaction:

[0081]

[0082]

[0083] Where M = (H / s)×(W / s).

[0084] The process of feature enhancement and self-attention operation described above is as follows: Depthwise separable convolution feature extraction: Applying depthwise separable convolution to the downsampled frequency volume to enhance intra-band features:

[0085] Cross-band self-attention: Flatten the convolutionally enhanced features into a sequence format and perform cross-band self-attention operations.

[0086]

[0087]

[0088] The process of the above dimensional projection is as follows: The enhanced frequency features are projected onto the same embedding dimension D as the spatial domain features:

[0089] Here, Linear represents a fully connected layer, and MultiHeadAttention represents a multi-head self-attention mechanism.

[0090] The second step is to define the spatial domain features and fuse the frequency domain features with the spatial domain features to obtain the final features of the target remote sensing image.

[0091] The aforementioned spatial domain features are obtained from the image encoder of the SAM model, where the spatial domain features X ∈ , where N = H×W is the number of spatial locations and D is the feature dimension.

[0092] Specifically, the frequency domain features are upsampled and flattened to obtain the upsampled frequency domain features; based on the upsampled frequency domain features and spatial domain features, the gating weights are calculated; according to the gating weights, the upsampled frequency domain features and spatial domain features are fused to obtain the final features of the target remote sensing image.

[0093] For example, through the formula:

[0094]

[0095] Calculate the gating weight .

[0096] in, Indicates splicing characteristics, Representing spatial domain characteristics, This represents the frequency domain characteristics after upsampling. This indicates concatenation along the channel dimension. Indicates a fully connected layer. This represents the gating activation function.

[0097] Through the formula:

[0098]

[0099] Calculate the final features .

[0100] in, This represents the frequency domain characteristics after projection. Indicates the projection layer.

[0101] The expressions for the above upsampling and feature flattening are: Frequency feature upsampling: interpolating the sequence length of the frequency sequence feature F to be the same as that of the spatial domain feature X.

[0102] Frequency sequence feature extraction: Flatten the frequency features output from the depth axis mixer into a sequence format:

[0103] Step 14: Perform mask segmentation based on the final features to obtain the mask segmentation result of the target remote sensing image.

[0104] The above mask segmentation results are masks of remotely sensed ground features in the target remote sensing image.

[0105] For example, the enhanced features can be processed using SAM to obtain the mask segmentation result of the target remote sensing image. The fused features are then input into the Vision Transformer encoder of SAM for further processing. The encoder consists of multiple Transformer Blocks, each containing a Window Multi-Head Attention (WMA) and a feedforward network (MLP). Adaptive gated fusion modules are inserted at specific layers (layers 4, 8, and 12) to achieve progressive injection of frequency enhancement information.

[0106] For the i-th Transformer Block of the encoder, its output feature X_i is:

[0107] When i∈{3, 7, 11}, inject frequency enhancement information:

[0108] Here, Freq_seq represents the frequency sequence features output by the frequency encoder, and Fusion represents the adaptive gating fusion operation.

[0109] Finally, the SAM Mask Decoder combines the image embedding output from the encoder with the cue embeddings (points, boxes, etc.) to generate the final segmentation mask prediction. The cue embedding includes information such as bounding box coordinates and click point positions. After being encoded by the PromptEncoder, it is fused with the image embedding, and the Mask Decoder ultimately predicts the segmentation mask.

[0110] It is worth mentioning that wavelet transform decomposition of the target remote sensing image can decompose the target remote sensing image into multiple vectors, enabling a full and comprehensive analysis of the information of the target remote sensing image in different frequency bands. This improves the comprehensiveness and effectiveness of the frequency volume information obtained based on the frequency band components. Masking the remote sensing image based on the comprehensive frequency volume information effectively improves the accuracy of remote sensing image masking segmentation.

[0111] Furthermore, this application has the following advantages: Advantage 1: Compared with single spatial domain segmentation methods, this application introduces a multi-wavelet frequency domain enhancement mechanism. Existing interactive segmentation methods (such as SAM) mainly rely on spatial domain features for segmentation decisions, which limits the utilization of the rich frequency information contained in remote sensing images. This application simultaneously uses multiple wavelet basis functions such as Haar, db2, and sym2 for parallel decomposition, constructing a multi-scale, multi-directional frequency representation system. The Haar wavelet has the most compact time-domain support, suitable for capturing high-frequency details; the db2 wavelet has good frequency localization characteristics, suitable for capturing mid-frequency texture information; the sym2 wavelet has symmetry, which can reduce phase distortion. The synergistic effect of the three wavelets enables the model to more comprehensively perceive the frequency characteristics of ground targets, significantly improving segmentation accuracy under the same interactive cues.

[0112] Advantage 2: Compared with global cross-band attention methods, this application proposes a depth-axis mixer to achieve memory optimization. Existing attention mechanisms for cross-band feature interaction face severe computational resource consumption when processing high-dimensional frequency volumes, making them difficult to apply, especially in interactive segmentation scenarios requiring real-time response. This application, through spatial downsampling (s=16) and a grouped attention strategy, reduces the sequence length of attention computation from H×W to (H / s)×(W / s) while maintaining cross-band information interaction capabilities. This reduces memory usage by approximately 75% and improves computational efficiency by approximately 4 times, enabling the frequency enhancement mechanism to run in real-time in interactive segmentation scenarios.

[0113] Advantage 3: Compared with simple feature fusion strategies, this application designs an adaptive gating fusion module. Existing technologies typically employ simple channel splicing or element-wise addition for spatial and frequency domain feature fusion, lacking a deep interactive mechanism. This application achieves adaptive fusion of spatial and frequency domain features by learning dynamic gating weights, which can automatically adjust the fusion intensity according to different image content, maintaining high segmentation accuracy while providing better interpretability and adaptability.

[0114] Advantage 4: Compared to global cross-band attention, this application proposes a depth-axis mixer to achieve memory optimization. Cross-band feature interaction attention mechanisms face severe computational resource consumption when processing high-dimensional frequency volumes. This application, through spatial downsampling (s=16) and a grouped attention strategy, reduces the sequence length of attention computation from H×W to (H / s)×(W / s) while maintaining cross-band information interaction capabilities, resulting in approximately 75% less memory usage and approximately 4 times improved computational efficiency.

[0115] Advantage 5: Compared with the single-step fusion strategy, this application achieves progressive injection of frequency enhancement information. This application inserts adaptive gated fusion modules at multiple levels (layers 4, 8, and 12) of the SAM ViT encoder, avoiding the information overload problem caused by single-step fusion, and enabling frequency information to be gradually integrated into the spatial domain feature extraction process, thereby improving the effectiveness of feature fusion.

[0116] The method of this application will be illustrated below with a specific example.

[0117] The iSAID-50% dataset was used for both training and validation. iSAID (Instance Segmentation in Aerial Images Dataset) is a widely used instance segmentation dataset in the field of remote sensing, encompassing various typical land cover categories such as aircraft, ships, storage tanks, baseball fields, tennis courts, basketball courts, athletic fields, ports, bridges, and vehicles. The iSAID-50% subset contains approximately 2500+ pairs of remote sensing images and their corresponding segmentation annotation samples. Each image is 1024×1024 pixels in size and contains a rich collection of land cover instances. Experiments were conducted according to the official iSAID-50% training and validation set ratio.

[0118] Interactive prompt type: (1) Bounding Box Prompt: The target bounding box is segmented, and the box coordinates are determined by the bounding rectangle of the target.

[0119] (2) Multi-Click Prompt: The number of clicks is adaptively generated based on the target size. Small objects generate 1 click, medium objects generate 2 clicks, and large objects generate 3 clicks. Each click includes a positive click (located inside the target) and a negative click (located outside the target). The generation strategy for positive and negative clicks is as follows: Positive click generation: For a target region, the geometric center of the target is extracted based on connected component analysis, and the number of positive clicks is determined according to the target size strategy. Positive clicks are located at multiple geometric center positions of the target.

[0120] Negative click generation: Negative click points are sampled at a certain distance outside the target boundary. The negative click sampling area is determined through morphological dilation operations, ensuring that the negative click maintains a specified pixel distance from the target boundary.

[0121] Using the original SAM model as a baseline, fine-tuning was performed on the same iSAID-50% training set, and evaluation was conducted on the validation set. All comparative experiments employed the same training hyperparameters and data augmentation strategies to ensure fairness. Experimental results show that for bounding box cue segmentation, the fine-tuned model achieved an Intersection over Union (IoU) increase of 7.37%, a Dice Similarity Coefficient (DICE) increase of 5.11%, Precision increase of 1.54%, Recall increase of 7.41%, and F1-Score increase of 5.11%. For multi-click cue segmentation, the fine-tuned model achieved an IoU increase of 5.09%, DICE increase of 4.44%, Precision increase of 3.25%, Recall increase of 3.48%, and F1-Score increase of 4.44%. The combined quantitative and qualitative analyses validate the effectiveness and advancement of the proposed multi-wavelet frequency domain augmentation method for interactive remote sensing image segmentation. This method significantly improves segmentation accuracy in both box-hint and multi-click interaction modes, particularly excelling in regions with blurred boundaries and complex textures. Experimental results on the iSAID-50% dataset demonstrate that this application possesses good generalization performance and practical value for large-scale remote sensing image segmentation tasks.

[0122] The following is an exemplary description of the remote sensing image segmentation device based on wavelet band enhancement provided in this application.

[0123] like Figure 2 As shown, this application provides a remote sensing image segmentation device based on wavelet band enhancement. The remote sensing image segmentation device 200 includes: The decomposition module 201 is used to perform wavelet transform decomposition on the target remote sensing image to obtain multiple frequency band components of the target remote sensing image; the frequency band components are the components of the target remote sensing image under different frequency bands. Projection module 202 is used to perform feature projection on each frequency band component to obtain the frequency band features corresponding to each frequency band component; The fusion module 203 is used to stitch together all frequency band features to obtain frequency volume information, and to perform deep feature interaction and fusion on the frequency volume information to obtain the final features of the target remote sensing image; the frequency volume information is used to describe the frequency attributes of remote sensing objects in the target remote sensing image. The segmentation module 204 is used to perform mask segmentation based on the final features to obtain the mask segmentation result of the target remote sensing image.

[0124] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0125] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0126] like Figure 3 As shown, an embodiment of this application provides a terminal device, wherein the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 3 The diagram shows only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 executes the computer program D102 to implement the steps in any of the above method embodiments.

[0127] Specifically, when the processor D100 executes the computer program D102, it performs wavelet transform decomposition on the target remote sensing image to obtain multiple frequency band components. Then, it performs feature projection on each frequency band component to obtain the corresponding frequency band features. Finally, it concatenates all frequency band features to obtain frequency volume information. Deep feature extraction and fusion are then performed on the frequency volume information to obtain the final features of the target remote sensing image. Finally, mask segmentation is performed based on the final features to obtain the mask segmentation result of the target remote sensing image. The wavelet transform decomposition of the target remote sensing image decomposes it into multiple vectors, enabling a full and comprehensive analysis of information in different frequency bands. This increases the comprehensiveness and effectiveness of the frequency volume information obtained from the frequency band components. Performing mask segmentation on the remote sensing image based on this comprehensive frequency volume information effectively improves the accuracy of the remote sensing image mask segmentation.

[0128] The processor D100 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0129] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may be an external storage device of the terminal device D10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device D10. Furthermore, the memory D101 may include both internal and external storage units of the terminal device D10. The memory D101 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory D101 can also be used to temporarily store data that has been output or will be output.

[0130] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0131] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a device / terminal equipment for a remote sensing image segmentation method based on wavelet band enhancement, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0133] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention.

Claims

1. A remote sensing image segmentation method based on wavelet band enhancement, characterized in that, include: Wavelet transform decomposition is performed on the target remote sensing image to obtain multiple frequency band components of the target remote sensing image; The frequency band components are the components of the target remote sensing image under different frequency bands; Perform feature projection on each frequency band component to obtain the frequency band features corresponding to each frequency band component; All frequency band features are stitched together to obtain frequency volume information, and deep feature interaction and fusion are performed on the frequency volume information to obtain the final features of the target remote sensing image. The frequency domain volume information is used to describe the frequency domain attributes of remotely sensed ground features in the target remote sensing image. Mask segmentation is performed based on the final features to obtain the mask segmentation result of the target remote sensing image.

2. The remote sensing image segmentation method according to claim 1, characterized in that, The step of performing feature projection on each frequency band component to obtain the frequency band features corresponding to each frequency band component includes: For each of the frequency band components, the following steps are performed: The frequency band components are subjected to feature projection to obtain the initial frequency band features; The corresponding frequency band embedding is added to the initial frequency band features to obtain the frequency band features.

3. The remote sensing image segmentation method according to claim 2, characterized in that, The step of performing feature projection on the frequency band components to obtain initial frequency band features includes: Through the formula: Perform feature projection to obtain the initial frequency band features. ; in, Indicates frequency band components, This represents pointwise convolution. This represents the activation function. Represents depthwise convolution. The dimension of the initial frequency band features. For the batch size during training, Image length, Image width, For depth dimension.

4. The remote sensing image segmentation method according to claim 3, characterized in that, The step of adding corresponding frequency band embeddings to the initial frequency band features to obtain frequency band features includes: Through the formula: Calculate frequency band characteristics ; in, This indicates frequency band embedding.

5. The remote sensing image segmentation method according to claim 1, characterized in that, The process of performing deep-dimensional feature interaction and fusion on the frequency volume information to obtain the final features of the target remote sensing image includes: Based on the frequency volume information, depth feature interaction and dimensional projection are performed to obtain frequency domain features; Spatial domain features are defined, and the frequency domain features are fused with the spatial domain features to obtain the final features of the target remote sensing image.

6. The remote sensing image segmentation method according to claim 5, characterized in that, The process of performing depth feature interaction and dimensional projection based on the frequency volume information to obtain frequency domain features includes: The frequency volume information is downsampled to obtain downsampled features; The downsampled features are subjected to feature enhancement and self-attention operations to obtain enhanced features; The enhanced features are then subjected to dimensional projection to obtain frequency domain features.

7. The remote sensing image segmentation method according to claim 5, characterized in that, The process of fusing the frequency domain features with the spatial domain features to obtain the final features of the target remote sensing image includes: The frequency domain features are upsampled and flattened to obtain the upsampled frequency domain features. Based on the upsampled frequency domain features and the spatial domain features, the gate weights are calculated. Based on the gating weights, the upsampled frequency domain features and spatial domain features are fused to obtain the final features of the target remote sensing image.

8. The remote sensing image segmentation method according to claim 7, characterized in that, The calculation of the gating weights based on the upsampled frequency domain features and the spatial domain features includes: Through the formula: Calculate the gating weight ; in, Indicates splicing characteristics, Representing spatial domain characteristics, This represents the frequency domain characteristics after upsampling. This indicates concatenation along the channel dimension. Indicates a fully connected layer. Indicates the gated activation function; The step of fusing the upsampled frequency domain features and spatial domain features according to the gating weights to obtain the final features of the target remote sensing image includes: Through the formula: Calculate the final features ; in, This represents the frequency domain characteristics after projection. Indicates the projection layer.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the remote sensing image segmentation method based on wavelet band enhancement as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image segmentation method based on wavelet band enhancement as described in any one of claims 1 to 8.

Images