Reference image segmentation method based on space-frequency duality tuning

Abstract

The present application relates to the technical field of image segmentation, in particular to a referential image segmentation method based on space-frequency dual tuning, in which two types of learnable prompts of frequency domain and space domain are dynamically generated through space-frequency dual prompting technology; the space domain prompt is spliced and interacted with the encoder feature to strengthen the semantic representation of the referential target; the Laplacian pyramid and the frequency domain encoder based on LoRA are used to restore high-frequency details, and the multiplication operation of the frequency domain prompt is used to enhance the key features; finally, the dual-domain optimized features are fused to generate a boundary-accurate segmentation result through a mask decoder, effectively solving the problems of detail degradation and boundary blur caused by the lack of frequency domain information in traditional methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, in particular to a referential image segmentation method based on space-frequency dual optimization. BACKGROUND

[0002] With the booming development of multi-modal large models, parameter-efficient fine-tuning methods have become the dominant technology route in the field of referential image segmentation, as they can fully utilize the general prior knowledge in pre-trained models while significantly reducing computational and storage costs. The current mainstream method paradigm can be mainly summarized into four categories: the first category is full-fine-tuning, which updates all parameters of the entire multi-modal model during fine-tuning, although the effect is significant, but the computational overhead is huge; the second category is parameter-efficient independent encoder fine-tuning, which uses adapter, side network or prompt tuning techniques to independently fine-tune the visual encoder and text encoder; the third category is parameter-efficient interactive encoder fine-tuning, which focuses on efficient parameter updating of the interaction module (such as cross-attention mechanism) between visual and text features to enhance the fusion between modalities; the fourth category is parameter-efficient segmentation token decoding, which directly generates a special segmentation token similar to a language token to drive the mask decoder, realizing pixel-level segmentation.

[0003] Although the above-mentioned parameter-efficient fine-tuning-based paradigm has achieved remarkable results, they all have a fundamental common defect: all these methods are completely confined to feature learning and optimization in the spatial domain. Spatial domain processing mainly focuses on surface information such as pixel-level color, texture and shape, however, in the process of layer-by-layer transmission of deep network, high-frequency detail information (such as the edges of objects, sharp corner points, and subtle texture patterns) is easily smoothed and degraded, leading to the gradual "blurring" of feature maps. More critically, these methods completely ignore the rich discriminative information contained in the frequency domain; in the frequency domain, high-frequency components carry the fine details of object boundaries, while low-frequency components retain the overall structure information and background context. This neglect of frequency domain information makes the model unable to utilize the potential and powerful complementary and synergistic effects between spatial and frequency domains (for example, spatial domain provides semantic context, frequency domain provides contour details), thus limiting the model's deep semantic understanding ability of the referential target, and ultimately leading to rough or inaccurate object boundaries in the pixel-level segmentation stage.

[0004] Therefore, developing a new method that can synergistically utilize spatial and frequency domain information has become a key technical challenge to improve the performance of referential image segmentation. SUMMARY

[0005] The purpose of this invention is to provide a method for indexical image segmentation based on space-frequency duality tuning, in order to solve the problems mentioned in the background section. Specifically, the technical problem includes how to design a lightweight tuning method that can effectively mine and fuse frequency domain information and space-frequency interaction cues, thereby addressing the issues of high-frequency detail degradation and inaccurate segmentation boundaries caused by existing indexical image segmentation techniques being limited to the spatial domain.

[0006] To achieve the above objectives, the present invention aims to solve the problems of high-frequency degradation and inaccurate segmentation boundaries in the denotation image segmentation method based on space-frequency duality tuning, specifically including the following method steps:

[0007] S1. Acquire the input image and dynamically generate two types of learnable cues, including frequency domain cues and spatial domain cues, using spatial-frequency domain dual cueing techniques. The specific process for dynamically generating these two types of learnable cues includes:

[0008] The system receives an input image and a textual representation expression, where the input image is processed by a visual encoder to extract spatial feature maps, and the textual representation expression is processed by a semantic encoder to generate a text embedding vector.

[0009] Dynamically generate frequency domain cueing and spatial domain cueing using spatial-frequency domain dual cueing techniques;

[0010] The frequency domain cue is integrated from three components through a concatenation operation, including naive frequency domain cue, spatially conditionally guided frequency domain cue, and semantically guided frequency domain cue. The three components of the frequency domain cue are generated through corresponding cue control networks and cue guidance networks. The naive frequency domain cue consists of a learnable parameter matrix; the spatially conditionally guided frequency domain cue is generated by processing spatial feature maps through a spatial-to-frequency domain cue control network; and the semantically guided frequency domain cue is generated by processing text embedding vectors through a semantic-to-frequency domain cue guidance network.

[0011] The spatial domain cue is integrated from three components through a concatenation operation, including a naive spatial domain cue, a frequency-conditional guided spatial domain cue, and a semantically guided spatial domain cue. The three components of the spatial domain cue are generated through corresponding cue control networks and cue guidance networks. The naive spatial domain cue is composed of a learnable parameter matrix; the frequency-conditional guided spatial domain cue is generated by processing frequency domain features through a frequency-to-spatial domain cue control network; and the semantically guided spatial domain cue is generated by processing text embedding vectors through a semantic-to-spatial domain cue guidance network.

[0012] The step S1 dynamically generates two types of learnable prompts through a spatial frequency domain duality prompting technique, and the effect of the two types of learnable prompts is to construct a frequency domain prompt fusing a spatial condition and a semantic guide and a spatial domain prompt fusing a frequency domain condition and a semantic guide, respectively, based on vision and semantics, effectively mining the space-frequency interaction clues, and providing precise guide information for subsequent feature optimization.

[0013] S2, inputting the input image into a spatial domain encoder to extract a basic spatial domain feature, and interacting with the spatial domain prompt, realizing token superposition through a splicing operation to generate an optimized spatial domain feature, specifically comprising:

[0014] inputting the input image into a pre-trained spatial domain encoder to extract a basic spatial domain feature map;

[0015] interacting the basic spatial domain feature map with the spatial domain prompt, splicing the spatial domain prompt as a learnable prompt token with the token sequence of the basic spatial domain feature map to form an extended token sequence;

[0016] the extended token sequence is forward propagated through the subsequent layers of the encoder, exchanges information through self-attention calculation, and finally outputs the optimized spatial domain feature.

[0017] The step S2 splices and interacts the spatial domain prompt with the basic spatial domain feature, and the effect is that the prompt token and the image token fully exchange information and attention calculation in the encoder, thereby adaptively modulating and optimizing the original spatial feature, strengthening the semantic feature of the reference target, and alleviating the feature degradation problem caused by the limitation of the spatial domain.

[0018] S3, performing Laplace transform on the input image to generate a multi-scale Laplace pyramid, and processing the multi-scale Laplace pyramid through a high-frequency auxiliary enhancement module based on LoRA, wherein the generation process of the multi-scale Laplace pyramid comprises:

[0019] constructing a Gaussian pyramid, the bottom layer being the input image, generating images with halved resolution layer by layer through continuously applying Gaussian blur and downsampling operation;

[0020] generating each layer of the multi-scale Laplace pyramid by calculating the difference between each layer of the Gaussian pyramid image and the upsampled image of the previous layer.

[0021] inputting the multi-scale Laplace pyramid into the frozen frequency domain encoder, restoring high-frequency details by using a parameter-efficient fine-tuning technique, and using multiplication operation to apply the frequency domain prompt to the frequency domain feature to output the compensated frequency domain feature; wherein the specific process of the parameter-efficient fine-tuning technique for restoring high-frequency details comprises:

[0022] The multi-scale Laplacian pyramid is sent to a frozen frequency domain encoder; the frozen encoder is fine-tuned by using LoRA technology, the original weight is kept unchanged by introducing a learnable low-rank matrix, and a preliminary frequency domain feature is output.

[0023] The multiplication operation includes the following steps:

[0024] The tensor operation of the frequency domain prompt and the preliminary frequency domain feature generates a weight map; after the weight map is activated by an activation function, it is multiplied element by element with the preliminary frequency domain feature, and the compensated frequency domain feature is output.

[0025] Step S3 extracts and recovers the high-frequency detail information of the image from multiple scales by constructing a multi-scale Laplacian pyramid and using a LoRA-based frequency domain encoder for high-frequency enhancement. The effect of this step is to further strengthen the key edge and texture features through the multiplication selection mechanism of the frequency domain prompt, significantly improving the segmentation boundary blur problem caused by the loss of high-frequency information.

[0026] S4, the fine-tuned spatial domain feature and the compensated frequency domain feature are fused, wherein the fusion process specifically includes:

[0027] The fine-tuned spatial domain feature and the compensated frequency domain feature are subjected to a feature alignment operation to ensure that the dimensions of the two features match through size adjustment and channel projection; the fusion is performed using a channel dimension splicing strategy to form a fused feature tensor.

[0028] The fused feature tensor is input into the mask decoder to generate the target segmentation mask through a feature integration strategy, specifically including:

[0029] The fused feature tensor is input into the mask decoder, which is built based on a lightweight architecture;

[0030] Inside the mask decoder, the fused feature is reshaped into a token sequence and cross-attention calculation is performed with the text embedding feature.

[0031] After network layer processing, the feature is gradually upsampled and transformed to the original input image resolution.

[0032] Through the prediction head compression channel and the activation function mapping, a pixel-level probability map is obtained, and the final target segmentation mask is generated after binarization.

[0033] Step S4 fuses the fine-tuned spatial domain feature and the compensated frequency domain feature, which has the effect of complementary integration of the spatial feature rich in semantic information and the frequency domain feature rich in details, and through the cross-modal interaction and up-sampling reconstruction in the mask decoder, a clear and accurate target segmentation mask is finally generated, realizing efficient fusion and collaborative enhancement of spatial and frequency information under lightweight optimization.

[0034] Compared with the prior art, the present application has the beneficial effects that:

[0035] The present application effectively mines and fuses the space-frequency interaction clues by dynamically generating learnable prompts with semantic guidance and inter-domain conditional constraints through space-frequency domain duality prompting technology; In particular, the LoRA technology is used to efficiently fine-tune the frozen frequency domain encoder, and by introducing a trainable low-rank matrix, the high-frequency details are accurately restored while maintaining the integrity of the pre-training knowledge, significantly improving the edge blur problem caused by spatial domain limitations; Finally, through dual-domain feature fusion and lightweight optimization strategy, the segmentation effect of the referential target boundary is more accurate and detailed under the premise of significantly reducing the computational overhead. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The present application is a whole method step schematic diagram;

[0037] Figure 2 The present application is a core flowchart of step S3. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0039] Next, please refer to Figure 1 The present embodiment is aimed at a referential image segmentation method based on space-frequency duality optimization, including the following steps:

[0040] S1, an input image is obtained, and two types of learnable prompts, including frequency domain prompts and spatial domain prompts, are dynamically generated through space-frequency domain duality prompting technology, and the specific process includes:

[0041] The input image is received and the text representation expression is received, wherein the input image extracts a spatial feature map (denoted as ) through a visual encoder, and the text representation expression generates a text embedding vector (denoted as ) through a semantic encoder, providing a visual and semantic basis for dual-domain prompting;

[0042] Two types of learnable prompts, frequency domain prompts and spatial domain prompts, are dynamically generated using space-frequency domain duality prompting technology, wherein:

[0043] Frequency domain cues are used to capture frequency domain features (such as high-frequency details and low-frequency structure). A frequency domain cue consists of three components, which are combined through a concatenation operation to form naive frequency domain cues, spatially guided frequency domain cues, and semantically guided frequency domain cues.

[0044] The `concat` function is a concatenation function used to concatenate naive frequency domain cues, spatially guided frequency domain cues, and semantically guided frequency domain cues. The vector representation of the naive frequency domain cues is as follows:

[0045] ;in This represents the basic unit of the prompt vector. This represents the 1st, 2nd, and so on up to the Nfpth naive frequency domain hint token; This represents the set of real numbers, indicating that the prompt consists of real numbers; This represents the channel dimension of each naive frequency domain cue token, for example, in the Transformer model, the dimension of the corresponding feature vector (such as 512-dimensional or 768-dimensional). This indicates the number of naive frequency domain cue tokens, defining the length of the cue vector or the number of tokens. Let represent a matrix with Nfp rows and C columns, which is a trainable parameter matrix with shape [Nfp, C].

[0046] The vector representation of the frequency domain cue guided by spatial conditions is as follows:

[0047] ;in This represents the basic unit of the prompt vector. Indicates the 1st, 2nd, and so on up to the 1st. A spatial condition-guided frequency domain prompt token; It is a lightweight spatial-to-frequency domain cue control network;

[0048] The vector representation of semantically guided frequency domain cues is as follows:

[0049] To indicate the quantity, among which This represents the basic unit of the prompt vector. Indicates the 1st, 2nd, and so on up to the 1st. A semantically guided frequency domain prompt token; It is a semantic-to-frequency domain cueing and guidance network.

[0050] Spatial domain cues are also composed of three types of components: naive spatial domain cues, frequency-domain condition-guided spatial domain cues, and semantically guided spatial domain cues, namely:

[0051] where the vector representation of the prompt is

[0052] where denotes the basic unit of the prompt vector, denotes the 1st, 2nd, up to the th naive spatial domain prompt token; denotes the dimension of each naive spatial domain prompt token;

[0053] The vector representation of the frequency domain conditioned spatial domain prompt is

[0054] where denotes the basic unit of the prompt vector, denotes the 1st, 2nd, up to the th frequency domain conditioned spatial domain prompt token; is a lightweight frequency-to-spatial domain prompt control network;

[0055] The vector representation of the semantic guided spatial domain prompt is

[0056] where denotes the basic unit of the prompt vector, denotes the 1st, 2nd, up to the th semantic guided spatial domain prompt token; is a semantic-to-spatial domain prompt guided network.

[0057] S2, input the input image into the spatial domain encoder to extract the basic spatial domain feature, and interact with the spatial domain prompt, realize token superposition through splicing operation, generate the optimized spatial domain feature, specifically including:

[0058] The input image is input into a pre-trained spatial domain encoder, which extracts visual features of the image layer by layer, and finally outputs a basic spatial domain feature map tensor, denoted as ; the spatial domain feature map tensor carries the semantic and structural information of the image in the spatial domain;

[0059] The spatial domain feature map tensor interacts with the spatial domain prompt , and the specific interaction method is splicing operation, and the process of splicing operation is as follows:

[0060] In the Transformer architecture, the spatial domain feature map tensor is represented as a token sequence of a series of image blocks;

[0061] The spatial domain prompt It is viewed as a series of learnable cue tokens, directly related to the spatial domain feature map tensor. The token sequences are concatenated along the sequence length dimension to form a new, extended token sequence; this is equivalent to injecting additional, controllable guiding information into the model. This information integrates cues from the frequency domain and the semantic domain, thereby guiding the spatial domain encoder to pay more attention to the target region features related to text reference in subsequent self-attention and other computational processes.

[0062] After the above splicing operation, the extended token sequence will continue to propagate forward through the subsequent Transformer layers in the spatial domain encoder. In these layers, the cue token and the original image token will exchange information and perform attention calculations, ultimately adaptively modulating and optimizing the original spatial domain features.

[0063] Finally, the features obtained from the encoder output are the spatial domain features after spatial domain cue tuning, i.e., the tuned spatial domain features, denoted as... This feature not only preserves the original spatial information but also enhances the semantic features related to the referent, laying a solid foundation for subsequent fusion with frequency domain features and accurate segmentation.

[0064] S3, please refer to Figure 2 The Laplacian transform is performed on the input image to generate a multi-scale Laplacian pyramid, specifically including:

[0065] First, construct a Gaussian pyramid, starting from the bottom layer (layer 0). Given an input image, a series of images with progressively halved resolution are generated by continuously applying Gaussian blur and downsampling operations.

[0066] , ,..., ;in Represents the downsampling function; The first of the Gaussian pyramids layer;

[0067] Then, each layer of the multi-scale Laplacian pyramid is generated by calculating the difference between each layer of the Gaussian pyramid image and the upsampled image of the layer above it. The specific calculation formula is as follows:

[0068] ,in Indicates the upsampling function; The first representing the Pyramid of Laplace Layers, each layer Both contain high-frequency detail information (edges, textures, etc.) of the input image at a specific scale, as well as some residual information; the larger the scale ( The smaller the scale, the more detailed the represented details are; the larger the scale, the more coarse the represented details are. The smaller the scale, the more detailed the represented details are; the larger the scale, the more coarse the represented details are.

[0069] The multi-scale Laplacian pyramid is processed by a high-frequency auxiliary enhancement module based on LoRA (low-rank adaptation), the multi-scale Laplacian pyramid is input into a frozen frequency domain encoder, and a parameter efficient fine-tuning technology is adopted to recover high-frequency details, specifically including:

[0070] The images of each layer of the multi-scale Laplacian pyramid are input as input into a frozen frequency domain encoder, the frequency domain encoder has the same pre-training weight architecture as the spatial domain encoder in the S2 step, but the weights are frozen in this process, that is, not participating in gradient update, so as to make full use of the prior knowledge of extracting general features and ensure parameter efficiency;

[0071] In order to parameter efficient fine-tune the frozen frequency domain encoder to adapt to the high-frequency detail extraction task, the present application adopts LoRA (low-rank adaptation) technology, that is, a high-frequency auxiliary enhancement module based on LoRA, specifically, in the linear layer inside the frequency domain encoder, a learnable low-rank matrix B and A are introduced, for a frozen layer with original frozen weights , the forward propagation process is modified as:

[0072] , wherein , wherein represents the first layer of the frequency domain block; represents the first layer of the frequency domain block; represents the frozen weight; represents the result of the introduced learnable low-rank matrix product; by training only and while keeping unchanged, the trainable parameter amount is greatly reduced, and the frozen encoder is effectively guided to focus on recovering and extracting key high-frequency detail features from the Laplacian input, and the output of the process is regarded as the preliminary frequency domain features.

[0073] The frequency domain prompt generated dynamically from step S1 is applied to the preliminary frequency domain features obtained in the last step, and the specific interaction mode is multiplication operation, and the specific process is as follows:

[0074] First, the frequency domain prompt generates a weight map through tensor operation with the frequency domain features, and after the weight map is activated by the Sigmoid function, it is multiplied with the preliminary frequency domain features element by element;

[0075] This operation is equivalent to an attention mechanism or gating mechanism, allowing the frequency domain cues to adaptively weight different parts of the frequency domain features, filtering and enhancing frequency domain information beneficial to the target segmentation, and suppressing irrelevant or noise information.

[0076] After the above LoRA technology fine-tuning of feature extraction and multiplication selection operation of frequency domain cues, the final output is the compensated frequency domain features, denoted as , which effectively contains fine edges and texture information captured by the multi-scale Laplacian pyramid and adaptively selected by the model.

[0077] S4, fuse the fine-tuned spatial domain features and the compensated frequency domain features, and input the mask decoder through the feature integration strategy to generate the final target segmentation mask, specifically including:

[0078] fuse the fine-tuned spatial domain features output by S2 and the compensated frequency domain features output by S3 Before fusion, feature alignment operation is needed to ensure that the two feature tensors are dimensionally matched in spatial resolution and channel number, adjust the spatial size through upsampling or downsampling, and adjust the channel number through 1x1 convolution linear projection;

[0079] The fusion strategy adopted is channel dimension splicing, that is, the fine-tuned spatial domain features and the compensated frequency domain features are spliced in the channel dimension C to form a fused feature tensor , which maximizes the preservation of all information from space and frequency domains, providing the decoder with a composite feature representation that combines global semantic understanding and local detail discrimination.

[0080] The spliced fusion features will be input into the mask decoder, which is based on a lightweight Transformer architecture, such as composed of 3 Transformer layers (each layer contains 8 attention heads) and a feedforward network (dimension 512);

[0081] The feature integration strategy is reflected in the processing flow inside the mask decoder, including:

[0082] Serialization: the fusion features are first reshaped into a token sequence;

[0083] Interaction and enhancement: the token sequence is cross-attention calculated with the text embedding features in the Transformer layer, which enables the visual features to be further focused and adjusted according to the semantics expressed by the text, ensuring that the decoding process is closely around the target referred to by the language;

[0084] Up-sampling and reconstruction: after the Transformer layer processing, the features are fed into the feedforward network of the mask decoder, and are gradually up-sampled and transformed, and finally mapped to the same spatial size as the original input image resolution;

[0085] The output of the mask decoder is a feature map with the same spatial size as the input image, which is compressed to 1 channel through a prediction head, such as a convolutional layer;

[0086] Finally, through a Sigmoid activation function, each pixel value of the single-channel feature map is mapped to between 0 and 1, representing the probability that the pixel belongs to the referred target;

[0087] The final output generated is the binary target segmentation mask, usually set a threshold (such as 0.5), the probability value greater than the threshold of the pixel is determined as the target object (value is 1), the rest of the pixel is determined as the background (value is 0), so as to get clear and accurate segmentation boundary.

[0088] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above embodiments, the above embodiments and descriptions in the specification are only preferred examples of the present application, and are not intended to limit the present application, various changes and improvements can be made without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for segmenting an image based on space-frequency duality tuning, characterized in that, The method comprises the following steps: S1, obtaining an input image, and dynamically generating two types of learnable prompts, including a frequency domain prompt and a spatial domain prompt, through a spatial frequency domain dual prompt technology; wherein the dynamic generation process of the two types of learnable prompts comprises: Receiving an input image and a text representation expression, wherein the input image extracts a spatial feature map through a visual encoder, and the text representation expression generates a text embedding vector through a semantic encoder; Dynamically generating a frequency domain prompt and a spatial domain prompt using a spatial frequency domain dual prompt technology; The frequency domain prompt is integrated by splicing operation of three components, including naive frequency domain prompt, spatial condition guided frequency domain prompt and semantic guided frequency domain prompt; The spatial domain prompt is integrated by splicing operation of three components, including naive spatial domain prompt, frequency domain condition guided spatial domain prompt and semantic guided spatial domain prompt; S2, inputting the input image into a spatial domain encoder to extract basic spatial domain features, and interacting with the spatial domain prompt, realizing token superposition through splicing operation, and generating optimized spatial domain features; S3, performing Laplace transform on the input image to generate a multi-scale Laplace pyramid, processing the multi-scale Laplace pyramid through a high-frequency auxiliary enhancement module based on LoRA, inputting the multi-scale Laplace pyramid into a frozen frequency domain encoder, restoring high-frequency details by using parameter efficient fine-tuning technology, and outputting compensated frequency domain features by using multiplication operation to apply the frequency domain prompt to the frequency domain features; S4, fusing the optimized spatial domain features and the compensated frequency domain features, and inputting a mask decoder through a feature integration strategy to generate a target segmentation mask.

2. The image segmentation method based on space-frequency duality tuning according to claim 1, wherein, The three components of the frequency domain prompt are generated through corresponding prompt control networks and prompt guide networks, specifically including: The naive frequency domain prompt is composed of a learnable parameter matrix; The spatial condition guided frequency domain prompt is generated by processing the spatial feature map through a spatial domain to frequency domain prompt control network; The semantic guided frequency domain prompt is generated by processing the text embedding vector through a semantic to frequency domain prompt guide network.

3. The image segmentation method based on space-frequency duality tuning according to claim 1, wherein, The three components of the spatial domain prompt are generated through corresponding prompt control networks and prompt guide networks, specifically including: The naive spatial domain prompt is composed of a learnable parameter matrix; The frequency domain condition guided spatial domain prompt is generated by processing the frequency domain features through a frequency domain to spatial domain prompt control network; The semantic guided spatial domain prompt is generated by processing the text embedding vector through a semantic to spatial domain prompt guide network.

4. The method of image segmentation by space-frequency duality tuning according to claim 1, wherein, The generation process of the optimized spatial domain features specifically includes: Inputting the input image into a pre-trained spatial domain encoder to extract a basic spatial domain feature map; Interacting the basic spatial domain feature map with the spatial domain prompt, splicing the spatial domain prompt as a learnable prompt token with the token sequence of the basic spatial domain feature map to form an extended token sequence; The extended token sequence is forward propagated through the subsequent layers of the encoder, exchanges information through self-attention calculation, and finally outputs the optimized spatial domain features.

5. The method of image segmentation by space-frequency duality tuning according to claim 1, wherein, The generation process of the multi-scale Laplace pyramid specifically includes: A Gaussian pyramid is constructed, with the bottom layer being the input image, and images with halved resolution are generated by continuously applying Gaussian blur and downsampling operations; Each layer of the multi-scale Laplacian pyramid is generated by calculating the difference between the current layer of the Gaussian pyramid image and the upsampled image of the previous layer.

6. The method of image segmentation by space-frequency duality tuning according to claim 1, wherein, The specific process of the parameter efficient fine-tuning technique to restore high-frequency details includes: The multi-scale Laplacian pyramid is input into the frozen frequency domain encoder. The LoRA technique is used to fine-tune the frozen encoder, which introduces a learnable low-rank matrix while keeping the original weights unchanged, and outputs the preliminary frequency domain features.

7. The method of image segmentation by space-frequency duality tuning according to claim 1, wherein, The specific process of the multiplication operation to apply the frequency domain prompt to the frequency domain features includes: The frequency domain prompt and the preliminary frequency domain features are subjected to tensor operation to generate a weight map. After the weight map is activated by an activation function, it is multiplied element by element with the preliminary frequency domain features to output the compensated frequency domain features.

8. The method of image segmentation by space-frequency duality tuning according to claim 1, wherein, The fusion process in step S4 specifically includes: The fine-tuned spatial domain features and the compensated frequency domain features are subjected to feature alignment operations to ensure that the dimensions of the two features match through size adjustment and channel projection. The strategy of channel dimension splicing is used for fusion to form a fused feature tensor.

9. The method of image segmentation by space-frequency duality tuning according to claim 1, wherein, The generation process of the target segmentation mask specifically includes: The fused feature tensor is input into the mask decoder, which is built based on a lightweight architecture; Inside the mask decoder, the fused features are reshaped into a token sequence and cross-attention calculation is performed with the text embedding features; After network layer processing, the resolution is gradually upsampled to the original input image resolution; Through the prediction head compression channel and the activation function mapping, a pixel-level probability map is obtained, and the final target segmentation mask is generated after binarization.

Images