Underwater instance segmentation method and device based on band decomposition adapter

Abstract

The embodiment of the application provides a kind of underwater instance segmentation method and equipment based on frequency band decomposition adapter, it is related to image information processing and computer vision technical field, by extracting the intermediate layer feature of pre-training visual basic model from the underwater image to be segmented, and the amplitude spectrum is low, middle, high three-band smooth decomposition and independent adaptation correction, then according to global amplitude distribution dynamic routing weighted fusion each frequency band correction result, with phase spectrum reorganization inverse transform back to spatial domain, while using light weight spatial branch to supplement local structure and boundary details, finally the frequency domain and spatial double-branch feature residual are fused and input instance segmentation decoder.The scheme can be fine-grained correction to the different degradation factors on their own frequency band differentiation influence, avoid the reconstruction artifact caused by boundary band division, and adjust the contribution of each frequency band according to the input scene degradation distribution, improve the separation precision and mask boundary quality of target instance in complex underwater environment.

Description

Technical Field

[0001] This invention relates to the fields of image information processing and computer vision technology, and in particular to an underwater instance segmentation method and device based on a frequency band decomposition adapter. Background Technology

[0002] Currently, the technologies related to this invention mainly include three aspects: the first is an underwater instance segmentation method; the second is a method for transferring visual basic models in underwater scenes; and the third is a frequency domain modeling method for underwater degradation features.

[0003] Underwater instance segmentation aims to perform pixel-level localization and category recognition of targets in complex underwater images, and to distinguish different individual instances within the same category. Compared to natural terrestrial scenes, underwater images are affected by factors such as water absorption, scattering, suspended particles, and non-uniform lighting, often exhibiting degradation phenomena such as color shift, decreased contrast, blurring, noise, and weakened boundaries. These degradations weaken target boundaries and local details, and exacerbate the confusion between target instances and complex background textures, making underwater instance segmentation more challenging than conventional instance segmentation.

[0004] For underwater instance segmentation, existing methods typically incorporate underwater scene-specific modules into a general instance segmentation framework. Lian et al., in their paper "WaterMask: Instance Segmentation for Underwater Imagery," designed an instance segmentation method for underwater images based on the Mask R-CNN framework to enhance the model's adaptability to degraded underwater scenes. Cong et al., in their paper "UIS-Mamba: Exploring Mamba for Underwater Instance Segmentation via Dynamic Tree Scan and Hidden State Weaken," introduced a state-space modeling mechanism to improve global perception in complex underwater scenes. While these methods can alleviate the segmentation performance degradation caused by underwater degradation to some extent, they still primarily rely on task-specific modules and are insufficient in handling the weakening of target boundaries and foreground-background blurring issues under complex degradation.

[0005] In recent years, visual base models have been increasingly adopted for underwater instance segmentation tasks due to their strong general representation capabilities. Lian et al., in their paper "Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset," utilized SAM to guide underwater salient instance segmentation. While such methods can leverage the general semantic representations of visual base models to improve underwater instance segmentation performance, their adaptation process often focuses on cue learning, overall feature alignment, or domain transfer. They typically treat underwater degradation as a uniform domain shift, making it difficult to characterize the differentiated impacts of various degradation factors such as color shift, blurring, and noise on the model's internal features.

[0006] Frequency domain modeling methods targeting underwater degradation features offer a new approach to adapting visual base models. Chen et al., in their paper "Empowering DINO Representations for Underwater Instance Segmentation via Aligner and Prompter," proposed DiveSeg, which uses AquaStyle Aligner to extract color style features from underwater images through Fourier decomposition and embeds them into the DINOv2 fine-tuning process to alleviate the style mismatch between underwater images and pre-trained models. This method demonstrates that frequency domain style modeling helps improve the adaptability of visual base models to underwater scenes. However, it primarily focuses on underwater color style alignment and global domain adaptation, failing to further distinguish the different impacts of degradation factors such as color shift, blurring, and noise on low-frequency, mid-frequency, and high-frequency feature components, and lacks a mechanism for adaptively adjusting the contributions of different frequency bands based on the degradation distribution of the input image.

[0007] In summary, existing underwater instance segmentation methods still have the following shortcomings: First, traditional underwater instance segmentation methods do not adequately utilize the general representation capabilities of the visual baseline model; second, existing visual baseline model transfer methods mostly adapt underwater degradation as a global domain offset, lacking modeling of frequency band differences in different degradation factors; third, existing frequency domain adaptation methods mostly focus on overall style alignment or global frequency domain processing, making it difficult to independently correct feature offsets in different frequency bands. Therefore, existing technical solutions face the challenge of how to address the differentiated impact of underwater degradation in different frequency bands, while maintaining the stability of the main parameters of the visual baseline model, to achieve fine-grained, adaptive frequency band feature correction, thereby improving the accuracy of underwater instance segmentation and the quality of mask boundaries.

[0008] Therefore, it is necessary to provide a frequency band decomposition and adaptation method for underwater instance segmentation. While maintaining the stability of the main parameters of the visual basic model, the amplitude characteristics of different frequency bands are independently corrected, and the frequency band branches are adaptively fused according to the input scene, thereby improving the target separation capability and mask boundary quality in complex underwater scenes. Summary of the Invention

[0009] This invention proposes an underwater instance segmentation method and device based on a frequency band decomposition adapter. It solves the problem that existing technical solutions cannot address the differentiated impact of underwater degradation on different frequency bands. While maintaining the stability of the main parameters of the visual basic model, it achieves fine-grained and adaptive frequency band feature correction, thereby improving the accuracy of underwater instance segmentation and the quality of mask boundaries.

[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: According to a first aspect of the present invention, an underwater instance segmentation method based on a frequency band decomposition adapter is provided, the method comprising: Input the underwater image to be segmented and input the underwater image into the pre-trained visual base model to obtain the underwater image features output by at least one network layer in the pre-trained visual base model; Fourier transform is performed on the underwater image features to obtain the corresponding frequency domain features, and the frequency domain features are decomposed into amplitude spectrum and phase spectrum; Multiple smooth frequency band masks are constructed based on Gaussian functions, and the amplitude spectrum is divided into low-frequency amplitude components, mid-frequency amplitude components and high-frequency amplitude components using multiple smooth frequency band masks; The low-frequency amplitude component, mid-frequency amplitude component, and high-frequency amplitude component are respectively input into the corresponding independent frequency band adaptation subspace to obtain the amplitude correction characteristics corresponding to different frequency bands; Dynamic routing weights are generated based on the global distribution of the amplitude spectrum, and the amplitude correction features corresponding to different frequency bands are weighted and fused using the dynamic routing weights to obtain the fused amplitude correction results. The fused amplitude correction result is fused with the original amplitude spectrum to obtain the corrected amplitude spectrum. The corrected amplitude spectrum and the phase spectrum are then reconstructed in the frequency domain, and the frequency-domain corrected spatial features are obtained by inverse Fourier transform. Spatial semantic supplementation of underwater image features is performed by lightweight spatial branching to obtain spatial supplementation features; The frequency-domain rectified spatial features are fused with the spatial supplementary features and then residually connected with the underwater image features to obtain the adapted visual basic model features. The adapted visual base model features are input into the instance segmentation decoder, which outputs the category prediction results and mask prediction results of the target instances in the underwater image.

[0011] In some implementations of the first aspect, the aforementioned pre-trained visual base model is a visual base model with frozen parameters, the frequency band decomposition adapter is inserted after at least one network layer of the pre-trained visual base model, and only the parameters in the frequency band decomposition adapter and the instance segmentation decoder are trained.

[0012] In some implementations of the first aspect, the underwater image features are subjected to Fourier transform to obtain the corresponding frequency domain features, and the frequency domain features are decomposed into amplitude spectrum and phase spectrum, including: For the Underwater image features output by each network layer Perform a two-dimensional Fourier transform to obtain the frequency domain features. ; for frequency domain features Perform spectrum centering processing to obtain the centered frequency domain features; The modulus of the centered frequency domain features is used as the amplitude spectrum. The phase angle of the centered frequency domain features is calculated as the phase spectrum. .

[0013] In some implementations of the first aspect, the above-mentioned construction of multiple smooth frequency band masks based on Gaussian functions includes: Using the center of the frequency domain as the origin, calculate the radial distance from each frequency position to the center of the frequency domain; Construct a first Gaussian function and a second Gaussian function with different scale parameters based on radial distance; The first Gaussian function is used as the low-frequency mask, the difference between the second Gaussian function and the first Gaussian function is used as the mid-frequency mask, and the result of subtracting the second Gaussian function from the value 1 is used as the high-frequency mask.

[0014] In some implementations of the first aspect, the aforementioned low-frequency amplitude component, mid-frequency amplitude component, and high-frequency amplitude component are obtained by multiplying the amplitude spectrum element-wise with the low-frequency mask, mid-frequency mask, and high-frequency mask, respectively.

[0015] In some implementations of the first aspect, the aforementioned independent frequency band adaptation subspace includes a first linear projection layer, a nonlinear activation layer, and a second linear projection layer connected in sequence. The first linear projection layer is used to map the amplitude components of the corresponding frequency band to a low-dimensional latent space, the nonlinear activation layer is used to introduce nonlinear correction capability, and the second linear projection layer is used to map the low-dimensional latent space features back to the original channel dimension to obtain the amplitude correction features of the corresponding frequency band.

[0016] In some implementations of the first aspect, the process of generating the aforementioned dynamic routing weights includes: Global pooling is performed on the amplitude spectrum to obtain the global amplitude description vector; The global amplitude description vector is input into the route predictor to obtain the routing responses for the low-frequency, mid-frequency, and high-frequency bands; pass The function normalizes the routing response to obtain low-frequency routing weights, medium-frequency routing weights, and high-frequency routing weights.

[0017] In some implementations of the first aspect, the aforementioned lightweight spatial branch includes a bottleneck structure, which includes a dimension reduction projection layer, a nonlinear activation layer, and a dimension increase projection layer, used to supplement the local structural information and boundary detail information that are not fully expressed by the spatial features after frequency domain correction in the spatial domain.

[0018] In some implementations of the first aspect, during the training phase, the instance segmentation joint loss function is used to supervise and optimize the category prediction results and the mask prediction results. The joint loss function for instance segmentation includes classification loss, binary cross-entropy mask loss, and Dice mask loss.

[0019] According to a second aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and memory that is communicatively connected to at least one processor; The invention is characterized in that the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to implement the method of the first aspect of the invention.

[0020] This invention provides an underwater instance segmentation method, device, and storage medium based on a frequency band decomposition adapter. The method first performs Fourier transform on the intermediate features of the visual base model to obtain the amplitude spectrum and phase spectrum. A smooth frequency band mask is constructed using a Gaussian function to decompose the amplitude spectrum into three frequency band components: low-frequency, mid-frequency, and high-frequency. Then, an independent adaptation subspace is set for each frequency band to perform parameter-efficient differential amplitude correction. Next, dynamic routing weights are generated based on the global distribution of the amplitude spectrum, and the correction results for each frequency band are adaptively weighted and fused. These results are then fused with the residual of the original amplitude spectrum and combined with the phase spectrum through an inverse Fourier transform to reconstruct the frequency domain correction features. Simultaneously, a lightweight spatial branch is introduced to supplement local structure and boundary details. Finally, the features fused with the frequency domain and spatial branches are input into the instance segmentation decoder. This method achieves fine-grained feature correction of the differential impact of underwater degradation on different frequency bands through frequency band decomposition and independent adaptation. The Gaussian smooth mask avoids reconstruction instability caused by spectral discontinuity. The dynamic routing mechanism can adaptively adjust the contribution of each frequency band according to the degradation distribution of the input scene. It complements the spatial branch and effectively improves the target separation capability and mask boundary quality. Under the premise of keeping the core parameters of the visual basic model frozen, it significantly improves the instance segmentation accuracy in complex underwater scenes. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 The diagram illustrates a flowchart of an underwater instance segmentation method based on a frequency band decomposition adapter provided by an embodiment of the present invention. Figure 2 The diagram illustrates the overall flow of an underwater instance segmentation method based on a frequency band decomposition adapter provided by an embodiment of the present invention. Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present invention is shown. Detailed Implementation

[0023] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0024] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0025] In order to solve the above-mentioned technical problems, this invention discloses an underwater instance segmentation method, device, and storage medium based on a frequency band decomposition adapter. First, Fourier transform is performed on the intermediate features of the visual base model to obtain the amplitude spectrum and phase spectrum. A smooth frequency band mask is constructed using a Gaussian function to decompose the amplitude spectrum into three frequency band components: low frequency, mid frequency, and high frequency. Then, an independent adaptation subspace is set for each frequency band to perform parameter-efficient differential amplitude correction. Next, dynamic routing weights are generated based on the global distribution of the amplitude spectrum, and the correction results of each frequency band are adaptively weighted and fused. After fusing with the residual of the original amplitude spectrum, the frequency domain correction features are reconstructed by combining the phase spectrum with the inverse Fourier transform. At the same time, a lightweight spatial branch is introduced to supplement local structure and boundary details. Finally, the features after fusing the frequency domain and spatial branches are input into the instance segmentation decoder. This method achieves fine-grained feature correction of the differential impact of underwater degradation on different frequency bands through frequency band decomposition and independent adaptation. The Gaussian smooth mask avoids reconstruction instability caused by spectral discontinuity. The dynamic routing mechanism can adaptively adjust the contribution of each frequency band according to the degradation distribution of the input scene. It complements the spatial branch and effectively improves the target separation capability and mask boundary quality. Under the premise of keeping the core parameters of the visual basic model frozen, it significantly improves the instance segmentation accuracy in complex underwater scenes.

[0026] The technical solutions provided by the embodiments of the present invention will now be described with reference to the accompanying drawings.

[0027] Figure 1 A flowchart illustrating an underwater instance segmentation method based on a frequency band decomposition adapter according to an embodiment of the present invention is shown.

[0028] like Figure 1 As shown, the underwater instance segmentation method based on the frequency band decomposition adapter may include: S101, Input the underwater image to be segmented, and input the underwater image into the pre-trained visual base model to obtain the underwater image features output by at least one network layer in the pre-trained visual base model; S102, Perform Fourier transform on the underwater image features to obtain the corresponding frequency domain features, and decompose the frequency domain features into amplitude spectrum and phase spectrum; S103, based on the Gaussian function, constructs multiple smooth frequency band masks, and uses multiple smooth frequency band masks to divide the amplitude spectrum into low-frequency amplitude components, mid-frequency amplitude components and high-frequency amplitude components; S104, input the low-frequency amplitude component, mid-frequency amplitude component and high-frequency amplitude component into the corresponding independent frequency band adaptation subspace to obtain the amplitude correction characteristics corresponding to different frequency bands; S105. Dynamic routing weights are generated based on the global distribution of the amplitude spectrum, and the amplitude correction features corresponding to different frequency bands are weighted and fused using the dynamic routing weights to obtain the fused amplitude correction results. S106, the fused amplitude correction result is fused with the original amplitude spectrum to obtain the corrected amplitude spectrum, and the corrected amplitude spectrum and phase spectrum are reconstructed in the frequency domain, and the frequency domain corrected spatial features are reconstructed by inverse Fourier transform. S107, spatial semantic supplementation of underwater image features is performed through lightweight spatial branching to obtain spatial supplementation features; S108 fuses the frequency-domain corrected spatial features with the spatial supplementary features and performs residual connection with the underwater image features to obtain the adapted visual basic model features. S109 inputs the adapted visual base model features into the instance segmentation decoder and outputs the category prediction results and mask prediction results of the target instances in the underwater image.

[0029] In the underwater instance segmentation method based on a frequency band decomposition adapter described in this invention, intermediate layer features of a pre-trained visual model are extracted from the underwater image to be segmented. The amplitude spectrum is then smoothly decomposed and independently adapted and corrected across low, medium, and high frequency bands. The correction results of each frequency band are dynamically routed and weighted according to the global amplitude distribution. After recombination with the phase spectrum, the results are inversely transformed back to the spatial domain. Simultaneously, a lightweight spatial branch is used to supplement local structure and boundary details. Finally, the residuals of the frequency domain and spatial dual-branch features are fused and input into the instance segmentation decoder. This scheme can perform fine-grained correction for the differentiated effects of various degradation factors such as color cast, blur, and noise on their respective frequency bands, avoiding reconstruction artifacts caused by boundary frequency band division. Furthermore, it adaptively adjusts the contribution of each frequency band according to the degradation distribution of the input scene, significantly improving the separation accuracy and mask boundary quality of target instances in complex underwater environments.

[0030] In some embodiments, the pre-trained visual base model is a visual base model with frozen parameters, the band decomposition adapter is inserted after at least one network layer of the pre-trained visual base model, and only the parameters in the band decomposition adapter and the instance segmentation decoder are trained.

[0031] In the above embodiments, the main parameters of the pre-trained visual base model are frozen, and a frequency band decomposition adapter is inserted only after at least one network layer. Only the adapter and the instance segmentation decoder are trained. This configuration can preserve general visual knowledge, significantly reduce the number of trainable parameters and memory usage, achieve efficient domain adaptation of parameters, avoid overfitting, and quickly obtain instance segmentation capabilities adapted to underwater degradation scenarios with low annotation costs.

[0032] In some embodiments, the above-mentioned Fourier transform of underwater image features to obtain corresponding frequency domain features, and the decomposition of frequency domain features into amplitude spectrum and phase spectrum, includes: For the Underwater image features output by each network layer Perform a two-dimensional Fourier transform to obtain the frequency domain features. ; for frequency domain features Perform spectrum centering processing to obtain the centered frequency domain features; The modulus of the centered frequency domain features is used as the amplitude spectrum. The phase angle of the centered frequency domain features is calculated as the phase spectrum. .

[0033] In the above embodiment, after performing a two-dimensional Fourier transform and spectral centering on the underwater image features output by the k-th network layer, the magnitude is extracted as the amplitude spectrum and the phase angle is extracted as the phase spectrum. This operation effectively decouples the frequency domain from the spatial domain, allowing subsequent corrections to target only the amplitude spectrum significantly affected by degradation, while preserving the phase spectrum carrying semantic layout information. This improves adaptability while avoiding damage to the original structural features, and the centering process facilitates subsequent frequency band division based on radial distance.

[0034] In some embodiments, the above-mentioned construction of multiple smooth frequency band masks based on Gaussian functions includes: Using the center of the frequency domain as the origin, calculate the radial distance from each frequency position to the center of the frequency domain; Construct a first Gaussian function and a second Gaussian function with different scale parameters based on radial distance; The first Gaussian function is used as the low-frequency mask, the difference between the second Gaussian function and the first Gaussian function is used as the mid-frequency mask, and the result of subtracting the second Gaussian function from the value 1 is used as the high-frequency mask.

[0035] In the above embodiment, the radial distance is calculated with the frequency domain center as the origin, and two Gaussian functions with different scale parameters are constructed. The first Gaussian function is used as a low-frequency mask, the difference between the second and first Gaussian functions is used as a mid-frequency mask, and the value 1 minus the second Gaussian function is used as a high-frequency mask. Using Gaussian functions to form a smooth-transition frequency band mask avoids spectral abrupt changes caused by hard-boundary frequency band division, thereby suppressing ringing or checkerboard artifacts during inverse Fourier reconstruction, ensuring the stability of the reconstructed spatial features, and simultaneously making the frequency band division conform to the concentric circle distribution law of low-frequency, mid-frequency, and high-frequency in a physically meaningful sense.

[0036] In some embodiments, the low-frequency amplitude component, mid-frequency amplitude component, and high-frequency amplitude component are obtained by multiplying the amplitude spectrum element-wise with the low-frequency mask, mid-frequency mask, and high-frequency mask, respectively.

[0037] In the above embodiments, the original amplitude spectrum is multiplied element-wise with the low-frequency, mid-frequency, and high-frequency smoothing masks respectively to obtain independent amplitude components of the three frequency bands. This operation achieves fully differentiable and energy-leakage-free frequency band decomposition. The sum of the three masks is always 1, ensuring the completeness of the decomposition and energy conservation. This facilitates gradient backpropagation in end-to-end training and provides clean input components for subsequent independent correction of each frequency band.

[0038] In some embodiments, the aforementioned independent frequency band adaptation subspace includes a first linear projection layer, a nonlinear activation layer, and a second linear projection layer connected in sequence. The first linear projection layer is used to map the amplitude components of the corresponding frequency band to a low-dimensional latent space, the nonlinear activation layer is used to introduce nonlinear correction capability, and the second linear projection layer is used to map the low-dimensional latent space features back to the original channel dimension to obtain the amplitude correction features of the corresponding frequency band.

[0039] In the above embodiments, each independent frequency band adaptation subspace consists of a first linear projection layer (dimensionality reduction), a nonlinear activation layer, and a second linear projection layer (dimensionality increase) connected sequentially. The corresponding frequency band amplitude components are mapped to a low-dimensional latent space, nonlinearly transformed, and then the original dimension is restored. This structure achieves frequency band-specific amplitude correction with a minimal number of parameters. Different frequency bands have independent learnable parameters, enabling the establishment of nonlinear mappings for different degradation modes such as low-frequency color cast, mid-frequency blurring, and high-frequency noise. This avoids mutual interference caused by shared adapters and achieves frequency band correction with high parameter efficiency and strong expressive power.

[0040] In some embodiments, the process of generating the dynamic routing weights includes: Global pooling is performed on the amplitude spectrum to obtain the global amplitude description vector; The global amplitude description vector is input into the route predictor to obtain the routing responses for the low-frequency, mid-frequency, and high-frequency bands; pass The function normalizes the routing response to obtain low-frequency routing weights, medium-frequency routing weights, and high-frequency routing weights.

[0041] In the above embodiment, global pooling of the amplitude spectrum is used to obtain a global amplitude description vector, which is then input into the route predictor to obtain the routing response for three frequency bands, and then... Normalization yields dynamic routing weights for low, mid, and high frequencies. This mechanism ensures that the routing weights depend entirely on the global distribution of the amplitude spectrum of the current input image. It can adaptively adjust the fusion ratio of each frequency band correction branch according to the actual type and intensity of degradation (higher weights for low frequencies when color cast is severe, and higher weights for high frequencies when noise is severe), achieving adaptive dynamic fusion of the input. Furthermore, the number of parameters in the routing predictor is extremely small, adding almost no computational burden.

[0042] In some embodiments, the aforementioned lightweight spatial branch includes a bottleneck structure, which includes a dimension reduction projection layer, a nonlinear activation layer, and a dimension increase projection layer, for supplementing local structural information and boundary detail information that are not fully expressed by the frequency domain corrected spatial features in the spatial domain.

[0043] In the above embodiments, the lightweight spatial branch employs a bottleneck structure consisting of a dimensionality reduction projection layer, a nonlinear activation layer, and an dimensionality increase projection layer. This structure supplements the local structural and boundary detail information that was not fully expressed after frequency domain correction in the spatial domain. This branch complements the frequency domain correction branch: the frequency domain branch focuses on restoring global color, contrast, and blur, while the spatial branch enhances high-frequency spatial details such as edges and textures. Furthermore, the bottleneck structure ensures an extremely low parameter count. Ultimately, residual fusion preserves both the global structure and restores fine local information, improving the sharpness of the mask boundaries.

[0044] In some embodiments, during the training phase described above, the instance segmentation joint loss function is used to supervise and optimize the category prediction results and the mask prediction results. The joint loss function for instance segmentation includes classification loss, binary cross-entropy mask loss, and Dice mask loss.

[0045] In the above embodiments, a joint loss function comprising classification loss, binary cross-entropy mask loss, and Dice mask loss is used to supervise the prediction results during the training phase. The classification loss ensures accurate instance category discrimination, the binary cross-entropy loss optimizes foreground / background classification pixel-by-pixel, and the Dice loss optimizes the regional overlap between the predicted mask and the ground truth mask, making it particularly suitable for underwater scenarios with imbalanced foreground and background for small targets. The weighted combination of these three factors simultaneously improves mask boundary sharpness, regional integrity, and overall IoU, mitigating mask holes or fragmentation caused by underwater degradation, accelerating convergence, and improving segmentation accuracy.

[0046] Figure 2 This is a schematic diagram of the overall process of an underwater instance segmentation method based on a frequency band decomposition adapter provided in an embodiment of the present invention, combined with... Figure 2 ,right Figure 1 The underwater instance segmentation method based on the frequency band decomposition adapter will be described further.

[0047] In one specific embodiment, combined with Figure 2As shown, the underwater instance segmentation method based on a frequency band decomposition adapter, as presented in this invention, mainly includes steps such as underwater image input, visual basic model feature extraction, frequency domain decomposition, frequency band division, independent frequency band adaptation, dynamic route fusion, inverse Fourier reconstruction, spatial branch supplementation, residual fusion, and instance segmentation decoding. The frequency band decomposition adapter includes a frequency domain decomposition unit, a frequency band mask construction unit, a frequency band independent adaptation unit, a dynamic route fusion unit, a frequency domain reconstruction unit, and a lightweight spatial branch. The frequency domain decomposition unit decomposes underwater image features into amplitude and phase spectra; the frequency band mask construction unit generates low-frequency, mid-frequency, and high-frequency masks; the independent frequency band adaptation unit independently corrects the amplitude components of different frequency bands; the dynamic route fusion unit adaptively fuses the correction results of different frequency bands according to the global amplitude distribution; the frequency domain reconstruction unit reconstructs the corrected frequency domain features into spatial features by combining the phase spectrum; and the lightweight spatial branch supplements local structural information and boundary detail information.

[0048] Specifically, (1) underwater image input and visual basic model feature extraction Input the underwater image to be segmented The data is then input into a pre-trained visual base model for feature extraction. The pre-trained visual base model can be DINOv2, SAM, CLIP, EVA, or other visual base models with general visual representation capabilities. In a preferred embodiment, a visual base model with frozen parameters is used as the backbone network to reduce training overhead and retain the general visual knowledge of the pre-trained model.

[0049] Let the first The underwater image features output by each network layer are: ,in The size is , Indicates the number of channels. and These represent the height and width of the feature map, respectively. The band decomposition adapter can be inserted after one or more network layers of the visual base model to optimize the features output by the corresponding layer. Perform frequency domain adaptation.

[0050] (2) Frequency domain decomposition For the Underwater image features output by each network layer First, a two-dimensional Fourier transform is performed on it to obtain the frequency domain features. The specific expression is as follows: , in, and Represents frequency domain coordinates, and Represents spatial domain coordinates, Represents the imaginary unit. and These represent the height and width of the feature map, respectively.

[0051] Subsequently, frequency domain features Spectrum centering is performed to obtain the amplitude spectrum. and phase spectrum Specifically: , in, This represents the frequency domain characteristics after centering. This indicates a spectrum centering operation. Represents the amplitude spectrum. Represents the phase spectrum. This indicates a modulo operation. This indicates the phase angle taking operation.

[0052] In this embodiment, the amplitude spectrum is mainly used to characterize the impact of underwater degradation on the characteristic energy distribution, while the phase spectrum mainly preserves spatial structure and semantic layout information. Therefore, this invention mainly performs frequency band decomposition and adaptive correction on the amplitude spectrum, and retains the original phase spectrum during the reconstruction stage.

[0053] (3) Frequency band division based on Gaussian function To avoid spectral discontinuities caused by hard-boundary frequency band divisions, this invention employs a Gaussian function to smoothly divide the amplitude spectrum into frequency bands. First, taking the frequency domain center as the origin, the radial distance from any frequency position to the frequency domain center is calculated. : ， in, and Let these represent the coordinates of the frequency domain center in the height and width directions, respectively. This represents the square root operation.

[0054] Then, construct two Gaussian functions with different scale parameters. and : , in, and Let represent the first scale parameter and the second scale parameter, respectively, and Less than ; Used to describe the low-frequency region near the center of the frequency domain Used to describe the low-to-mid frequency region with a wider coverage area.

[0055] based on and Constructing a low-frequency mask Intermediate frequency mask and high frequency mask : , in, , and These represent the low-frequency, mid-frequency, and high-frequency smoothing masks, respectively. This method creates a smooth transition between the low-frequency, mid-frequency, and high-frequency regions, helping to reduce the impact of discontinuities during frequency domain reconstruction.

[0056] Furthermore, the amplitude spectrum By multiplying the three frequency band masks element-wise, the amplitude components of different frequency bands are obtained: ， in, This indicates element-wise multiplication. , and These represent the low-frequency amplitude component, the mid-frequency amplitude component, and the high-frequency amplitude component, respectively.

[0057] (4) Independent frequency band adaptation subspace To differentiate the degradation characteristics across different frequency bands, this invention constructs independent frequency band adaptation subspaces for the low-frequency, mid-frequency, and high-frequency amplitude components. Each frequency band adaptation subspace can consist of a dimension reduction projection layer, a nonlinear activation layer, and a dimension increase projection layer.

[0058] For any frequency band ,in It can be , or Corresponding frequency band amplitude components The correction process can be expressed as: , in, Indicates the first Dimensionality reduction projection parameters in a frequency band adaptation subspace Indicates the first Upward projection parameters in a frequency band adaptation subspace Represents a non-linear activation function. Indicates the first Amplitude correction characteristics corresponding to each frequency band.

[0059] In a preferred embodiment, the dimensionality reduction projection layer reduces the number of channels from... Reduce to The upscaling projection layer will increase the number of channels from Restore to ,in Less than Therefore, frequency band-specific amplitude correction can be achieved with lower parameters, enabling efficient fine-tuning of parameters.

[0060] (5) Dynamic route fusion Color cast, blur, and noise in real underwater images typically appear in mixed forms, and the intensity of different degradations is not fixed in different frequency bands. Therefore, this invention is based on the original amplitude spectrum. The global distribution generates dynamic routing weights to adaptively adjust the contribution of different frequency band branches.

[0061] Specifically, for the amplitude spectrum Global pooling is performed to obtain a global amplitude description vector; this global amplitude description vector is then input into the route predictor. and through The function obtains the routing weights for the three frequency bands: , in, , and These represent the dynamic routing weights corresponding to low frequency, medium frequency, and high frequency, respectively. This represents a route predictor consisting of a global pooling layer and a linear mapping layer.

[0062] Subsequently, the amplitude correction features of different frequency bands are weighted and fused using dynamic routing weights to obtain the final amplitude correction result: , in, This indicates the amplitude correction result after fusion.

[0063] (6) Frequency domain reconstruction and inverse Fourier reconstruction Obtaining amplitude correction results Then, it was compared with the original amplitude spectrum. Residual fusion is performed to obtain the corrected amplitude spectrum. : , Where s represents a settable or learnable scaling factor used to control the amplitude correction intensity.

[0064] Subsequently, the corrected amplitude spectrum Compared with the original phase spectrum Reconstruct the data to obtain the corrected frequency domain features. : , in, This represents a complex phase term composed of the original phase spectrum.

[0065] Finally, the corrected frequency domain characteristics By performing inverse centering and inverse Fourier transform, the spatial features after frequency domain correction are obtained. : , in, This indicates a spectrum inverse centralization operation. This represents the inverse Fourier transform.

[0066] (7) Lightweight spatial branching and residual fusion While frequency domain branching can mitigate feature shifts caused by underwater degradation from the perspective of amplitude modulation, underwater instance segmentation still requires accurate local structure, boundary, and spatial semantic information. Therefore, this invention further introduces lightweight spatial branching to refine the original underwater image features. To supplement spatial semantics.

[0067] Lightweight spatial branching can employ a bottleneck structure, including a dimension reduction projection layer, a nonlinear activation layer, and a dimension increase projection layer, specifically represented as follows: , in, This represents the dimension reduction projection parameters in the spatial branch. This represents the up-dimensional projection parameters in the spatial branch. Represents a non-linear activation function. This represents supplementary features in the representation space.

[0068] Furthermore, the spatial characteristics after frequency domain correction Spatial supplementary features Residual fusion is performed with the original underwater image features X^k to obtain the adapted visual base model features. : , in, Indicates the first Output characteristics of each network layer after processing by a frequency band decomposition adapter. It can be fed into subsequent network layers of the visual base model, or it can be used as one of the input features of the instance segmentation decoder.

[0069] (8) Instance segmentation, decoding and result output The adapted visual base model features The input is fed into the instance segmentation decoder, which performs class prediction and mask prediction on the target instances in the underwater image, and obtains the class prediction results, confidence results and pixel-level mask results for each target instance in the underwater image.

[0070] In one implementation, the instance segmentation decoder employs a query-based mask decoding structure. Specifically, the instance segmentation decoder sets up multiple learnable instance queries and interacts with the adapted visual base model features to obtain instance representations corresponding to different target instances. Subsequently, the classification prediction branch outputs the category prediction results for each instance, and the mask prediction branch outputs the pixel-level mask prediction results for each instance.

[0071] In another implementation, the instance segmentation decoder may also employ a candidate region-based instance segmentation decoding structure or other decoding structures capable of outputting instance categories and instance masks, and the present invention does not limit this to any particular one.

[0072] (9) Model training process During the training phase, underwater images from the training set are input into the pre-trained visual base model and frequency band decomposition adapter to obtain corresponding instance category prediction results and mask prediction results. Based on the difference between the prediction results and the annotation results, the model is supervised and optimized using the instance segmentation joint loss function.

[0073] The joint loss function for instance segmentation includes classification loss, binary cross-entropy mask loss, and Dice mask loss, and its expression is: , in, This represents the classification loss, used to constrain the class prediction results of the target instance; This represents the binary cross-entropy mask loss, used to constrain the pixel-level differences between the predicted mask and the ground truth mask; This represents the Dice mask loss, used to improve the regional overlap between the predicted mask and the ground mask; , and These represent the weighting coefficients of the corresponding loss terms.

[0074] In a preferred embodiment, the main parameters of the pre-trained visual base model are frozen during training, and only the trainable parameters in the band decomposition adapter and instance segmentation decoder are updated. The band decomposition adapter includes a lightweight spatial branch and a dynamic routing fusion unit. This approach reduces the number of training parameters and computational overhead while enabling the visual base model to adapt to complex underwater instance segmentation scenarios.

[0075] (10) Model reasoning process During the inference phase, the underwater image to be segmented is input into the trained model. The model first extracts underwater image features through a pre-trained visual base model, then uses a frequency band decomposition adapter to perform frequency domain correction and spatial supplementation on the underwater image features to obtain adapted visual base model features; finally, the instance segmentation decoder outputs the category, confidence level, and pixel-level mask of the target instance in the underwater image.

[0076] Therefore, this invention can perform feature correction for the differentiated effects of underwater degradation in different frequency bands while maintaining the stability of the main parameters of the visual basic model, and adaptively adjust the contribution of each frequency band branch through a dynamic routing mechanism, thereby improving the target separation capability and mask boundary quality in complex underwater scenes.

[0077] Furthermore, the frequency band decomposition adapter can be inserted after one or more network layers of a pre-trained visual base model to achieve frequency domain correction of features at different levels.

[0078] In a preferred embodiment, the frequency band decomposition adapter includes three frequency band branches: low frequency, medium frequency, and high frequency, which correspond to the low frequency amplitude component, the medium frequency amplitude component, and the high frequency amplitude component, respectively; the dynamic routing weight includes low frequency routing weight, medium frequency routing weight, and high frequency routing weight, which are used to adjust the contribution of the three frequency band branches, respectively.

[0079] The beneficial effects of this invention are as follows: First, this invention models the impact of underwater degradation on the features of the visual basic model as differential perturbations on amplitude components of different frequency bands, thereby enabling more fine-grained feature correction.

[0080] Second, the present invention uses a Gaussian function to construct a smooth frequency band mask, avoiding the problem of spectral discontinuity caused by hard boundary frequency band division, which is beneficial to maintaining feature stability during inverse Fourier transform reconstruction.

[0081] Third, this invention constructs independent frequency band adaptation subspaces for low-frequency, mid-frequency, and high-frequency amplitude components, enabling the model to specifically model degradation information in different frequency bands.

[0082] Fourth, this invention introduces a dynamic routing mechanism that adaptively adjusts the contribution of different frequency band branches based on the global amplitude distribution of the input features, which can adapt to the situation where multiple degradations occur in a mixed manner in real underwater scenarios.

[0083] Fifth, this invention combines lightweight spatial branching to supplement local structural and boundary detail information, so that the frequency domain correction results and spatial semantic information complement each other, thereby improving the accuracy and robustness of underwater instance segmentation.

[0084] This invention proposes an underwater instance segmentation method based on a frequency band decomposition adapter. The core invention lies in: performing Fourier transform on the intermediate features of the visual base model to obtain the amplitude and phase spectra; using a Gaussian function to construct a smooth frequency band mask to decompose the amplitude spectrum into three frequency band components: low-frequency, mid-frequency, and high-frequency; setting an independent adaptation subspace for each frequency band to perform parameter-efficient differentiated amplitude correction; further generating dynamic routing weights based on the global distribution of the amplitude spectrum; adaptively weighting and fusing the correction results of each frequency band; and then fusing them with the residuals of the original amplitude spectrum, combining them with the phase spectrum and reconstructing the frequency domain correction features through inverse Fourier transform; simultaneously introducing lightweight spatial branches to supplement local structure and boundary details; and finally inputting the features fused from the frequency domain and spatial branches into the instance segmentation decoder. This method achieves fine-grained feature correction for the differential impact of underwater degradation on different frequency bands through frequency band decomposition and independent adaptation. A Gaussian smooth mask avoids reconstruction instability caused by spectral discontinuities. A dynamic routing mechanism adaptively adjusts the contribution of each frequency band according to the degradation distribution of the input scene, complementing spatial branching to effectively improve target separation capability and mask boundary quality. It significantly improves instance segmentation accuracy in complex underwater scenes while keeping the core parameters of the visual base model frozen. This method addresses the problems in existing underwater instance segmentation methods that treat underwater degradation as a global domain shift, struggle to characterize color cast, blur, and noise, and other degradation factors that affect the differential impact of these factors on the internal features of the visual base model across different frequency bands. Furthermore, existing frequency domain adaptation methods often focus on overall style alignment without independent correction and adaptive fusion mechanisms for different frequency bands, leading to target-background confusion and poor mask boundary quality in complex underwater scenes.

[0085] The purpose of this invention is to address the problem of target-background confusion caused by degradation such as color cast, blurring, and noise in underwater instance segmentation. It provides an underwater instance segmentation method based on a frequency band decomposition adapter. The method inputs the underwater image into a visual base model, performs Fourier transform on intermediate features at the feature layer to obtain the amplitude and phase spectra, decomposes the amplitude spectrum into multiple frequency bands using a Gaussian frequency band mask, and performs targeted amplitude correction in each independent frequency band subspace. Dynamic routing weights are generated based on the global amplitude distribution, and the correction results of each frequency band are adaptively fused. Then, inverse Fourier reconstruction is performed using the original phase, and the results are fused with the spatial branch feature residuals. Finally, the segmentation decoder outputs the instance category and mask result. This method can model the differentiated impact of underwater degradation on different frequency bands, improving the instance segmentation accuracy in complex underwater scenes.

[0086] The visual base model features are decomposed into amplitude and phase spectra using Fourier transform, and the amplitude spectrum is smoothly divided into multiple frequency bands based on a Gaussian function. Subsequently, independent adaptation subspaces are constructed for different frequency bands to achieve amplitude correction with frequency band differentiation. Furthermore, the contribution of different frequency band branches is adaptively adjusted according to the global amplitude distribution through a dynamic routing mechanism. Finally, lightweight spatial branches are combined to supplement local structural information and boundary detail information, thereby improving the instance segmentation performance in complex underwater scenes.

[0087] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0088] Device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0089] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0090] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as... Figure 1 The method described. For example, in some embodiments, Figure 1 The methods described above can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by computing unit 301, the above-described methods can be performed. Figure 1 One or more steps of the method in the document.

[0091] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0095] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0096] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0097] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0098] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An underwater instance segmentation method based on a frequency band decomposition adapter, characterized in that, The method includes: Input the underwater image to be segmented, and input the underwater image into a pre-trained visual base model to obtain the underwater image features output by at least one network layer in the pre-trained visual base model; The underwater image features are subjected to Fourier transform to obtain the corresponding frequency domain features, and the frequency domain features are decomposed into amplitude spectrum and phase spectrum; Multiple smooth frequency band masks are constructed based on Gaussian functions, and the amplitude spectrum is divided into low-frequency amplitude components, mid-frequency amplitude components, and high-frequency amplitude components using the multiple smooth frequency band masks. The low-frequency amplitude component, mid-frequency amplitude component, and high-frequency amplitude component are respectively input into the corresponding independent frequency band adaptation subspace to obtain the amplitude correction characteristics corresponding to different frequency bands; Dynamic routing weights are generated based on the global distribution of the amplitude spectrum, and the dynamic routing weights are used to perform weighted fusion of amplitude correction features corresponding to different frequency bands to obtain the fused amplitude correction result. The fused amplitude correction result is fused with the original amplitude spectrum to obtain the corrected amplitude spectrum. The corrected amplitude spectrum and the phase spectrum are then reconstructed in the frequency domain, and the frequency-domain corrected spatial features are obtained by inverse Fourier transform. Spatial supplement features are obtained by performing spatial semantic supplementation on the underwater image features through lightweight spatial branching. The frequency-domain corrected spatial features are fused with the spatial supplementary features, and residual connections are made with the underwater image features to obtain the adapted visual basic model features. The adapted visual base model features are input into the instance segmentation decoder, which outputs the category prediction results and mask prediction results of the target instances in the underwater image.

2. The method according to claim 1, characterized in that, The pre-trained visual base model is a visual base model with frozen parameters. The frequency band decomposition adapter is inserted after at least one network layer of the pre-trained visual base model, and only the parameters in the frequency band decomposition adapter and the instance segmentation decoder are trained.

3. The method according to claim 1, characterized in that, The underwater image features are subjected to Fourier transform to obtain the corresponding frequency domain features, and the frequency domain features are decomposed into amplitude spectrum and phase spectrum, including: For the Underwater image features output by each network layer Perform a two-dimensional Fourier transform to obtain the frequency domain features. ; for the frequency domain features Perform spectrum centering processing to obtain the centered frequency domain features; The modulus of the centered frequency domain feature is calculated as the amplitude spectrum. The phase angle of the centered frequency domain feature is calculated as the phase spectrum. .

4. The method according to claim 1, characterized in that, The method of constructing multiple smooth frequency band masks based on Gaussian functions includes: Using the frequency domain center as the origin, calculate the radial distance from each frequency position to the frequency domain center; Based on the radial distance, construct a first Gaussian function and a second Gaussian function with different scale parameters; The first Gaussian function is used as the low-frequency mask, the difference between the second Gaussian function and the first Gaussian function is used as the mid-frequency mask, and the result of subtracting the second Gaussian function from the value 1 is used as the high-frequency mask.

5. The method according to claim 1, characterized in that, The low-frequency amplitude component, mid-frequency amplitude component, and high-frequency amplitude component are obtained by multiplying the amplitude spectrum element-wise with the low-frequency mask, mid-frequency mask, and high-frequency mask, respectively.

6. The method according to claim 1, characterized in that, The independent frequency band adaptation subspace includes a first linear projection layer, a nonlinear activation layer, and a second linear projection layer connected in sequence. The first linear projection layer is used to map the amplitude components of the corresponding frequency band to a low-dimensional latent space, the nonlinear activation layer is used to introduce nonlinear correction capability, and the second linear projection layer is used to map the low-dimensional latent space features back to the original channel dimension to obtain the amplitude correction features of the corresponding frequency band.

7. The method according to claim 1, characterized in that, The process of generating the dynamic routing weights includes: Global pooling is performed on the amplitude spectrum to obtain a global amplitude description vector; The global amplitude description vector is input into the route predictor to obtain the routing responses for the low-frequency, mid-frequency, and high-frequency bands; pass The function normalizes the routing response to obtain low-frequency routing weights, medium-frequency routing weights, and high-frequency routing weights.

8. The method according to claim 1, characterized in that, The lightweight spatial branch includes a bottleneck structure, which comprises a dimension reduction projection layer, a nonlinear activation layer, and a dimension increase projection layer, used to supplement the local structural information and boundary detail information that are not fully expressed in the frequency domain-corrected spatial features in the spatial domain.

9. The method according to claim 1, characterized in that, During the training phase, the instance segmentation joint loss function is used to supervise and optimize the category prediction results and mask prediction results. The instance segmentation joint loss function includes classification loss, binary cross-entropy mask loss, and Dice mask loss.

10. An electronic device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory is characterized in that it stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method according to any one of claims 1 to 9.

Images