Underwater new class instance segmentation method and device based on prototype-driven SAM

Abstract

The embodiment of the application provides a kind of underwater new class instance segmentation method and equipment based on prototype drive SAM, it is first by the cross-domain style alignment of frequency domain low-frequency amplitude selective injection and phase spectrum preservation;Utilize the consistency check of back projection and real mask Dynamic filtering only relies on the background artifact that cannot be distinguished by forward propagation, then the spatial heteroscedastic uncertainty modeling is introduced into the candidate point regression process, and in the clustering stage, the uncertainty is used as an exponential penalty factor to nonlinearly amplify the candidate point distance, forcibly break the false visual connection, realize the robust decoupling of adhesion instance, then feed into SAM decoder, output high-precision mask by the zero-sample generalization ability of large model, significantly improve the discovery accuracy and positioning robustness of underwater new class target, strong anti-interference, good morphological adaptability, and edge segmentation is accurate.

Description

Technical Field

[0001] This invention relates to the fields of image information processing and computer vision technology, and in particular to a method and device for underwater new category instance segmentation based on prototype-driven SAM. Background Technology

[0002] Instance segmentation constitutes a core perception capability for ocean exploration and underwater robot operation. However, in actual deployments, target categories are often temporarily assigned for specific tasks, and only a very small number of reference samples are available (i.e., the few-sample instance segmentation problem). In addition, severe visual degradation factors in the underwater environment, such as water turbidity, dynamic lighting, and color distortion, pose a significant challenge to accurate instance segmentation for specifying new categories.

[0003] A direct paradigm for solving few-shot instance segmentation is "pre-training followed by fine-tuning," which involves fine-tuning the model's parameters using a small number of reference samples during the inference phase. The paper "One-shot instance segmentation" presents pioneering work using Siamese networks for single-shot segmentation. However, most existing few-shot instance segmentation methods are designed for natural or medical images. When transferred to underwater scenes, water turbidity and color distortion lead to severe feature misalignment between the support set (reference image) and the query set (image to be segmented). The paper "Frustratingly simple few-shot object detection" points out that fine-tuning mechanisms based on Region Proposal Networks (RPNs) are prone to overfitting under extremely limited data and often suffer from instance sticking and localization failures in dense underwater scenes with blurred boundaries. Furthermore, the paper "Reference twice: A simple and unified baseline for few-shot instance segmentation" observes that fine-tuning paradigms under extreme data scarcity are highly susceptible to exacerbating "base class bias," causing a large number of background elements to be misclassified as foreground objects.

[0004] In recent years, large-scale visual models such as the Segment Anything Model (SAM) have demonstrated powerful zero-shot generalization capabilities. The paper "Segment Anything" points out that large models, with their cue-driven reasoning mechanisms and excellent boundary awareness, have a natural advantage in avoiding RPN dependence and base class bias. For underwater scenarios, some studies have attempted to adapt large models to specific domains. For example, the paper "Diving into underwater: Segment Anything model guided underwater salient instance segmentation and a large-scale dataset" customized the USIS-SAM model for underwater salient targets; the paper "Empowering DINO Representations for Underwater Instance Segmentation via Aligner and Prompter" proposed DiveSeg, which enhances discovery capabilities by aligning DINO features with underwater color styles. However, SAM mask generation highly depends on sufficient cue information, making it difficult to construct effective cuees with very few reference samples. Furthermore, most of these methods still rely on tedious parameter fine-tuning, facing a severe style gap when directly transferred to underwater scenarios.

[0005] To overcome the limitations of fine-tuning and construct effective cues, prototype learning methods have been introduced. However, existing solutions propose prototype networks that compress samples into compact prototype vectors and measure similarity, an approach naturally suited to single-sample scenarios. Nevertheless, traditional prototype learning is highly susceptible to inaccurate prototype extraction from single samples when dealing with the variable visual degradation and target morphology changes underwater, leading to the failure of the final semantic cues. Summary of the Invention

[0006] This invention proposes a new underwater category instance segmentation method and device based on prototype-driven SAM, which solves the problem that existing technical solutions are prone to inaccurate extraction of single sample prototypes when dealing with the ever-changing visual degradation and target shape changes underwater, resulting in the failure of the final output semantic prompt signal.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: According to a first aspect of the present invention, a method for segmenting new underwater categories of instances based on prototype-driven SAM is provided, the method comprising: Obtain the query image of the underwater scene, as well as the supporting images containing the new category of targets and their corresponding ground truth masks; Cross-domain style alignment is performed on the supporting images, and low-frequency selective injection in the frequency domain is used to preserve high-frequency geometry while eliminating underwater color cast and turbidity interference. The style-aligned support features are input into the style-aligned prototype refinement module to build a closed-loop coupling mechanism of forward mining and reverse consistency verification, dynamically filter out underwater background noise that cannot be distinguished by forward propagation alone, and extract the pure category foreground prototype and the adaptively constructed dynamic background prototype. The foreground prototype, dynamic background prototype, and query image features are input into the uncertainty-guided instance separation module. Using a heteroscedastic uncertainty weighting strategy pre-designed for underwater blurred boundaries, the distance between candidate points is dynamically penalized to decouple stuck instances, generating a geometric cue set containing bounding boxes and positive and negative point mixtures. The geometric cue set and the query image features are input into the decoder of the Segmentation All Model (SAM). A cross-domain prior alignment method without parameter adjustment is used to output the instance segmentation mask of the new class of target in the query image.

[0008] In some implementations of the first aspect, the above-mentioned cross-domain style alignment processing for supporting images includes: Perform Fourier transforms on the supporting image and the query image respectively, and extract their amplitude spectrum and phase spectrum; To preserve the phase spectrum of the supporting image and maintain the geometric structure of the target, amplitude spectrum mixing is performed on the low-frequency region isolated by a preset low-frequency binary mask, satisfying the formula: ; in, and These represent the amplitude spectra of the supporting image and the query image, respectively. A binary mask representing the isolation of low-frequency regions. Indicates the mixing coefficient. This represents element-wise multiplication; The mixed amplitude spectrum and the phase spectrum of the supporting image are reconstructed by inverse Fourier transform to generate a style-aligned image and corresponding features.

[0009] Among some implementations of the first aspect, the above method also includes: An initial foreground prototype is built on supporting features using masked average pooling; By calculating the forward cosine similarity between the initial foreground prototype and the query features, a query pseudomask is generated. Based on the pseudomask, the query features are then subjected to mask average pooling again to aggregate and generate an adaptive prototype that includes the morphological changes of the query image. The adaptive prototype is back-projected onto the supporting features, and a back attention map is calculated to measure the degree of matching between the adaptive prototype and the supporting features at each spatial location. The back attention map is then multiplied element-wise with the ground truth mask of the supporting image and summed. Finally, the sum is compared with an activation function and a threshold to generate a reliability gating signal. Based on the initial foreground prototype, the adaptive prototype, and the reliability gating signal, the refined category foreground prototype after reverse verification and correction is obtained through residual update calculation.

[0010] In some implementations of the first aspect, the construction of the aforementioned dynamic background prototype includes: A dynamic background prototype with mask initialization is obtained based on the inversion of the real mask. The dynamic background prototype is updated by using the adaptive prototype part that has not been verified as a reliable foreground identified by the reliability gating signal. The initial dynamic background prototype and the adaptive prototype filtered by the reliability gating signal are weighted and fused according to the preset update intensity, and then normalized so that the dynamic background prototype can adaptively absorb the interference features that are difficult to distinguish from the foreground in the current underwater scene. In the dense feature matching stage, the category foreground prototype, which has been corrected by reverse verification, and the dynamic background prototype, which has absorbed the interference features of the current underwater scene, compete probabilities to calculate the matching probability of each position in the query feature map belonging to the foreground prototype and the background prototype, respectively. The foreground probability map is then output through a normalized exponential function to display visually similar underwater environmental interference objects that have been suppressed.

[0011] In some implementations of the first aspect, the aforementioned uncertainty-guided instance separation module utilizes a pre-designed heteroscedastic uncertainty weighting strategy for underwater fuzzy boundaries to dynamically penalize candidate point distances to decouple adhered instances, including: Upsampling and smoothing are performed based on the foreground probability map. To address the uneven distribution of underwater noise, low-confidence noise is filtered out using dynamic thresholding, and discrete candidate point sets are generated through multinomial sampling. For each candidate point, after extracting its feature vector, the refined category foreground prototype, which has been corrected by reverse verification, is used as the semantic anchor point, and prototype-guided feature correction is performed through a cross-attention mechanism. Attention is calculated using candidate point features as query terms and foreground prototypes as key terms. The attention-enhanced features are superimposed on the original candidate point features and normalized. The discriminative power of candidate point features is enhanced by using pure global category semantics, thus narrowing the distance between candidate points belonging to the same instance in the feature space.

[0012] Among some implementations of the first aspect, the above method also includes: Based on spatial heteroscedasticity uncertainty modeling, a multilayer perceptron is used to simultaneously predict the coordinate offset and spatial heteroscedasticity uncertainty variance of each candidate point. The regression loss function adopts an uncertainty weighted form. For each candidate point, the coordinate regression error of the point is weighted by the reciprocal of its prediction variance, and a logarithmic regularization term of the variance is added. At the same time, an absolute value regularization term for underwater outlier noise is introduced. The average value of the loss of all candidate points is taken as the total regression loss.

[0013] In some implementations of the first aspect, the regression loss function described above satisfies the formula: ; in, For the candidate point set, This represents the total number of candidate points; Index for candidate points; For the network prediction of the first The coordinate offset of each candidate point; For the first The true coordinate offset of each candidate point; For the network prediction of the first Spatial heteroscedasticity uncertainty variance of candidate points; Represents the square of the L2 norm; The coefficients for the L1 regularization term are added to address underwater outlier noise.

[0014] In some implementations of the first aspect, the above utilizes a pre-designed heteroscedastic uncertainty weighting strategy for underwater fuzzy boundaries to dynamically penalize candidate point distances to decouple adhering instances, generating a set of geometric hints containing bounding boxes and a mixture of positive and negative points, including: Calculate the Euclidean distance between two candidate points, and then multiply it by the value of an exponential function with the larger uncertainty variance of the two candidate points as the independent variable as a penalty factor to obtain the uncertainty-weighted distance. The clustering distance metric uses an exponential function to nonlinearly accelerate and amplify the distance between boundary points that meet the preset high uncertainty, thereby exponentially expanding the distance between high uncertainty boundary points, forcibly breaking the false visual connections between dense underwater targets, and separating the sticky candidate points into independent clusters. For each independent target cluster, a bounding box containing extended tolerance is extracted, and high-confidence positive points are sampled from the target cluster, while negative points are sampled from neighboring clusters. These are combined to form a semantically clean and mutually isolated set of geometric hints that is output to the decoder of the segmentation model.

[0015] In some implementations of the first aspect, the features of the query image and the features of the supporting image are both extracted by a shared-weight unlabeled distillation DINO feature extraction network, and the network keeps its parameters frozen during the inference phase.

[0016] According to a second aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and memory that is communicatively connected to at least one processor; The invention is characterized in that the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to implement the method of the first aspect of the invention.

[0017] This invention provides an underwater novel category instance segmentation method and device based on prototype-driven SAM. This method solves the feature mismatch problem by selectively injecting low-frequency amplitude in the frequency domain and maintaining cross-domain style alignment with phase spectrum preservation, isolating high-frequency scattering noise at the source while preserving geometric structure. Furthermore, it constructs a closed-loop coupling mechanism of "forward adaptive prototype mining - reverse attention gating verification," dynamically filtering out background artifacts that cannot be identified by forward propagation alone by using consistency verification between back projection and the real mask. Simultaneously, it introduces a dynamic background prototype that can adaptively absorb scene interference features, competing probabilistically with a refined foreground prototype to explicitly suppress similar interference, thereby extracting a high-fidelity pure category prior. Based on this, addressing instance edge blurring and adhesion caused by underwater optical scattering, it innovatively introduces spatial heteroscedasticity uncertainty modeling into the candidate point regression process. This allows the network to autonomously output high variance for blurred boundaries to reduce deterministic regression bias. In the clustering stage, this uncertainty is used as an exponential penalty factor to nonlinearly amplify the distance between candidate points, forcibly breaking false visual connections and achieving robust decoupling of adhered instances. Finally, the pure geometric clues, validated through closed-loop verification and decoupled from uncertainty, are directly fed into the Segment All Model (SAM) decoder, leveraging the zero-shot generalization capability of the large model to output a high-precision mask. Thus, this invention, without requiring any parameter fine-tuning, completely avoids the spatial overfitting and base class bias traps that easily occur in traditional few-shot fine-tuning paradigms under extremely scarce underwater conditions. It significantly improves the detection accuracy and localization robustness of new underwater target categories, achieving technical effects such as strong anti-interference capability, good morphological adaptability, and accurate edge segmentation. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1The diagram illustrates a flowchart of a prototype-driven SAM-based underwater new category instance segmentation method provided by an embodiment of the present invention. Figure 2 This diagram illustrates the overall network architecture of a prototype-driven SAM-based underwater new category instance segmentation method provided by an embodiment of the present invention. Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present invention is shown. Detailed Implementation

[0020] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0022] In order to solve the above-mentioned technical problems, this invention discloses an underwater new category instance segmentation method and device based on prototype-driven SAM. First, by selectively injecting low-frequency amplitude in the frequency domain and maintaining cross-domain style alignment with phase spectrum preservation, high-frequency scattering noise is isolated from the source while preserving geometric structure, thus solving the feature mismatch problem. Then, a closed-loop coupling mechanism of "forward adaptive prototype mining - reverse attention gating verification" is constructed. The consistency verification between back projection and the real mask is used to dynamically filter out background artifacts that cannot be distinguished by forward propagation alone. At the same time, a dynamic background prototype that can adaptively absorb scene interference features is introduced to compete with the refined foreground prototype in probability to explicitly suppress similar interference objects, thereby extracting a high-fidelity pure category prior. On this basis, in order to address the instance edge blurring and adhesion caused by underwater optical scattering, spatial heteroscedasticity uncertainty modeling is introduced into the candidate point regression process for the first time. This allows the network to autonomously output high variance for blurred boundaries to reduce deterministic regression bias. In the clustering stage, this uncertainty is used as an exponential penalty factor to nonlinearly amplify the distance of candidate points, forcibly breaking false visual connections and achieving robust decoupling of adhesion instances. Finally, the pure geometric clues, validated through closed-loop verification and decoupled from uncertainty, are directly fed into the Segment All Model (SAM) decoder, leveraging the zero-shot generalization capability of the large model to output a high-precision mask. Thus, this invention, without requiring any parameter fine-tuning, completely avoids the spatial overfitting and base class bias traps that easily occur in traditional few-shot fine-tuning paradigms under extremely scarce underwater conditions. It significantly improves the detection accuracy and localization robustness of new underwater target categories, achieving technical effects such as strong anti-interference capability, good morphological adaptability, and accurate edge segmentation.

[0023] The technical solutions provided by the embodiments of the present invention will now be described with reference to the accompanying drawings.

[0024] Figure 1 A flowchart illustrating a method for segmenting new underwater categories based on prototype-driven SAM according to an embodiment of the present invention is shown.

[0025] like Figure 1 As shown, the underwater new category instance segmentation method based on prototype-driven SAM can include: S101, Obtain the query image of the underwater scene, as well as the supporting image containing the new category of targets and the corresponding ground truth mask; S102 performs cross-domain style alignment processing on the supported images, using selective injection of low frequencies in the frequency domain to preserve high-frequency geometry while eliminating underwater color shift and turbidity interference. S103 inputs the style-aligned support features into the style-aligned prototype refinement module, constructs a closed-loop coupling mechanism for forward mining and reverse consistency verification, dynamically filters out underwater background noise that cannot be distinguished by forward propagation alone, and extracts the pure category foreground prototype and the adaptively constructed dynamic background prototype. S104, input the foreground prototype, dynamic background prototype and query image features into the uncertainty-guided instance separation module, and use the heteroscedastic uncertainty weighting strategy designed in advance for underwater blurred boundaries to dynamically penalize the distance of candidate points to decouple the stuck instances and generate a geometric cue set containing bounding boxes and positive and negative point mixtures. S105 inputs the geometric cue set and query image features into the decoder of the Segmentation Model (SAM), and outputs the instance segmentation mask of the new class of target in the query image using a parameter-free cross-domain prior alignment method.

[0026] In some embodiments, the above-described cross-domain style alignment processing of supporting images includes: Perform Fourier transforms on the supporting image and the query image respectively, and extract their amplitude spectrum and phase spectrum; To preserve the phase spectrum of the supporting image and maintain the geometric structure of the target, amplitude spectrum mixing is performed on the low-frequency region isolated by a preset low-frequency binary mask, satisfying the formula: ; in, and These represent the amplitude spectra of the supporting image and the query image, respectively. A binary mask representing the isolation of low-frequency regions. Indicates the mixing coefficient. This represents element-wise multiplication; The mixed amplitude spectrum and the phase spectrum of the supporting image are reconstructed by inverse Fourier transform to generate a style-aligned image and corresponding features.

[0027] In some embodiments, the above method may further include: An initial foreground prototype is built on supporting features using masked average pooling; By calculating the forward cosine similarity between the initial foreground prototype and the query features, a query pseudomask is generated. Based on the pseudomask, the query features are then subjected to mask average pooling again to aggregate and generate an adaptive prototype that includes the morphological changes of the query image. The adaptive prototype is back-projected onto the supporting features, and a back attention map is calculated to measure the degree of matching between the adaptive prototype and the supporting features at each spatial location. The back attention map is then multiplied element-wise with the ground truth mask of the supporting image and summed. Finally, the sum is compared with an activation function and a threshold to generate a reliability gating signal. Based on the initial foreground prototype, the adaptive prototype, and the reliability gating signal, the refined category foreground prototype after reverse verification and correction is obtained through residual update calculation.

[0028] In some embodiments, the construction of the above-described dynamic background prototype includes: A dynamic background prototype with mask initialization is obtained based on the inversion of the real mask. The dynamic background prototype is updated by using the adaptive prototype part that has not been verified as a reliable foreground identified by the reliability gating signal. The initial dynamic background prototype and the adaptive prototype filtered by the reliability gating signal are weighted and fused according to the preset update intensity, and then normalized so that the dynamic background prototype can adaptively absorb the interference features that are difficult to distinguish from the foreground in the current underwater scene. In the dense feature matching stage, the category foreground prototype, which has been corrected by reverse verification, and the dynamic background prototype, which has absorbed the interference features of the current underwater scene, compete probabilities to calculate the matching probability of each position in the query feature map belonging to the foreground prototype and the background prototype, respectively. The foreground probability map is then output through a normalized exponential function to display visually similar underwater environmental interference objects that have been suppressed.

[0029] In some embodiments, the uncertainty-guided instance separation module described above utilizes a heteroscedastic uncertainty weighting strategy pre-designed for underwater fuzzy boundaries to dynamically penalize candidate point distances to decouple adhered instances, including: Upsampling and smoothing are performed based on the foreground probability map. To address the uneven distribution of underwater noise, low-confidence noise is filtered out using dynamic thresholding, and discrete candidate point sets are generated through multinomial sampling. For each candidate point, after extracting its feature vector, the refined category foreground prototype, which has been corrected by reverse verification, is used as the semantic anchor point, and the prototype-guided feature correction is performed through the cross attention mechanism. Attention is calculated using candidate point features as query terms and foreground prototypes as key terms. The attention-enhanced features are superimposed on the original candidate point features and normalized. The discriminative power of candidate point features is enhanced by using pure global category semantics, thus narrowing the distance between candidate points belonging to the same instance in the feature space.

[0030] In some embodiments, the above method further includes: Based on spatial heteroscedasticity uncertainty modeling, a multilayer perceptron is used to simultaneously predict the coordinate offset and spatial heteroscedasticity uncertainty variance of each candidate point. The regression loss function adopts an uncertainty weighted form. For each candidate point, the coordinate regression error of the point is weighted by the reciprocal of its prediction variance, and a logarithmic regularization term of the variance is added. At the same time, an absolute value regularization term for underwater outlier noise is introduced. The average value of the loss of all candidate points is taken as the total regression loss.

[0031] In some embodiments, the above regression loss function satisfies the formula: ; in, For the candidate point set, This represents the total number of candidate points; Index for candidate points; For the network prediction of the first The coordinate offset of each candidate point; For the first The true coordinate offset of each candidate point; For the network prediction of the first Spatial heteroscedasticity uncertainty variance of candidate points; Represents the square of the L2 norm; The coefficients for the L1 regularization term are added to address underwater outlier noise.

[0032] In some embodiments, the above-described heteroscedastic uncertainty weighting strategy, pre-designed for underwater fuzzy boundaries, dynamically penalizes candidate point distances to decouple adhering instances, generating a set of geometric hints containing bounding boxes and a mixture of positive and negative points, including: Calculate the Euclidean distance between two candidate points, and then multiply it by the value of an exponential function with the larger uncertainty variance of the two candidate points as the independent variable as a penalty factor to obtain the uncertainty-weighted distance. The clustering distance metric uses an exponential function to nonlinearly accelerate and amplify the distance between boundary points that meet the preset high uncertainty, thereby exponentially expanding the distance between high uncertainty boundary points, forcibly breaking the false visual connections between dense underwater targets, and separating the sticky candidate points into independent clusters. For each independent target cluster, a bounding box containing extended tolerance is extracted, and high-confidence positive points are sampled from the target cluster, while negative points are sampled from neighboring clusters. These are combined to form a semantically clean and mutually isolated set of geometric hints that is output to the decoder of the segmentation model.

[0033] In some embodiments, the features of the query image and the features of the supporting image are both extracted by a shared-weight unlabeled distillation DINO feature extraction network, and the network keeps its parameters frozen during the inference phase.

[0034] This invention provides an underwater novel category instance segmentation method based on prototype-driven SAM. It addresses the feature mismatch problem by selectively injecting low-frequency amplitude in the frequency domain and maintaining cross-domain style alignment with phase spectrum preservation, thus isolating high-frequency scattering noise and preserving geometric structure at the source. Furthermore, it constructs a closed-loop coupling mechanism of "forward adaptive prototype mining - reverse attention gating verification," dynamically filtering out background artifacts that cannot be identified by forward propagation alone through consistency verification between back projection and the real mask. Simultaneously, it introduces a dynamic background prototype that adaptively absorbs scene interference features, competing probabilistically with a refined foreground prototype to explicitly suppress similar interference, thereby extracting a high-fidelity pure category prior. Building upon this, and addressing instance edge blurring and adhesion caused by underwater optical scattering, it innovatively introduces spatial heteroscedasticity uncertainty modeling into the candidate point regression process. This allows the network to autonomously output high variance for blurred boundaries to reduce deterministic regression bias. During the clustering stage, this uncertainty is used as an exponential penalty factor to nonlinearly amplify the distance between candidate points, forcibly breaking false visual connections and achieving robust decoupling of adhered instances. Finally, the pure geometric clues, validated through closed-loop verification and decoupled from uncertainty, are directly fed into the Segment All Model (SAM) decoder, leveraging the zero-shot generalization capability of the large model to output a high-precision mask. Thus, this invention, without requiring any parameter fine-tuning, completely avoids the spatial overfitting and base class bias traps that easily occur in traditional few-shot fine-tuning paradigms under extremely scarce underwater conditions. It significantly improves the detection accuracy and localization robustness of new underwater target categories, achieving technical effects such as strong anti-interference capability, good morphological adaptability, and accurate edge segmentation.

[0035] Figure 2 This is a schematic diagram of the overall network architecture of an underwater new category instance segmentation method based on prototype-driven SAM provided in an embodiment of the present invention, combined with... Figure 2 ,right Figure 1 The description of the underwater new category instance segmentation method based on prototype-driven SAM continues.

[0036] In one specific embodiment, combined with Figure 2 As shown, the underwater new category instance segmentation method based on prototype-driven SAM of the present invention may specifically include: Obtain query images of underwater scenes, as well as a very small number of supporting images containing new target categories and their corresponding ground truth masks; Cross-domain style alignment is performed on the supporting and query images. By selectively injecting low frequencies in the frequency domain, high-frequency scattering noise is isolated while eliminating underwater color shift and turbidity interference, thus preserving the target geometry for subsequent prototype learning. Style alignment features are input into the style alignment prototype refinement module to build a closed-loop coupling mechanism of forward mining and reverse consistency verification. This dynamically filters out underwater background artifacts that cannot be distinguished by forward propagation alone, and extracts the pure category foreground prototype corrected by reverse verification and the adaptively constructed dynamic background prototype. The foreground prototype, dynamic background prototype, and query image features are input into the uncertainty-guided instance separation module. Using a joint strategy of heteroscedastic uncertainty modeling and exponential distance penalty designed for underwater fuzzy boundaries, candidate points are dynamically penalized to force decoupling of adhering instances, generating a semantically pure and mutually isolated set of geometric hints containing bounding boxes and a mixture of positive and negative points. The hybrid geometric cue set and query image features are input into the mask decoder of SAM. A cross-domain prior alignment method without parameter fine-tuning is used to directly output the instance segmentation mask of the new class of target in the query image.

[0037] Furthermore, cross-domain style alignment is performed on the supporting images and the query images, specifically including: Fast Fourier Transform is performed on the supporting image and the query image respectively to extract their amplitude spectrum and phase spectrum; Preserving the phase spectrum of the supporting image to maintain the target's geometry, and addressing the issues of underwater high-frequency components containing scattering noise from suspended particles and cross-contamination easily caused by conventional full-band mixing, a preset low-frequency binary mask is used. Directional stripping and mixing of amplitude spectra; By generating style-aligned images and corresponding features through inverse fast Fourier transform, underwater color cast and turbidity interference are eliminated, achieving dual protection of the target's geometric structure.

[0038] Furthermore, the process of dynamically extracting the pure category foreground prototype and dynamic background prototype in the style alignment prototype refinement module includes: Initial foreground prototypes are built on supporting features using masked average pooling. : ; in, , These represent the horizontal and vertical coordinates of the spatial location on the feature map, respectively. This is a support feature for style alignment. To support the true mask of the image, To prevent extremely small constants with a denominator of zero; To address the mismatch between the initial prototype and query features caused by changes in the underwater target's morphology, an initial foreground prototype is calculated. With query features Forward cosine similarity is used to generate a query pseudomask. Based on this pseudomask, an adaptive prototype containing morphological changes of the query image is generated. : ; To prevent forward excavation from introducing background noise in the underwater scattering environment, an adaptive prototype will be used. Back projection to supporting features Calculate the back attention map and the real mask supporting the image. Comparisons are made to generate reliability gating signals. ; Based on the initial foreground prototype Adaptive Prototype and reliability gating signals The refined category foreground prototype, corrected by reverse verification, is obtained through residual update calculation. .

[0039] Furthermore, to address the issue that static background prototypes cannot adapt to dynamically changing similar interfering objects in complex underwater environments, the construction and competition mechanism of dynamic background prototypes includes: The dynamic background prototype is initialized with the inverted support mask and updated by incorporating query features rejected by the inverted gating signal. ; In the dense feature matching stage, the foreground prototype and a dynamic background prototype that incorporates the interference characteristics of the current underwater scene. pass The operation employs probability competition to explicitly suppress visually similar underwater environmental distractions and outputs a foreground probability map.

[0040] Furthermore, the uncertainty-guided instance separation module's process for candidate point generation and clustering includes: Upsampling and smoothing are performed based on the foreground probability map. To address the uneven distribution of underwater noise, low-confidence noise is filtered out using dynamic thresholding, and discrete candidate point sets are generated through multinomial sampling. Due to the semantic ambiguity of local features at individual candidate points caused by underwater scattering, for each candidate point, its feature vector is extracted and refined using a foreground prototype corrected by back-validation. As a semantic anchor, prototype-guided feature correction is performed through a cross-attention mechanism, which enhances the discriminative power of candidate point features by utilizing pure global category semantics, and narrows the distance between candidate points belonging to the same instance in the feature space.

[0041] Furthermore, to address the challenges of blurred instance edges and uncertain determination of boundary pixel attribution caused by underwater optical scattering, this invention also includes spatial heteroscedasticity uncertainty modeling, which uses a multilayer perceptron to simultaneously predict the coordinate offset of each candidate point. Spatial heteroscedasticity, uncertainty, and variance This enables the network to autonomously output high variance when encountering indistinguishable blurry edges. The regression penalty weight in this area is adaptively reduced to avoid positioning errors caused by forcibly performing deterministic regression on the ambiguous boundary caused by underwater scattering.

[0042] Furthermore, spatial heteroscedasticity uncertainty is utilized. As a key criterion for identifying fuzzy boundaries, adhered instances are decoupled into independent sets of geometric cues containing bounding boxes and a mixture of positive and negative points, including: HDBSCAN clustering is performed using the coordinates of candidate points corrected for coordinate offset, and prediction uncertainty is introduced into the clustering distance metric. As an exponential weight, in response to the boundary blurring and instance visual adhesion characteristics caused by underwater optical scattering, the uncertainty of the exponential function is used as a penalty factor to nonlinearly accelerate and amplify the blurred region, thereby exponentially increasing the distance between high uncertainty boundary points, forcibly breaking the false visual connection between dense underwater targets, and separating the adhered candidate points into independent clusters. For each independent target cluster, a bounding box containing extended tolerance is extracted, and high-confidence positive points are sampled from the target cluster, while negative points are sampled from neighboring clusters. These are combined to form a semantically clean and mutually isolated geometric cue set that is output to the SAM decoder.

[0043] The technical solution of the present invention will be described in detail below with reference to specific calculation formulas and processes.

[0044] This invention proposes a novel underwater category instance segmentation framework based on prototype-driven SAM (UW-ProSAM). Addressing the cascading challenges of underwater visual degradation and extreme data scarcity, this framework transforms a limited number of reference samples into an optimization-free inference process. It primarily consists of three deeply coupled core modules: a SAPR module for cross-domain style alignment and refined prototype extraction, a UGIS module responsible for instance separation and cue generation, and a SAM decoder module. The specific implementation steps are as follows: (1) Cross-domain style alignment and prototype refinement extraction tasks.

[0045] In underwater scenarios, dynamic lighting and suspended particles can cause significant style discrepancies between supporting and query images. To address the feature mismatch problem, this invention introduces frequency domain cross-domain style alignment. Amplitude and phase spectra are extracted using Fast Fourier Transform, strictly preserving the phase spectrum of the supporting image. To maintain the geometric structure and address the issues of underwater high-frequency components containing scattered noise from suspended particles and the potential for cross-contamination caused by conventional full-band mixing, a pre-defined low-frequency binary mask is used. The directional stripping and mixing of amplitude spectra is illustrated below: ; Finally, through inverse transformation Generate style-aligned images and features to eliminate underwater color casts and turbidity interference, while achieving dual protection of the target geometry: ; After obtaining the alignment features, style alignment prototypes are refined. First, masked average pooling is used to refine the supporting features. Establish the initial foreground prototype : ; To address the mismatch between the initial prototype and query features caused by changes in the morphology of underwater targets, forward cosine similarity is calculated for the query features. High-response regions are identified in the data, and query pseudomasks are generated. It also aggregates an adaptive prototype that includes the actual shape changes of the query image. : ; To prevent forward mining from introducing background noise in the underwater scattering environment, a reverse consistency verification mechanism is introduced, in which the adaptive prototype is back-projected back to the back attention map calculated from the supporting features. and the real mask Compare and generate reliability-gated signals : ; ; Finally, the pure foreground prototype, corrected by reverse verification, is obtained through residual refinement. Meanwhile, to address the issue that static background prototypes cannot adapt to dynamically changing similar interfering objects in complex underwater environments, a dynamic background prototype is established. It absorbs query features rejected by gating signals for adaptive updates: ; ; In the feature matching stage, the category foreground prototype is corrected through reverse validation. With a dynamic background prototype that incorporates the interference characteristics of the current underwater scene pass The operation involves probability competition, explicitly suppressing visually similar underwater environmental distractions and outputting a foreground probability map.

[0046] (2) Uncertainty-guided spatial instance separation task To address the issues of instance sticking and boundary blurring in dense underwater scenes, this task aims to transform abstract prototypes into clear geometric cues. First, considering the uneven distribution of underwater noise, the foreground probability map is smoothed, dynamically thresholded, and multinomial sampled to generate a discrete set of candidate points. Since underwater scattering causes semantic ambiguity in the local features of individual candidate points, a refined foreground prototype is used as the basis for further analysis. Using semantic anchors, feature correction is performed through cross-attention, leveraging pure global category semantics to bring similar candidate points closer together. Distance in feature space: ; To address the challenges of blurred instance edges and uncertain determination of boundary pixel attribution caused by underwater optical scattering, this invention designs a spatial heteroscedasticity uncertainty model. The network simultaneously predicts the coordinate offsets of candidate points using a multilayer perceptron. Spatial heteroscedasticity, uncertainty, and variance Its regression loss function is defined as: ; When encountering indistinguishable fuzzy edges, the network dynamically outputs high variance to reduce regression penalties. In the HDBSCAN clustering during the inference phase, this spatial heteroscedasticity uncertainty is utilized. As a key criterion for identifying fuzzy boundaries, it is used as an exponential weight to scale the cluster distance: ; This exponentially increases the distance between highly uncertain and fuzzy boundary points, forcibly separating clusters that are stuck together. Finally, bounding boxes with extended tolerance are extracted from the independent target clusters, and high-confidence positive and negative points are sampled and transformed into hybrid geometric cues.

[0047] (3) Zero-shot large model mask decoding task This task is responsible for fusing and aligning the hybrid geometric cue set of "bounding boxes + positive / negative points" generated by the previous task with the query image features. The purified cue information, which has undergone closed-loop verification and uncertainty decoupling, is then input into the decoder of the Segmentation Model. The entire inference process eliminates any parameter optimization fine-tuning, directly relying on SAM's powerful zero-shot generalization ability and the robust semantic prior provided by this invention to output a high-precision instance segmentation mask for the new underwater category, effectively avoiding the spatial overfitting and base class bias traps of traditional fine-tuning.

[0048] The specific implementation of the method of the present invention is as follows: Figure 2 This is a schematic diagram of the overall network architecture of the underwater new category instance segmentation method (UW-ProSAM) based on prototype-driven SAM of the present invention.

[0049] The first step involves acquiring the query image of the underwater scene and a very small number of support images containing the new target category, and performing a Fast Fourier Transform on both to extract the amplitude spectrum and phase spectrum. Then, according to formula (1), to address the underwater high-frequency scattering noise problem, a preset low-frequency binary mask is used. The low-frequency amplitude spectrum is mixed in a directional manner, and the style-aligned image and features are generated by the inverse transformation of formula (2). This eliminates underwater color shift and turbidity while isolating high-frequency noise, thus preserving a clear geometric structure for subsequent prototype learning.

[0050] The second step involves inputting the style alignment features into the style alignment prototype refinement module. This module first establishes an initial foreground prototype using mask average pooling according to formula (3). Then, to address the issue of underwater target morphological changes, it calculates the forward cosine similarity with the query features according to formula (4) to generate an adaptive prototype. To prevent underwater background artifacts from being introduced by forward mining, it back-projects the adaptive prototype to calculate the attention map according to formulas (5) and (6), and performs consistency verification with the real mask of the supporting image to generate a reliability gating signal. Finally, through residual updates using formulas (7) and (8), it obtains a clean category foreground prototype and a dynamic background prototype corrected by back-verification. Simultaneously, it enables the two prototypes to compete probably during the feature matching stage, explicitly suppressing underwater similar interference objects and outputting a foreground probability map.

[0051] The third step involves inputting the foreground probability map and prototype into the uncertainty-guided instance separation module. First, the network addresses the uneven distribution of underwater noise by performing dynamic threshold filtering sampling on the foreground probability map to generate a discrete candidate point set. Then, according to formula (9), cross-attention feature correction is performed using the clean foreground prototype as the semantic anchor point to compensate for the semantic ambiguity of candidate points caused by underwater scattering. To address the edge ambiguity problem caused by underwater optical scattering, spatial heteroscedasticity uncertainty modeling is performed according to formula (10), jointly predicting coordinate offset and uncertainty variance. During the inference phase, HDBSCAN clustering is performed using the updated coordinates, and the predicted uncertainty is introduced as an exponential weight according to formula (11). Nonlinear acceleration distance penalties are applied to blurred boundary points to forcibly break the false visual connections caused by scattering in dense underwater scenes. Next, the bounding boxes of independent target clusters are extracted and positive and negative points are sampled to generate a semantically clean and mutually isolated set of hybrid geometric cues.

[0052] The fourth step involves inputting the generated hybrid geometric cue set and the query image features into the SAM mask decoder. Leveraging the zero-shot generalization capability of the large model and the accurate prior cue provided by this invention, which has undergone closed-loop verification and uncertainty decoupling, a high-precision instance segmentation mask for the new underwater category is directly output.

[0053] In a specific example Figure 1 A prototype-driven SAM-based underwater novel instance segmentation method addresses the cascading problem of feature misalignment caused by underwater visual degradation, which in turn leads to prototype contamination and instance sticking. Specifically, this method may include: Obtain query images of underwater scenes, as well as a very small number of supporting images containing new target categories and their corresponding ground truth masks; Cross-domain style alignment is performed on the supporting images. By selectively injecting low frequencies in the frequency domain, high-frequency geometric structures are preserved while underwater color cast and turbidity interference are eliminated, providing uncontaminated geometrically preserved features for subsequent prototype learning. Style alignment features are input into the style alignment prototype refinement module to build a closed-loop coupling mechanism of forward mining and reverse consistency verification, dynamically filter out underwater background noise that cannot be distinguished by forward propagation alone, and extract pure category foreground prototypes and adaptively constructed dynamic background prototypes. The foreground prototype, dynamic background prototype, and query image features are input into the uncertainty-guided instance separation module. Using a heteroscedastic uncertainty weighting strategy designed for underwater blurred boundaries, the distance between candidate points is dynamically penalized to force the decoupling of stuck instances, generating a clean set of geometric cueings containing bounding boxes and a mixture of positive and negative points. The hybrid geometric cue set and query image features are input into the decoder of the Segment Anything Model (SAM). A cross-domain prior alignment method without parameter fine-tuning is used to directly output the instance segmentation mask of the new class of target in the query image.

[0054] In some embodiments, cross-domain style alignment processing is performed on the supporting images, specifically including: Fast Fourier Transform is performed on the supporting image and the query image respectively to extract their amplitude spectrum and phase spectrum; Preserving the phase spectrum of the supporting image to maintain the target's geometry, and addressing the issues of underwater high-frequency components containing scattering noise from suspended particles and cross-contamination easily caused by conventional full-band mixing, a preset low-frequency binary mask is used. The process of directional stripping and mixing of amplitude spectra is represented as follows: ; in, and These represent the amplitude spectra of the supporting image and the query image, respectively. A binary mask representing the isolation of low-frequency regions. Represents the mixing coefficient. This represents element-wise multiplication; by generating style-aligned images and corresponding features through inverse fast Fourier transform, the target's geometric structure is protected while eliminating underwater color cast and turbidity interference.

[0055] In some embodiments, the process by which the style alignment prototyping refinement module dynamically extracts a clean category foreground prototype and a dynamic background prototype includes: Initial foreground prototypes are built on supporting features using masked average pooling. : ; Where x and y represent the horizontal and vertical coordinates of the spatial location on the feature map, respectively. This is a support feature for style alignment. To support the true mask of the image, To prevent extremely small constants with a denominator of zero; To address the mismatch between the initial prototype and query features caused by changes in the underwater target's morphology, an initial foreground prototype is calculated. With query features Forward cosine similarity is used to generate a query pseudomask. Based on this pseudomask, an adaptive prototype containing morphological changes of the query image is generated. : ; To prevent forward excavation from introducing background noise in the underwater scattering environment, an adaptive prototype will be used. Back projection to supporting features Calculate the back attention map It is then compared with the true mask of the supporting image to generate a reliability gating signal: ; in, This represents the Sigmoid activation function. Indicates the scaling factor. Indicates the verification threshold. This is a reverse attention map. For real mask; Based on the initial foreground prototype Adaptive Prototype and reliability gating signals The refined category foreground prototype, corrected by reverse verification, is obtained through residual update calculation. .

[0056] In some embodiments, to address the problem that static background prototypes cannot adapt to dynamically changing similar interfering objects in complex underwater environments, a dynamic background prototype construction and competition mechanism may include: By inverted support mask Initialize the dynamic background prototype and utilize the aforementioned reverse gating signal. The dynamic background prototype is updated based on the identified features of rejected queries. : ; in, Indicates the update strength. express Normalization; In the dense feature matching stage, the category foreground prototype is corrected through reverse validation. With a dynamic background prototype that incorporates the interference characteristics of the current underwater scene pass The operation employs probability competition to explicitly suppress visually similar underwater environmental distractions and outputs a foreground probability map.

[0057] In some embodiments, the process of the uncertainty-guided instance separation module generating and clustering candidate points includes: Upsampling and smoothing are performed based on the foreground probability map. To address the uneven distribution of underwater noise, low-confidence noise is filtered out using dynamic thresholding, and discrete candidate point sets are generated through multinomial sampling. Because underwater scattering causes semantic ambiguity in the local features of individual candidate points, for each candidate point, after extracting its feature vector, the refined category foreground prototype, as described above and corrected through reverse verification, is used. As a semantic anchor, prototype-guided feature correction is performed through a cross-attention mechanism, which enhances the discriminative power of candidate point features by utilizing pure global category semantics, and narrows the distance between candidate points belonging to the same instance in the feature space.

[0058] In some embodiments, to address the challenges of blurred instance edges and difficulty in determining the attribution of boundary pixels caused by underwater optical scattering, spatial heteroscedasticity uncertainty modeling is also included, which uses a multilayer perceptron to simultaneously predict the coordinate offset of each candidate point. Spatial heteroscedasticity uncertainty Its regression loss function is defined as: ; in, For the candidate point set, This represents the total number of candidate points; Index for candidate points; For the network prediction of the first The coordinate offset of each candidate point; For the first The true coordinate offset of each candidate point; For the network prediction of the first Spatial heteroscedasticity uncertainty variance of candidate points; Represents the square of the L2 norm; The coefficients for the L1 regularization term are added to address underwater outlier noise.

[0059] Through this loss function, the network autonomously learns to output high variance for blurred edge regions during training. This capability allows for adaptive reduction of regression penalty weights for indistinguishable edges during inference, avoiding positioning errors caused by forcibly performing deterministic regressions on ambiguous boundaries caused by underwater scattering.

[0060] In some embodiments, spatial heteroscedasticity uncertainty generated using the above process As a key criterion for identifying fuzzy boundaries, adhered instances are decoupled into independent sets of geometric cues containing bounding boxes and a mixture of positive and negative points, including: HDBSCAN clustering is performed using the candidate point coordinates corrected for the aforementioned coordinate offset, and the aforementioned prediction uncertainty is introduced into the clustering distance metric. As an index weight, its uncertainty-weighted distance is defined as: ; in Indicate candidate points With candidate points Uncertainty-weighted distance between them; and These represent the first and second coordinates after coordinate offset correction. The and the first Spatial coordinates of the candidate points; This indicates the operation of retrieving the maximum value; This is an exponential scaling factor used to control the strength of the penalty imposed on distance by uncertainty; This metric follows the general multiplicative paradigm of uncertainty-weighted distance, but this invention specifically addresses the boundary blurring and instance visual adhesion characteristics caused by underwater optical scattering, employing an exponential function uncertainty. As a penalty factor, the blurred region is nonlinearly accelerated and amplified, thereby exponentially increasing the distance between high-uncertainty boundary points, forcibly breaking the false visual connections between dense underwater targets, and separating the adhered candidate points into independent clusters; For each independent target cluster, a bounding box containing extended tolerance is extracted, and high-confidence positive points are sampled from the target cluster, while negative points are sampled from neighboring clusters. These are combined to form a semantically clean and mutually isolated geometric cue set that is output to the SAM decoder.

[0061] Compared with the prior art, the present invention has the following beneficial effects: Innovatively, the underwater few-sample instance segmentation problem is transformed into an integrated, fine-tuning-free inference process that combines selective style alignment, closed-loop prototype dynamic extraction, and uncertainty-guided prompt generation. No parameter fine-tuning is required during the inference phase, completely avoiding the spatial overfitting and base class forgetting trap that traditional fine-tuning paradigms are prone to cause under extreme underwater data scarcity.

[0062] The proposed style-aligned prototype refinement module effectively solves the problem of cross-image feature misalignment caused by underwater visual degradation through the coupled design of frequency domain low-frequency selective injection and cyclic consistency verification. It uses the reverse attention gating and real mask verification mechanism to dynamically filter out background artifacts that cannot be detected by forward propagation. The extracted pure prototype has strong anti-interference and anti-morphological change capabilities.

[0063] The proposed uncertainty-guided instance separation algorithm models the heteroscedastic uncertainty of underwater fuzzy boundaries and enables the network to autonomously learn the high variance output capability of fuzzy edges. It uses this uncertainty to apply an exponential dynamic penalty to the clustering distance, successfully breaking the false visual connections caused by scattering in dense underwater scenes. This provides accurate and clean geometric cues for large models and significantly improves the localization accuracy of discovering new underwater targets.

[0064] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0065] Device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0066] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0067] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as... Figure 1 The method described. For example, in some embodiments, Figure 1 The methods described above can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by computing unit 301, the above-described methods can be performed. Figure 1 One or more steps of the method in the document.

[0068] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0069] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0070] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0071] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0072] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0073] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0074] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0075] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for segmenting new underwater categories of instances based on prototype-driven SAM, characterized in that, The method includes: Obtain the query image of the underwater scene, as well as the supporting images containing the new category of targets and their corresponding ground truth masks; The supported images are subjected to cross-domain style alignment processing, and low-frequency selective injection in the frequency domain is used to preserve high-frequency geometric structures while eliminating underwater color shift and turbidity interference. The style-aligned support features are input into the style-aligned prototype refinement module to build a closed-loop coupling mechanism of forward mining and reverse consistency verification, dynamically filter out underwater background noise that cannot be distinguished by forward propagation alone, and extract the pure category foreground prototype and the adaptively constructed dynamic background prototype. The foreground prototype, dynamic background prototype, and query image features are input into the uncertainty-guided instance separation module. Using a heteroscedastic uncertainty weighting strategy pre-designed for underwater blurred boundaries, the distance between candidate points is dynamically penalized to decouple stuck instances, generating a geometric cue set containing bounding boxes and a mixture of positive and negative points. The geometric cue set and the query image features are input into the decoder of the Segmentation All Model (SAM). A cross-domain prior alignment method without parameter adjustment is used to output the instance segmentation mask of the new category target in the query image.

2. The instance segmentation method according to claim 1, characterized in that, The cross-domain style alignment process for the supported images includes: Perform Fourier transforms on the supporting image and the query image respectively, and extract their amplitude spectrum and phase spectrum; The phase spectrum of the supporting image is preserved to maintain the geometric structure of the target. Amplitude spectrum mixing is performed on the low-frequency region isolated by a preset low-frequency binary mask, satisfying the formula: ； in, and These represent the amplitude spectra of the supporting image and the query image, respectively. A binary mask representing the isolation of low-frequency regions. Indicates the mixing coefficient. This represents element-wise multiplication; The mixed amplitude spectrum and the phase spectrum of the supporting image are reconstructed by inverse Fourier transform to generate a style-aligned image and corresponding features.

3. The instance segmentation method according to claim 1, characterized in that, The method further includes: An initial foreground prototype is built on supporting features using masked average pooling; By calculating the forward cosine similarity between the initial foreground prototype and the query features, a query pseudomask is generated. Based on the pseudomask, the query features are then subjected to mask average pooling again to aggregate and generate an adaptive prototype that includes changes in the morphology of the query image. The adaptive prototype is back-projected onto the supporting features, and a back attention map is calculated to measure the degree of matching between the adaptive prototype and the supporting features at each spatial location. The back attention map is then multiplied element-wise with the ground truth mask of the supporting image and summed. Finally, the sum is compared with an activation function and a threshold to generate a reliability gating signal. Based on the initial foreground prototype, the adaptive prototype, and the reliability gating signal, the refined category foreground prototype after reverse verification and correction is obtained through residual update calculation.

4. The instance segmentation method according to claim 3, characterized in that, The construction of the dynamic background prototype includes: A dynamic background prototype with mask initialization is obtained based on the inversion of the real mask. The dynamic background prototype is updated by using the adaptive prototype portion that has not been verified as a reliable foreground identified by the reliability gating signal. The initial dynamic background prototype and the adaptive prototype filtered by the reliability gating signal are weighted and fused according to a preset update intensity, and then normalized so that the dynamic background prototype can adaptively absorb the interference features that are difficult to distinguish from the foreground in the current underwater scene. In the dense feature matching stage, the category foreground prototype, which has been corrected by reverse verification, and the dynamic background prototype, which has absorbed the interference features of the current underwater scene, compete probabilities to calculate the matching probability of each position in the query feature map belonging to the foreground prototype and the background prototype, respectively. The foreground probability map is then output through a normalized exponential function to display visually similar underwater environmental interference objects that have been suppressed.

5. The instance segmentation method according to claim 4, characterized in that, The uncertainty-guided instance separation module utilizes a pre-designed heteroscedastic uncertainty weighting strategy for underwater fuzzy boundaries to dynamically penalize candidate point distances to decouple adhered instances, including: Upsampling and smoothing are performed based on the foreground probability map. To address the uneven distribution of underwater noise, low-confidence noise is filtered out using dynamic thresholding, and discrete candidate point sets are generated through multinomial sampling. For each candidate point, after extracting its feature vector, the refined category foreground prototype, which has been corrected by reverse verification, is used as the semantic anchor point, and the prototype-guided feature correction is performed through the cross attention mechanism. Attention is calculated using candidate point features as query terms and foreground prototypes as key terms. The attention-enhanced features are superimposed on the original candidate point features and normalized. The discriminative power of candidate point features is enhanced by using pure global category semantics, thus narrowing the distance between candidate points belonging to the same instance in the feature space.

6. The instance segmentation method according to claim 5, characterized in that, The method further includes: Based on spatial heteroscedasticity uncertainty modeling, a multilayer perceptron is used to simultaneously predict the coordinate offset and spatial heteroscedasticity uncertainty variance of each candidate point. The regression loss function adopts an uncertainty weighted form. For each candidate point, the coordinate regression error of the point is weighted by the reciprocal of its prediction variance, and a logarithmic regularization term of the variance is added. At the same time, an absolute value regularization term for underwater outlier noise is introduced. The average value of the loss of all candidate points is taken as the total regression loss.

7. The instance segmentation method according to claim 6, characterized in that, The regression loss function satisfies the formula: ； in, For the candidate point set, This represents the total number of candidate points; Index for candidate points; For the network prediction of the first The coordinate offset of each candidate point; For the first The true coordinate offset of each candidate point; For the network prediction of the first Spatial heteroscedasticity uncertainty variance of candidate points; Represents the square of the L2 norm; The coefficients for the L1 regularization term are added to address underwater outlier noise.

8. The instance segmentation method according to claim 6, characterized in that, The method utilizes a pre-designed heteroscedastic uncertainty weighting strategy for underwater fuzzy boundaries to dynamically penalize candidate point distances to decouple adhered instances, generating a geometric cue set containing bounding boxes and a mixture of positive and negative points, including: Calculate the Euclidean distance between two candidate points, and then multiply it by the value of an exponential function with the larger uncertainty variance of the two candidate points as the independent variable as a penalty factor to obtain the uncertainty-weighted distance. The clustering distance metric uses an exponential function to nonlinearly accelerate and amplify the distance between boundary points that meet the preset high uncertainty, thereby exponentially expanding the distance between high uncertainty boundary points, forcibly breaking the false visual connections between dense underwater targets, and separating the sticky candidate points into independent clusters. For each independent target cluster, a bounding box containing extended tolerance is extracted, and high-confidence positive points are sampled from the target cluster, while negative points are sampled from neighboring clusters. These are combined to form a semantically clean and mutually isolated set of geometric hints that is output to the decoder of the segmentation model.

9. The instance segmentation method according to claim 1, characterized in that: Both the query image features and the support image features are extracted by a shared-weight, unlabeled distillation DINO feature extraction network, and the network keeps its parameters frozen during the inference phase.

10. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory is characterized in that it stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method according to any one of claims 1 to 9.

Images