Underwater sonar image segmentation method based on physical prior multi-scale fusion

Abstract

The application relates to the technical field of image processing and computer vision, and particularly provides an underwater sonar image segmentation method based on physical prior multi-scale fusion. The method comprises the following steps: processing an original sonar image through a physical prior generation module PPGM, applying an acoustic propagation constraint to generate an enhanced image; inputting the enhanced image into a frequency adaptive module FAM, extracting high-frequency subband and low-frequency subband information, and obtaining enhanced features; extracting initial semantic features of the original sonar image through different encoding layers Transform in a stem encoder SAM, fusing the initial semantic features with the enhanced image, and obtaining optimized features; and inputting the enhanced features and the optimized features into a multi-source multi-scale fusion decoder respectively for multi-scale reconstruction and mask prediction, so that a segmentation result is obtained. The method realizes high-precision segmentation of a complex underwater scene.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to an underwater sonar image segmentation method based on physical prior multi-scale fusion. Background Technology

[0002] Underwater sonar image segmentation technology has important applications in marine exploration, deep-sea exploration, and underwater navigation. However, underwater sonar images suffer from low resolution, high noise, and blurred image boundaries in practical applications, which poses a significant challenge to sonar image segmentation. Due to the complexity of the underwater environment, traditional image segmentation methods (such as models based on convolutional neural networks (CNNs) have limited performance in handling these complex conditions, especially under strong noise interference and blurred image details, where segmentation accuracy and robustness are greatly constrained.

[0003] In recent years, deep learning technology, especially convolutional neural networks (CNNs) and their variants, has made significant progress. Through multi-layered feature learning and deep architectures, CNNs can extract spatial features from complex images. However, these methods still have the following shortcomings: First, facing complex underwater environments, early research has focused on CNNs and their variants. Traditional CNNs cannot fully model the spatial dependencies in underwater sonar images, resulting in weak local feature capture capabilities. Multichannel fused convolutional networks (MCF-CNNs) improve feature extraction efficiency by introducing deep separable residual modules; multi-branch structures are designed by borrowing from the FCN architecture to improve segmentation accuracy; semantic segmentation structures are optimized from the perspective of model compression. These methods have improved sonar image segmentation to varying degrees, but still struggle to maintain stability under strong noise and complex echo conditions. Second, the identification of targets with blurred boundaries and low contrast is challenging. Third, the lack of modeling that considers physical priors makes existing methods insufficiently robust under strong noise or shadow interference conditions.

[0004] With the rise of Transformer applications in visual tasks, its powerful global modeling capabilities have offered new possibilities for sonar image segmentation. Some studies have attempted to combine CNNs with Transformers to balance local details and global semantics. However, these methods exhibit significant limitations when dealing with sonar-specific problems such as signal attenuation and boundary blurring, especially in situations of data scarcity or cross-domain transfer, where their generalization ability is insufficient. Meanwhile, as a pioneering foundational segmentation model, the Segment Anything Model (SAM) demonstrates excellent zero-shot capability and has been widely applied to specific domains. Existing methods have alleviated the shortcomings of SAM in complex tasks to some extent, but they often fail to fully consider the physical characteristics of sonar imaging, such as energy attenuation and shadow effects, thus still exhibiting problems of boundary blurring and incomplete masking in environments with strong interference.

[0005] Overall, existing technologies have made significant progress in improving the accuracy of sonar image segmentation, but there are still three limitations: First, the CNN and Transformer fusion model is not robust enough under strong noise and weak boundary conditions; second, the existing SAM fine-tuning strategy ignores the physical mechanism of sonar imaging, resulting in limited cross-modal adaptability; and third, although the adapter method can reduce training costs, a trade-off still needs to be made between parameter scale and efficiency. Summary of the Invention

[0006] In view of this, the present invention provides an underwater sonar image segmentation method based on physical prior multi-scale fusion to achieve high-precision segmentation of complex underwater scenes.

[0007] In a first aspect, the present invention provides an underwater sonar image segmentation method based on physical prior multi-scale fusion, the method comprising:

[0008] Step 1: Process the original sonar image using the Physical Prior Generation (PPGM) module and apply acoustic propagation constraints to generate an enhanced image;

[0009] Step 2: Input the enhanced image into the frequency adaptive module FAM to extract high-frequency and low-frequency sub-band information and obtain the enhanced features;

[0010] Step 3: Extract initial semantic features from the original sonar image using different coding layers (Transformers) in the backbone encoder SAM, and then fuse them with the enhanced image to obtain optimized features.

[0011] Step 4: Input the enhanced features and optimized features into the multi-source multi-scale fusion decoder respectively. Multi-scale reconstruction and mask prediction are performed to obtain the segmentation results.

[0012] Optionally, step 1 includes:

[0013] The original sonar image I(X) is input into the physical prior generation module PPGM. The depth estimation network and the attenuation estimation network are used to estimate the spatially varying depth map d(X) and the environmental attenuation map A(X), respectively. Here, d(X) is used to describe the relative distance of the sound wave propagation path, and A(X) is used to model the imaging noise caused by background reflection and environmental interference.

[0014] A transport mapping term T(X) is introduced to simulate energy attenuation during underwater propagation, and it is defined as:

[0015] ;

[0016] in, This indicates a learnable or empirically set attenuation coefficient.

[0017] Based on the physical mechanism of sonar imaging, an image physical model is constructed from the original sonar image. A physically consistent enhanced image p is reconstructed; the modeling process draws on a variant of the atmospheric scattering model in underwater imaging and considers the combined effects of signal attenuation, background interference, and imaging noise. The reconstruction equation is as follows:

[0018] .

[0019] Optionally, step 2 includes:

[0020] By explicitly modeling the frequency domain noise distribution, the signal-to-noise components of different frequency bands are separated; the frequency adaptive module FAM uses the intermediate features of layers 6 and 12 of the backbone encoder SAM. The enhanced image is used as input, where i takes values ​​of 6 and 12. First, a Fast Fourier Transform (FFT) is performed on the enhanced image to obtain its frequency domain representation. :

[0021] ;

[0022] Subsequently, the Mask Generation Block (MGB) is introduced, from Adaptive extraction of high-frequency mask and low-frequency mask To guide the frequency domain modulation process, its mask extraction process is defined as follows:

[0023] ;

[0024] ;

[0025] in, , is the normalized frequency boundary parameter used to determine the range of high-frequency and low-frequency sub-bands; k represents the spectral resolution, set to k=128;

[0026] After filtering the high-frequency and low-frequency regions separately using masks, the reconstructed spatial domain features are obtained through inverse Fourier transform, and their frequency domain features are defined as follows:

[0027] ;

[0028] in, , Indicates high-frequency spectral characteristics. Indicates low-frequency spectral characteristics; This indicates element-wise multiplication;

[0029] Frequency domain features are fused using a cross-attention mechanism. and intermediate features To generate high-frequency and low-frequency representations The process is defined as follows:

[0030] ;

[0031] Where Q, K, and V correspond to query, keyword, and value prediction, respectively; Indicates the learnable scaling factor; ;

[0032] To achieve effective information exchange between different frequency bands, the Frequency Adaptive Module (FAM) employs a bidirectional cross-attention mechanism, utilizing low-frequency information to enhance the boundary sensitivity of high-frequency channels. This mechanism is defined as follows:

[0033] ;

[0034] ;

[0035] in, This represents the Sigmoid activation function; GAP and GMP represent the global average pooling operation and the global max pooling operation, respectively. and These represent high-frequency and low-frequency characteristics, respectively. This indicates that the convolution kernel is 7×7; MLP represents a multilayer perceptron. This indicates element-wise multiplication;

[0036] pass High-frequency features of convolution modulation and low frequency characteristics Aggregation is performed, and a cross-attention mechanism is used to obtain the final representation of the feature map:

[0037] .

[0038] Optionally, step 3 includes:

[0039] A shadow-aware adapter (SAA) is introduced, consisting of two key components: a bottleneck transformation layer and a shadow filtering module (SFM). The bottleneck transformation layer employs a downsampling, non-linear, upsampling projection structure to achieve feature refinement with minimal parameter overhead. Given initial semantic features from the Transformer encoding layer and the enhanced image,

[0040] First, the enhanced image is fed into a linear layer, and a dynamic channel attention mask M is calculated using the shadow filtering module SFM, which is defined as:

[0041] ;

[0042] in, ; This represents the input to SFM. ; Represents the learnable weight matrix;

[0043] The generated channel attention mask M is used to perform weighted suppression on the feature map, resulting in the shadow-free features. :

[0044] ;

[0045] in, This indicates element-wise multiplication;

[0046] Will Divided into multiple sub-blocks along the channel dimension to satisfy ; B represents the batch size, N represents the spatial token count, and D represents the feature dimension;

[0047] Each sub-block undergoes feature modeling in an independent branch, defined as follows:

[0048] ;

[0049] in, , The linear transformation weights are for the current feedforward neural network MLP branch; Indicates the activation function;

[0050] The outputs of all branches are combined into a complete representation through a concatenation operation:

[0051] ;

[0052] Wherein, Concat means fusion;

[0053] Finally, at the bottleneck transformation layer, the original sonar image is compressed and reconstructed by sequentially passing through the layer normalization and multi-head self-attention mechanism in the Transformer and the outputs of the shadow filtering module SFM, to obtain the output of SAA. Its definition is:

[0054] ;

[0055] in, and These represent the weights for upsampling and downsampling in the bottleneck conversion layer, respectively. .

[0056] Optionally, step 4 includes:

[0057] Multi-source multi-scale fusion decoder Enhanced and optimized features are fused to suppress irrelevant regions, and the output of the backbone encoder SAM is refined into fine-grained segmentation maps; feature maps of different scales are extracted from layers 3, 6, 9, and 12 of the Transformer coding layer. :

[0058] ;

[0059] Where F represents the initial semantic features, and the value range of l is... , respectively representing the 3rd, 6th, 9th, and 12th layers of the Transformer coding layer; Indicates the scaling factor;

[0060] In order to enhance To enhance spatial representation and context modeling capabilities, a scale-aware convolutional attention mechanism module, SACAM, is designed. SACAM employs a cascaded structure, sequentially integrating channel attention, spatial attention, and multi-scale separable convolution, defined as follows:

[0061] ;

[0062] Wherein, CAB represents channel attention block; SAB represents spatial attention block; and MSCB represents multi-scale convolutional block.

[0063] To improve the fusion effect of multi-source features, an improved feature fusion attention mechanism, RFFA, is introduced to fuse feature maps from multiple sources. In RFFA, the feature maps are first... Each feature map x from SACAM is independently convolved with a 3x3 group, then normalized and element-wise multiplied. The resulting features are activated by the ReLU activation function and then convolved with a 1x1 convolution, defined as follows:

[0064] ;

[0065] in, and Represent the relationship between feature map x and feature map x, respectively. 3×3 convolution is used; BN represents batch normalization; C represents the Sigmoid activation function; C represents... .

[0066] Optionally, this includes: First, the Channel Attention Block (CAB) readjusts the weights of each channel to emphasize task-relevant features while suppressing irrelevant features, defined as:

[0067] ;

[0068] in, and Represents adaptive max pooling and adaptive average pooling, respectively; C represents R represents the ReLU activation function; This represents the Sigmoid activation function; This indicates element-wise multiplication;

[0069] Secondly, the Spatial Attention Block (SAB) focuses on key regions in underwater sonar images to enhance the capture of local contextual information, and is defined as follows:

[0070] ;

[0071] in, and represents channel-based max pooling and average pooling of the input feature map x, respectively; LKC represents a convolutional layer with a 7×7 kernel, used to enhance the local contextual relationships between features;

[0072] Finally, the Multi-Scale Convolutional Block (MSCB) employs multi-scale depthwise separable convolution to extract scale-sensitive features and enhance multi-resolution context modeling. Its definition is as follows:

[0073] ;

[0074] Where PC represents a point convolutional layer; MSDC represents a multi-scale depthwise separable convolutional block; , This indicates parallel depthwise separable convolutions with different kernel sizes KS, where KS takes the value of... ;CS indicates channel shuffling operation; Represents the ReLU6 activation function, which restricts the input values ​​to a certain range. Within the range.

[0075] In a second aspect, embodiments of the present invention provide a computer-readable storage medium comprising a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to execute the underwater sonar image segmentation method based on physical prior multi-scale fusion in the first aspect or any possible implementation thereof.

[0076] Thirdly, embodiments of the present invention provide an electronic device, including: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the underwater sonar image segmentation method based on physical prior multi-scale fusion in the first aspect or any possible implementation of the first aspect.

[0077] The technical solution provided by this invention includes the following steps: processing the original sonar image through a Physical Prior Generation (PPGM) module to apply acoustic propagation constraints to generate an enhanced image; inputting the enhanced image into a Frequency Adaptive (FAM) module to extract high-frequency and low-frequency sub-band information to obtain enhanced features; extracting initial semantic features from the original sonar image through different coding layers (Transformers) in the backbone encoder (SAM) and fusing them with the enhanced image to obtain optimized features; and inputting the enhanced and optimized features into a multi-source, multi-scale fusion decoder, respectively. Multi-scale reconstruction and mask prediction are performed to obtain segmentation results. This method achieves high-precision segmentation of complex underwater scenes. Attached Figure Description

[0078] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a framework diagram of the underwater sonar image segmentation method based on physical prior multi-scale fusion provided in an embodiment of the present invention;

[0080] Figure 2 A framework diagram of the physical prior generation module provided in an embodiment of the present invention;

[0081] Figure 3 A framework diagram of the frequency adaptive module provided in an embodiment of the present invention;

[0082] Figure 4A framework diagram of a trunk encoder provided in an embodiment of the present invention;

[0083] Figure 5 A framework diagram of the scale-aware convolutional attention mechanism module provided in an embodiment of the present invention;

[0084] Figure 6 A framework diagram of the improved feature fusion attention mechanism provided in the embodiments of the present invention;

[0085] Figure 7 These are comparison images of visual segmentation effects in typical scenarios provided in embodiments of the present invention.

[0086] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0088] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0089] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0090] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0091] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0092] Figure 1 This is a framework diagram of the underwater sonar image segmentation method based on physical prior multi-scale fusion provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0093] In this invention, the physical prior model effectively compensates for image degradation caused by factors such as energy attenuation, background reflection, and environmental interference during sound wave propagation, thereby enhancing image quality. Meanwhile, frequency domain feature modeling, through explicit modeling of image frequency components, effectively distinguishes high-frequency details from low-frequency structures, further enhancing target boundary recognition capabilities. Against this backdrop, this invention proposes an underwater sonar image segmentation system framework (SonPF-SAM) based on physical priors and frequency domain characteristics, aiming to enhance the robustness and structural awareness of underwater sonar image segmentation while supporting joint modeling with the physical frequency branch. This framework integrates physical priors, frequency features, and semantic information, introducing a multi-dimensional information interaction mechanism in the decoding stage, significantly improving the model's ability to handle challenging conditions such as boundary ambiguity, shadow occlusion, and noise interference.

[0094] This invention designs three key modules: First, the PhyFre branch, including the Physics Prior Generation Module (PPGM) and the Frequency Adaptive Module (FAM). The PPGM estimates depth and attenuation maps and reconstructs enhanced images to achieve acoustic physics prior modeling and compensation. The FAM combines high- and low-frequency mask generation with a bidirectional frequency domain attention mechanism to achieve frequency decoupling modeling of sonar image structural information and boundary details. Second, the Shadow-Aware Adapter (SAA) introduces the Shadow Filtering Module (SFM) to adaptively suppress occlusion interference, enhancing the model's ability to perceive target edges. Third, a multi-source, multi-scale fusion decoder... These modules aim to overcome the shortcomings of the original SAM decoder in terms of spatial awareness and semantic accuracy under complex tasks. They are designed to enhance the feature representation capabilities of sonar images, especially in complex underwater environments, significantly improving the accuracy and robustness of segmentation.

[0095] Step 1: Process the original sonar image using the Physical Prior Generation (PPGM) module and apply acoustic propagation constraints to generate an enhanced image.

[0096] In embodiments of the present invention, such as Figure 2 As shown, step 1 includes:

[0097] The original sonar image I(X) is input into the Physical Prior Generation (PPGM) module. The Depth Estimation Network and Attenuation Estimation Network are used to estimate the spatially varying depth map d(X) and environmental attenuation map A(X), respectively. Here, d(X) is used to describe the relative distance of the sound wave propagation path, and A(X) is used to model the imaging noise caused by background reflection and environmental interference.

[0098] A transport mapping term T(X) is introduced to simulate energy attenuation during underwater propagation, and it is defined as:

[0099] ;

[0100] in, This indicates a learnable or empirically set attenuation coefficient.

[0101] Based on the physical mechanism of sonar imaging, an image physics model is constructed from the original sonar image. A physically consistent enhanced image p is reconstructed; the modeling process draws on a variant of the atmospheric scattering model in underwater imaging and considers the combined effects of signal attenuation, background interference, and imaging noise. The reconstruction equation is as follows:

[0102] .

[0103] Step 2: Input the enhanced image into the frequency adaptive module FAM to extract high-frequency and low-frequency sub-band information and obtain the enhanced features.

[0104] In this embodiment of the invention, FAM extracts high-frequency subband and low-frequency subband information to assist the backbone encoder in obtaining edge and texture details.

[0105] In embodiments of the present invention, such as Figure 3 As shown, step 2 includes:

[0106] Due to the complex marine environment and the imaging mechanism of sonar equipment itself, underwater sonar images often suffer from a significant low signal-to-noise ratio (SNR), severely interfering with downstream semantic understanding and segmentation tasks. By explicitly modeling the frequency domain noise distribution and separating the SNR components in different frequency bands, the feature representation capability and robustness of sonar images are improved. The frequency adaptive module (FAM) utilizes intermediate features from layers 6 and 12 of the backbone encoder SAM. The enhanced image is used as input, where i takes values ​​of 6 and 12. First, a Fast Fourier Transform (FFT) is performed on the enhanced image to obtain its frequency domain representation. :

[0107] ;

[0108] Subsequently, a Mask Generation Block (MGB) was introduced, from Adaptive extraction of high-frequency mask and low-frequency mask To guide the frequency domain modulation process, its mask extraction process is defined as follows:

[0109] ;

[0110] ;

[0111] in, , is the normalized frequency boundary parameter used to determine the range of high-frequency and low-frequency sub-bands; k represents the spectral resolution, set to k=128;

[0112] After filtering the high-frequency and low-frequency regions separately using masks, the reconstructed spatial domain features are obtained through inverse Fourier transform, and their frequency domain features are defined as follows:

[0113] ;

[0114] in, , Indicates high-frequency spectral characteristics. Indicates low-frequency spectral characteristics; This indicates element-wise multiplication;

[0115] Frequency domain features are fused using a cross-attention mechanism. and intermediate features To generate high-frequency and low-frequency representations The process is defined as follows:

[0116] ;

[0117] Where Q, K, and V correspond to query, keyword, and value prediction, respectively; Indicates the learnable scaling factor; ;

[0118] To achieve effective information exchange between different frequency bands, the Frequency Adaptive Module (FAM) employs a bidirectional cross-attention (BCA) mechanism, utilizing low-frequency information to enhance the boundary sensitivity of high-frequency channels. Its definition is:

[0119] ;

[0120] ;

[0121] in, This represents the Sigmoid activation function; GAP and GMP represent the global average pooling operation and the global max pooling operation, respectively. and These represent high-frequency and low-frequency characteristics, respectively. This indicates that the convolution kernel is 7×7; MLP represents a multilayer perceptron. This indicates element-wise multiplication;

[0122] pass High-frequency features of convolution modulation and low frequency characteristics Aggregation is performed, and a cross-attention (CA) mechanism is used to obtain the final representation of the feature map:

[0123] .

[0124] Step 3: Extract initial semantic features from the original sonar image using different coding layers (Transformers) in the backbone encoder SAM, and then fuse them with the enhanced image to obtain optimized features.

[0125] In embodiments of the present invention, such as Figure 4 As shown, step 3 includes:

[0126] Underwater sonar images are often severely affected by shadow interference due to multipath propagation and occlusion, leading to misclassification or loss of key target features in segmentation tasks. To address this challenge, a Shadow-Aware Adapter (SAA) is introduced. This lightweight module, integrated into the backbone encoder SAM, is specifically designed to suppress shadow interference while enhancing structure perception in sonar images. It consists of two key components: a bottleneck transformation layer and a shadow filter module (SFM). The bottleneck transformation layer employs a downsampling, nonlinear, upsampling projection structure to achieve feature refinement with minimal parameter overhead. Given initial semantic features from the Transformer encoding layer and the enhanced image, the enhanced image is first input into a linear layer, and a dynamic channel attention mask M is calculated through the shadow filter module SFM, defined as:

[0127] ;

[0128] in, ; This represents the input to SFM. When i=1, it corresponds to the input of SAA in the first coding layer Transformer. ,at this time This comes from the PhyFre branch, enabling early physics guidance; the rest... The values ​​all come from the output of the previous SAA. ; Represents the learnable weight matrix;

[0129] In this embodiment of the invention, the SAA generates a channel attention mask to guide the backbone encoder in making structural adjustments to the shadow region.

[0130] The generated channel attention mask M is used to perform weighted suppression on the feature map, resulting in the shadow-free features. :

[0131] ;

[0132] in, This indicates element-wise multiplication; this operation explicitly weakens the response of shadowed regions, thereby enhancing the salience of structurally stable regions.

[0133] To further improve feature recovery capabilities and region modeling accuracy, Divided into multiple sub-blocks along the channel dimension to satisfy ; B represents the batch size, N represents the spatial token count, and D represents the feature dimension;

[0134] Each sub-block undergoes feature modeling in an independent branch, defined as follows:

[0135] ;

[0136] in, , The linear transformation weights are for the current feedforward neural network MLP branch; Indicates the activation function;

[0137] The outputs of all branches are combined into a complete representation through a concatenation operation:

[0138] ;

[0139] Wherein, Concat means fusion;

[0140] Finally, at the bottleneck transformation layer, the initial semantic features F extracted from the original sonar image through LayerNorm and Multi-Head Attention in the Transformer, along with the outputs s of the shadow filtering module SFM, are compressed and reconstructed to obtain the output of SAA. Its definition is:

[0141] ;

[0142] in, and These represent the weights for upsampling and downsampling in the bottleneck conversion layer, respectively. .

[0143] Step 4: Input the enhanced features and optimized features into the multi-source multi-scale fusion decoder respectively. Multi-scale reconstruction and mask prediction are performed to obtain the segmentation results.

[0144] In this embodiment of the invention, step 4 includes:

[0145] The simplified decoder design in the original SAM architecture imposes limitations when handling complex segmentation tasks. To address this challenge, this invention proposes a multi-source, multi-scale fusion decoder (…). This significantly improves segmentation accuracy by fusing features from multiple sources and scales. It gradually incorporates features from the PhyFre branch and the trunk encoder SAM.

[0146] Multi-source multi-scale fusion decoder Enhanced and optimized features are fused to suppress irrelevant regions, and the output of the backbone encoder SAM is refined into fine-grained segmentation maps; feature maps of different scales are extracted from layers 3, 6, 9, and 12 of the Transformer coding layer. :

[0147] ;

[0148] Where F represents the initial semantic features, and the value range of l is... , respectively representing the 3rd, 6th, 9th, and 12th layers of the Transformer coding layer; Indicates the scaling factor; such as Figure 1 T3, T6, T9, T 12 These represent layers 3, 6, 9, and 12 of the Transformer coding layer.

[0149] In embodiments of the present invention, such as Figure 5 As shown, in order to enhance To enhance spatial representation and context modeling capabilities, a scale-aware convolutional attention mechanism module, SACAM, is designed. SACAM employs a cascaded structure, sequentially integrating channel attention, spatial attention, and multi-scale separable convolution, defined as follows:

[0150] ;

[0151] Wherein, CAB represents channel attention block; SAB represents spatial attention block; and MSCB represents multi-scale convolutional block.

[0152] In embodiments of the present invention, such as Figure 6 As shown, to improve the fusion effect of multi-source features, an improved Feature Fusion Attention (RFFA) mechanism is introduced to fuse feature maps from multiple sources. In RFFA, the feature maps are first processed... Each feature map x from SACAM is independently convolved with a 3x3 group, then normalized and element-wise multiplied. The resulting features are activated by the ReLU activation function and then convolved with a 1x1 convolution, defined as follows:

[0153] ;

[0154] in, and Represent the relationship between feature map x and feature map x, respectively. 3×3 convolution is used; BN represents batch normalization; C represents the Sigmoid activation function; C represents... .

[0155] In this embodiment of the invention, the method includes: First, the Channel Attention Block (CAB) readjusts the weights of each channel to emphasize task-relevant characteristics while suppressing irrelevant characteristics, defined as follows:

[0156] ;

[0157] in, and Represents adaptive max pooling and adaptive average pooling, respectively; C represents R represents the ReLU activation function; This represents the Sigmoid activation function; This indicates element-wise multiplication;

[0158] Secondly, the Spatial Attention Block (SAB) focuses on key regions in underwater sonar images to enhance the capture of local contextual information, and is defined as follows:

[0159] ;

[0160] in, and represents channel-based max pooling and average pooling of the input feature map x, respectively; LKC represents a convolutional layer with a 7×7 kernel, used to enhance the local contextual relationships between features;

[0161] Finally, the Multi-Scale Convolutional Block (MSCB) employs multi-scale depthwise separable convolution to extract scale-sensitive features and enhance multi-resolution context modeling. Its definition is as follows:

[0162] ;

[0163] Where PC represents a point convolutional layer; MSDC represents a multi-scale depthwise separable convolutional block; , This indicates parallel depthwise separable convolutions with different kernel sizes KS, where KS takes the value of... CS (Channel Shuffle) represents the channel shuffling operation. Represents the ReLU6 activation function, which restricts the input values ​​to a certain range. Within the interval. Through the joint operation of these sub-modules, SACAM effectively refines the feature representation, improves sensitivity to target details in underwater sonar images, and thus improves segmentation performance.

[0164] To further improve segmentation performance, especially when handling complex scenes, this invention introduces a multi-source, multi-scale fusion decoder. This new decoder addresses the shortcomings of the original SAM decoder in terms of spatial awareness and semantic accuracy under complex tasks. It adopts a multi-scale modeling approach, extracting multi-level feature representations from layers 3, 6, 9, and 12 of the Transformer encoding layer and fusing them layer-by-layer with the output of the PhyFre branch. Through Scale-Aware Convolutional Attention (SACAM), it combines channel attention, spatial attention, and multi-scale depthwise separable convolutional operations to achieve adaptive interaction and reconstruction of cross-level and cross-modal information. This structure not only enhances the segmentation head's ability to model global semantics and local structures but also significantly improves the recognition accuracy for low-contrast, small targets, and complex background regions.

[0165] The multi-source, multi-scale fusion decoder extracts and integrates features from physical priors, frequency adaptation modules, and the backbone encoder at multiple levels through multi-level, multi-scale feature fusion. This fusion strategy not only improves the model's spatial resolution but also enhances its ability to capture target details and boundaries, thereby effectively improving the accuracy and structural recovery capabilities of image segmentation. By progressively integrating feature information from different scales, It achieves semantic consistency across scales, further improving the performance of the segmentation model in complex underwater environments.

[0166] To achieve progressive refinement of segmentation results, the multi-source, multi-scale fusion decoder is designed to operate in multiple stages. At each stage, a prediction mask is generated based on intermediate feature representations, enabling the model to progressively improve spatial accuracy and semantic alignment at different resolutions. This staged improvement strategy allows SonPF-SAM to effectively integrate contextual information from different sources and resolutions. The model generates high-precision and fine-grained segmentation results, and is particularly adept at handling complex underwater sonar scenarios with targets exhibiting diverse shapes, sizes, and acoustic textures.

[0167] This invention uses the Dice coefficient as the primary evaluation metric and systematically compares SonPF-SAM with several mainstream segmentation models on the Marine Debris dataset, including SAM, MedSAM, SAMed, SonarSAM, SAM-Adapter, and Medical-Adapter. Table 1 summarizes the performance comparison results of different methods on various categories. Figure 7 The images show a comparison of visual segmentation performance in typical scenarios. The results demonstrate that SonPF-SAM achieves state-of-the-art performance across almost all categories, particularly in segmenting targets with blurred boundaries and low contrast, where it significantly outperforms existing methods. Even in complex environments with severe noise, shadow interference, and structural defects, our method maintains clear edge contours and complete target structures, exhibiting stronger robustness and generalization ability.

[0168] Table 1 Performance Comparison Results

[0169] ;

[0170] The method of this invention can simultaneously integrate physical priors, frequency domain features and structural modeling, aiming to improve the accuracy and robustness of image segmentation in complex underwater environments. It can be widely used in fields such as marine exploration, deep-sea exploration, and underwater navigation, and has significant performance advantages, especially in environments with low signal-to-noise ratio, shadow interference and structural ambiguity.

[0171] The technical solution provided by this invention includes the following steps: processing the original sonar image through a Physical Prior Generation (PPGM) module to apply acoustic propagation constraints to generate an enhanced image; inputting the enhanced image into a Frequency Adaptive (FAM) module to extract high-frequency and low-frequency sub-band information to obtain enhanced features; extracting initial semantic features from the original sonar image through different coding layers (Transformers) in the backbone encoder (SAM) and fusing them with the enhanced image to obtain optimized features; and inputting the enhanced and optimized features into a multi-source, multi-scale fusion decoder, respectively. Multi-scale reconstruction and mask prediction are performed to obtain segmentation results. This method achieves high-precision segmentation of complex underwater scenes.

[0172] The various steps in the embodiments of the present invention can be performed by an electronic device. This electronic device includes, but is not limited to, tablet computers, portable PCs, and desktop computers.

[0173] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is running, it controls the electronic device containing the computer-readable storage medium to execute the above-described embodiment of the underwater sonar image segmentation method based on physical prior multi-scale fusion.

[0174] Figure 8 A schematic diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 8 As shown, the electronic device 21 includes a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, it implements the underwater sonar image segmentation method based on physical prior multi-scale fusion in the embodiment. To avoid repetition, it will not be described in detail here.

[0175] Electronic device 21 includes, but is not limited to, processor 211 and memory 212. Those skilled in the art will understand that... Figure 8This is merely an example of electronic device 21 and does not constitute a limitation on electronic device 21. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0176] The processor 211 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0177] The memory 212 can be an internal storage unit of the electronic device 21, such as a hard disk or RAM of the electronic device 21. The memory 212 can also be an external storage device of the electronic device 21, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or FlashCard equipped on the electronic device 21. Furthermore, the memory 212 can include both internal and external storage units of the electronic device 21. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 can also be used to temporarily store data that has been output or will be output.

[0178] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0179] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for underwater sonar image segmentation based on physical prior multi-scale fusion, characterized in that, The method comprises: Step 1, processing the original sonar image by a physical prior generation module PPGM, applying acoustic propagation constraints to generate an enhanced image; Step 2, inputting the enhanced image into a frequency adaptive module FAM, extracting high-frequency sub-band and low-frequency sub-band information, and obtaining enhanced features; Step 3, extracting initial semantic features of the original sonar image through different encoding layers of the main encoder SAM, and fusing with the enhanced image to obtain optimized features; Step 4, input the enhanced features and the optimized features into a multi-source multi-scale fusion decoder respectively Multi-scale reconstruction and mask prediction are performed in the multi-source multi-scale fusion decoder to obtain a segmentation result. The step 3 comprises: A shadow perception adapter SAA is introduced, which is composed of a bottleneck conversion layer and a shadow filtering module SFM; the bottleneck conversion layer adopts a projection structure of down-sampling, nonlinearity and up-sampling to realize feature refinement with minimal parameter overhead; given the initial semantic features from the encoding layer Transformer and the enhanced image, first, the enhanced image is input into a linear layer, and a dynamic channel attention mask M is calculated through the shadow filtering module SFM, which is defined as: ； wherein, ; denotes the input of the SFM, ; denotes a learnable weight matrix; The generated channel attention mask M is used to weight and suppress the feature map to obtain the de-shadowed feature : ； wherein denotes element-wise multiplication; Will be divided into multiple sub-blocks along the channel dimension to meet ; B represents the batch size, N represents the spatial token count, and D represents the feature dimension. Each sub-block models features in an independent branch, which is defined as: ； wherein, , is a linear transformation weight of the current feed-forward neural network MLP branch; denotes an activation function; The outputs of all branches are combined into a complete representation through a concatenation operation: ； Where, Concat represents fusion; Finally, the output s of the initial semantic feature F extracted by the layer normalization and multi-head self-attention mechanism in the Transformer and the shadow filtering module SFM is compressed and reconstructed in the bottleneck transition layer to obtain the output of the SAA defined as: ； wherein, and denote the weights of the bottleneck transition layer upsampling and downsampling, respectively; .

2. The method of claim 1, wherein, The step 1 comprises: The original sonar image I(X) is input into the physical prior generation module PPGM, and a depth estimation network and an attenuation estimation network are used to estimate a spatially varying depth map d(X) and an environmental attenuation map A(X), respectively, wherein d(X) is used to describe the relative distance of the sound wave propagation path, and A(X) is used to model the imaging noise caused by background reflection and environmental interference; A transmission mapping term T(X) is introduced to simulate the energy attenuation in the underwater propagation process, which is defined as: ； wherein, denotes a learnable or empirically set decay coefficient; According to the physical mechanism of sonar imaging, a physical model of image is constructed from the original sonar image A physically consistent enhanced image p is reconstructed; the modeling process draws on the variant of atmospheric scattering model in underwater imaging, and considers the comprehensive influence of signal attenuation, background interference and imaging noise, and the reconstruction equation is: 。 3. The method of claim 2, wherein, The step 2 comprises: By explicitly modeling the noise distribution in the frequency domain, the signal and noise components in different frequency bands are separated; the frequency adaptive module FAM takes the intermediate features of the 6th layer and the 12th layer of the backbone encoder SAM and the enhanced image as input, where i takes the values 6, 12; first, the enhanced image is subjected to a fast Fourier transform FFT to obtain its frequency domain representation : ； Subsequently, a mask generation block MGB is introduced, from adaptively extracting high frequency masks and low frequency masks to guide the frequency domain modulation process, whose mask extraction process is defined as: ； ； wherein, , a normalized frequency boundary parameter for determining the high frequency subband and the low frequency subband range; k represents the frequency spectral resolution, set as k = 128; After filtering the high-frequency region and the low-frequency region using the mask respectively, the spatial domain reconstruction features are obtained through inverse Fourier transform, and the frequency domain features are defined as: ； wherein, , represents a high frequency spectral feature, represents a low frequency spectral feature; represents an element-wise multiplication; Fusing frequency domain features using cross-attention mechanism and intermediate features to generate high and low frequency representations whose process is defined as: ； where Q, K and V correspond to query, key and value prediction, respectively; denotes a learnable scale factor; ; In order to realize effective information interaction between different frequency bands, the frequency adaptive module FAM adopts a bidirectional cross-attention mechanism, which uses low-frequency information to enhance the boundary sensitivity of high-frequency channels, which is defined as: ； ； wherein, denotes a Sigmoid activation function; GAP and GMP denote global average pooling and max pooling, respectively; and denote high-frequency features and low-frequency features, respectively; denotes a convolution kernel of 7x7; MLP denotes a multi-layer perceptron; denotes element-wise multiplication; By convolving the modulated high-frequency features and low-frequency features and employing a cross-attention mechanism to obtain a final representation of the feature map: 。 4. The method of claim 1, wherein, The step 4 comprises: Multi-source multi-scale fusion decoder Fusing the enhanced features and the optimized features to suppress irrelevant regions and refine the output of the backbone encoder SAM into a fine-grained segmentation map; extracting feature maps of different scales from the 3rd, 6th, 9th and 12th layers of the encoding layer Transformer : ； Wherein, F represents an initial semantic feature, the value range of l is , respectively represent the 3rd, 6th, 9th and 12th layers of the encoding layer Transformer; Denotes a scaling factor; To enhance the spatial representation and context modeling capability, a scale-aware convolutional attention mechanism module (SACAM) is designed. SACAM adopts a cascaded structure to sequentially integrate channel attention, spatial attention, and multi-scale separable convolution, which is defined as follows: ； Where, CAB represents a channel attention block; SAB represents a spatial attention block; MSCB represents a multi-scale convolution block; In order to improve the fusion effect of multi-source features, an improved feature fusion attention mechanism RFFA is introduced to fuse multi-source feature maps. In RFFA, first, 3x3 group convolution independent operations are performed on the feature maps x and x from SACAM respectively, then normalization processing and element multiplication are performed, the generated features are activated through the ReLU activation function, and 1x1 convolution is performed, which is defined as: and the feature map x from SACAM respectively, then normalization processing and element multiplication are performed, the generated features are activated through the ReLU activation function, and 1x1 convolution is performed, which is defined as: ； wherein, and respectively represent the feature map x and the feature map with 3x3 convolution; BN represents batch normalization; represents a Sigmoid activation function; C represents .

5. The method of claim 4, wherein, It comprises: Firstly, the channel attention block CAB adjusts the weight of each channel to emphasize the characteristics related to the task while suppressing irrelevant characteristics, which is defined as: ； wherein, and respectively represent adaptive max pooling and adaptive average pooling; C represents ; R represents a RELU activation function; represents a Sigmoid activation function; represents element-wise multiplication; Secondly, the spatial attention block SAB focuses on the key areas in the underwater sonar image to enhance the capture of local context information, which is defined as: ； wherein, and respectively represent the channel-wise max-pooling and average-pooling of the input feature map x; LKC represents a convolutional layer with a 7x7 kernel for enhancing the local contextual relationship between features. Finally, the multi-scale convolution block MSCB adopts multi-scale depth separable convolution to extract scale-sensitive features and strengthen multi-resolution context modeling, which is defined as: ； where PC denotes a point convolution layer; MSDC denotes a multi-scale depth separable convolution block; , denotes a depth separable convolution in parallel for different kernel sizes KS, KS takes values ; CS denotes a channel shuffle operation; represents a ReLU6 activation function, i.e., limiting the input value in the interval .

6. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein when the program runs, the device where the computer readable storage medium is located executes the underwater sonar image segmentation method based on physical prior multi-scale fusion according to any one of claims 1-5.

7. An electronic device, comprising: It comprises: One or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions, which when executed by the device, cause the device to perform the method of underwater sonar image segmentation based on physical prior multi-scale fusion according to any one of claims 1 to 5.

Images