A phased feature enhancement semantic segmentation method for underwater sonar image

Abstract

The application provides a phased feature enhancement semantic segmentation method for underwater sonar images, in the shallow feature extraction stage, a partial convolution residual block PCRB is introduced, through the cascade structure of standard convolution and partial convolution, the redundant background noise response is suppressed while the local spatial context is aggregated; in the deep semantic modeling stage, a multi-scale convolution transformation block MSCFBlock is introduced, through the strip convolution attention and the multi-scale feedforward network, the long-range dependency modeling capability and the multi-scale target expression capability are enhanced; in the decoding output stage, a depth prediction head DPH is introduced, through the cascade convolution structure of context aggregation-detail refinement-pixel classification, the pixel-level classification robustness at the fuzzy boundary is improved. The method can improve the accuracy and boundary integrity of sonar image semantic segmentation in complex underwater environment while maintaining the lightweight inference advantage, and is suitable for underwater robot fine perception, target contour extraction, seabed target size estimation and underwater environment intelligent analysis scenes.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, deep learning, and underwater intelligent sensing, and particularly to a staged feature-enhanced semantic segmentation method, system, and computer-readable storage medium for underwater sonar images. This method can be applied to scenarios such as autonomous underwater robot exploration, pixel-level analysis of sonar images, underwater target contour extraction, seabed target size estimation, underwater environmental monitoring, and marine engineering inspection. Background Technology

[0002] Sonar imaging technology is an important means of underwater environmental perception, capable of acquiring information about underwater targets and their surroundings under complex conditions such as low visibility, no light, and turbid water. Compared with optical imaging, sonar imaging is less dependent on lighting conditions, and therefore is widely used in tasks such as underwater robot detection, seabed topography mapping, underwater target identification, marine engineering inspection, and underwater environmental monitoring.

[0003] In practical underwater operations, relying solely on target recognition or detection methods is often insufficient to meet the demands of sophisticated tasks. For example, scenarios such as underwater target size estimation, target contour extraction, seabed obstacle analysis, and underwater object salvage require precise target shape and pixel-level regional information. Therefore, semantic segmentation of sonar images has become a crucial task in underwater intelligent sensing.

[0004] However, the imaging mechanism of sonar images differs significantly from that of natural optical images. Affected by factors such as multipath reflection during sound wave propagation, environmental reverberation, medium absorption, speckle noise, and equipment noise, sonar images typically suffer from low signal-to-noise ratio, uneven brightness distribution, sparse texture information, blurred boundaries, and significant target scale variations. These problems lead to issues such as broken target boundaries, internal holes, missegmentation, and missed segmentation of small targets when existing semantic segmentation networks are directly applied to sonar images.

[0005] Existing real-time semantic segmentation networks typically employ lightweight convolutional networks to reduce inference overhead. Some methods also enhance the semantic representation capabilities of lightweight networks by introducing teacher branches or semantic alignment modules with global modeling capabilities during the training phase. However, in sonar image semantic segmentation tasks, relying solely on existing lightweight segmentation architectures is insufficient to adequately address issues such as strong speckle noise, sparse texture, target scale variations, and blurred boundaries in sonar images. Therefore, it is necessary to further design a staged feature enhancement structure tailored to the imaging characteristics of sonar images, building upon existing lightweight semantic segmentation frameworks, to improve the accuracy, robustness, and boundary integrity of underwater sonar image semantic segmentation. Summary of the Invention

[0006] To address the problems of strong noise interference, blurred boundaries, significant target scale variations, and insufficient segmentation accuracy in existing sonar image semantic segmentation methods in complex underwater environments, this invention proposes a staged feature enhancement semantic segmentation method for underwater sonar images. Based on existing lightweight semantic segmentation frameworks, this invention designs an improved structure with three stages: shallow noise suppression, deep semantic enhancement, and end-boundary refinement, tailored to the characteristics of sonar images.

[0007] The technical solution of the present invention is as follows: A staged feature-enhanced semantic segmentation method for underwater sonar images includes the following steps: First, the underwater sonar image to be segmented is acquired and preprocessed, including resizing, normalization, and format conversion, to meet the input requirements of the semantic segmentation network.

[0008] Secondly, the preprocessed sonar images are input into a lightweight semantic segmentation network. This lightweight semantic segmentation network can use an existing decoupled training and inference framework as its basic architecture. During the training phase, a teacher branch with global awareness assists the lightweight CNN branch in learning semantic information. During the inference phase, the semantic segmentation of the sonar images is completed solely through the lightweight CNN segmentation branch. This basic architecture provides the network operating environment and semantic segmentation process. The main improvement of this invention lies in its staged feature enhancement structure.

[0009] In the shallow feature extraction stage, this invention introduces a partially convolutional residual block (PCRB). The PCRB employs a structure of cascaded standard convolution and partially convolution. Standard convolution is used to fully aggregate local spatial context information and establish a stable shallow feature benchmark; partially convolution only performs convolutional transformations on a subset of channels, maintaining identity mappings for the remaining channels, thereby reducing computational redundancy and suppressing background scattering noise in insignificant channels. Through this design, the network can preserve the edge and texture information of sonar targets at a high spatial resolution.

[0010] In the deep semantic modeling stage, this invention introduces a multi-scale convolutional transform block (MSCFBlock). The MSCFBlock consists of a convolutional attention module and a multi-scale feedforward network. The convolutional attention module models long-range contextual dependencies in both the horizontal and vertical directions through strip convolutions, enabling the convolutional network to achieve stronger global semantic perception while maintaining computational efficiency. The multi-scale feedforward network extracts multi-scale features through parallel convolutional branches with different kernel scales, thereby improving the model's adaptability to sonar targets at different scales.

[0011] In the decoding output stage, this invention introduces a depth prediction head (DPH). Traditional semantic segmentation networks often use a single-layer 1×1 convolution as the prediction head, which struggles to handle boundary blurring and pixel-level noise interference in sonar images. The DPH described in this invention employs a coarse-to-fine cascaded convolutional structure, including three stages: context aggregation, detail refinement, and pixel classification. First, a larger convolutional kernel aggregates the neighborhood context and smooths local noise. Then, a detail refinement convolution corrects the target boundary structure. Finally, a pixel classification convolution outputs the final semantic segmentation result.

[0012] This invention also provides a staged feature-enhanced semantic segmentation system for underwater sonar images, comprising: The sonar image input module is used to receive underwater sonar images to be segmented; The image preprocessing module is used to resize, normalize, and convert the format of sonar images; A shallow feature enhancement module is used to enhance the local structural features of sonar images and suppress background noise response through partial convolutional residual blocks (PCRBs). The deep semantic modeling module is used to enhance long-range dependency modeling capabilities and multi-scale target representation capabilities through the multi-scale convolutional transform block MSCFBlock. The boundary refinement prediction module is used to perform context aggregation, detail refinement, and pixel classification on decoded features through the deep prediction head (DPH). The results output module is used to output the semantic segmentation results of sonar images.

[0013] In addition, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described phased feature enhancement semantic segmentation method for underwater sonar images.

[0014] Compared with the prior art, the beneficial effects of the present invention are: (1) The present invention introduces a partial convolution mechanism in the shallow stage through the PCRB module, which reduces computational redundancy while maintaining the continuity of feature information, and effectively suppresses speckle noise and background echo interference in sonar images.

[0015] (2) This invention combines strip convolutional attention and multi-scale feedforward structure in the deep stage through the MSCFBlock module to enhance the network’s ability to model long-range dependencies and multi-scale targets, which helps to alleviate the problems of target boundary fragmentation and missing segmentation of small targets.

[0016] (3) The present invention improves the robustness of pixel-level classification and enhances the continuity of segmentation boundaries and the integrity of target regions by adopting a cascade strategy of context aggregation, detail refinement and pixel classification in the prediction stage through the DPH module.

[0017] (4) This invention forms a complete feature reconstruction path of "shallow noise suppression - deep semantic enhancement - terminal boundary refinement", which enables the network to better adapt to the characteristics of low signal-to-noise ratio, sparse texture and blurred boundaries of sonar images. Experimental results show that the combination of the three can effectively improve the semantic segmentation accuracy and boundary integrity.

[0018] (5) This invention can be implemented on the basis of existing lightweight semantic segmentation frameworks, maintaining low inference overhead, and is suitable for deployment in underwater robots, unmanned underwater vehicles, shore-based sonar processing systems and edge computing platforms. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall structure of the phased feature enhancement semantic segmentation method for underwater sonar images according to the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of a portion of the convolutional residual block PCRB in this invention.

[0021] Figure 3 This is a schematic diagram of the MSCFBlock structure of the multi-scale convolutional transform block in this invention.

[0022] Figure 4 This is a schematic diagram of the depth prediction head (DPH) structure in this invention. Detailed Implementation

[0023] The application of this invention in semantic segmentation of underwater sonar images will be described in detail below with reference to the accompanying drawings in the embodiments of the invention.

[0024] This invention proposes a staged feature-enhanced semantic segmentation method for underwater sonar images. The method first acquires the underwater sonar image to be segmented and performs preprocessing operations such as resizing, normalization, and format conversion to meet the input requirements of the semantic segmentation network. Subsequently, the preprocessed sonar image is input into a lightweight semantic segmentation network.

[0025] like Figure 1 As shown, in one embodiment, the lightweight semantic segmentation network can adopt a decoupled training and inference semantic segmentation framework as its basic architecture. During the training phase, this architecture can include a lightweight CNN student branch, a Transformer teacher branch, and a semantic interaction alignment module to assist the lightweight CNN branch in learning richer global semantic information. During the inference phase, the teacher branch and semantic interaction alignment module can be separated, retaining only the lightweight CNN segmentation branch to perform fast inference. It should be noted that the above-mentioned decoupled training-inference framework and semantic alignment structure can serve as the basic architecture for implementing this invention. The improvement of this invention focuses on the staged feature enhancement design of the shallow PCRB, deep MSCFBlock, and terminal DPH layers on this basic architecture.

[0026] This invention establishes a shallow feature enhancement module, a deep semantic modeling module, a decoding module, and a boundary refinement prediction module within the lightweight CNN segmentation branch. Specifically, the shallow feature enhancement module employs PCRB, the deep semantic modeling module employs MSCFBlock, and the boundary refinement prediction module employs DPH. These three modules respectively address the noise suppression, semantic enhancement, and boundary refinement requirements in sonar image semantic segmentation.

[0027] In the shallow feature extraction stage, such as Figure 2 As shown, this invention introduces Partial Convolutional Residual Blocks (PCRBs). The PCRB module adopts a structure of cascaded standard convolution and partial convolution. First, the input features are aggregated locally using standard convolution to obtain a relatively stable shallow feature representation. Then, partial convolution is performed on some channels, while maintaining an identity mapping for the remaining channels. This approach reduces redundant channel computation, lowers memory access overhead, and suppresses noise responses in insignificant regions in low signal-to-noise ratio sonar image scenarios. Finally, residual connections preserve effective detail information in the input features and stabilize gradient propagation. Through this design, PCRBs can enhance edge, texture, and local structural information in shallow features, providing cleaner feature input for subsequent deep semantic modeling.

[0028] In the deep semantic modeling stage, such as Figure 3 As shown, this invention introduces a multi-scale convolutional transform block, MSCFBlock. MSCFBlock consists of a convolutional attention module and a multi-scale feedforward network. The convolutional attention module uses strip convolution to model contextual information in the horizontal and vertical directions to enhance the network's ability to capture long-range dependencies. Compared with the Transformer structure based on a self-attention mechanism, this convolutional attention module can effectively expand the receptive field and improve global semantic representation while maintaining convolutional computation efficiency. The multi-scale feedforward network includes multiple parallel convolutional branches with different kernel scales, used to extract channel interaction features, local detail features, and regional contextual features respectively. The outputs of each branch are concatenated and fused by convolution to generate enhanced deep semantic features. This design can improve the network's adaptability to sonar targets of different scales and alleviate problems such as incomplete segmentation of small targets and blurred boundaries of large targets.

[0029] During the decoding stage, this invention employs a multi-scale contextual decoding structure to progressively upsample deep semantic features and combines this with shallow detail information to restore spatial resolution. The decoded features are then further input into the Depth Prediction Head (DPH). Figure 4As shown, DPH employs a cascaded structure of context aggregation, detail refinement, and pixel classification. First, context aggregation is performed on the input features using a larger convolutional kernel to expand the effective receptive field and smooth local speckle noise. Second, detail refinement is performed on the features using a smaller convolutional kernel to correct boundary aliasing caused by upsampling and noise interference. Finally, pixel classification convolution outputs the class probability map of each pixel and generates the final semantic segmentation result.

[0030] In one specific implementation, PCRB can be deployed in the shallow encoding stage of the network to achieve high-resolution feature denoising with minimal computational cost; MSCFBlock can be deployed in the deep encoding stage of the network to enhance global semantic modeling and multi-scale target adaptation capabilities; and DPH can be deployed at the end of the decoder to improve boundary continuity and pixel-level classification stability. In one experimental embodiment, this design achieves noise suppression through PCRB in the shallow stage, enhances long-range dependencies and multi-scale expression through MSCFBlock in the deep stage, and performs coarse-to-fine boundary refinement through DPH at the end of the decoder.

[0031] In this embodiment, the present invention can be trained and validated using a publicly available sonar image semantic segmentation dataset. Target categories can include typical underwater targets such as shipwrecks, aircraft wreckage, and human bodies. Experimental results show that PCRB, MSCFBlock, and DPH can all improve the performance of sonar image semantic segmentation. The complete model achieves high mIoU and Dice coefficients on the SCTD dataset. Specifically, the complete model achieves an mIoU of 86.00% and a Dice coefficient of 92.28%, validating the effectiveness and practicality of this phased enhancement design in complex underwater environments.

[0032] The complete method flow of this invention is as described above. All content not described in detail in this specification is prior art known to those skilled in the art. Modifications to the embodiments of this invention, equivalent substitutions of technical features, etc., without departing from the spirit and principles of this invention, should be included within the scope of protection of this invention.

Claims

1. A staged feature-enhanced semantic segmentation method for underwater sonar images, characterized in that, Includes the following steps: (1) Obtain the underwater sonar image to be segmented, and perform size adjustment, normalization and preprocessing on the sonar image to obtain sonar image data that meets the network input requirements; (2) Input the preprocessed sonar image into the lightweight semantic segmentation network. The lightweight semantic segmentation network adopts a training and inference decoupling framework. In the inference stage, feature extraction, feature decoding and semantic segmentation output are completed through the lightweight CNN segmentation branch. (3) In the shallow feature extraction stage of the lightweight CNN segmentation branch, a partial convolutional residual block PCRB is introduced to selectively enhance and suppress the local texture, edge structure and background noise response in the sonar image; The partially convolutional residual block (PCRB) includes standard convolution, partial convolution, and residual connection processes. Standard convolution is used to aggregate local spatial context information, and partial convolution is used to perform convolution transformation on some channels and maintain identity mapping on the remaining channels to reduce computational redundancy and suppress insignificant noise responses in sonar images. (4) In the deep semantic modeling stage of the lightweight CNN segmentation branch, a multi-scale convolutional transform block MSCFBlock is introduced to enhance the long-range dependency modeling capability and multi-scale target expression capability through convolutional attention and multi-scale feedforward structure. The multi-scale convolutional transform block MSCFBlock includes a convolutional attention module and a multi-scale feedforward network. The convolutional attention module models long-range contextual dependencies in the horizontal and vertical directions through strip convolution, and the multi-scale feedforward network extracts multi-scale semantic features through parallel convolutional branches with different convolutional kernel scales. (5) In the decoding output stage, a depth prediction head (DPH) is introduced to perform context aggregation, detail refinement and pixel classification on the decoded features in sequence, and output the semantic segmentation results of the sonar image. The depth prediction head (DPH) includes context aggregation convolution, detail refinement convolution, and pixel classification convolution. The context aggregation convolution is used to expand the effective receptive field and smooth local noise, the detail refinement convolution is used to correct the details of the target boundary, and the pixel classification convolution is used to generate the final class probability map.

2. The staged feature enhancement semantic segmentation method for underwater sonar images according to claim 1, characterized in that, The partially convolutional residual block PCRB, multi-scale convolutional transform block MSCFBlock, and depth prediction head DPH are deployed in the shallow feature extraction stage, the deep semantic modeling stage, and the decoding output stage, respectively, forming a phased feature enhancement path of shallow noise suppression, deep semantic enhancement, and end boundary refinement.

3. A system for implementing the staged feature enhancement semantic segmentation method for underwater sonar images as described in claims 1-2, characterized in that, include: The sonar image input module is used to receive underwater sonar images to be segmented; The image preprocessing module is used to resize, normalize, and convert the format of sonar images; A shallow feature enhancement module is used to enhance the local structural features of sonar images and suppress background noise response through partial convolutional residual blocks (PCRBs). The deep semantic modeling module is used to enhance long-range dependency modeling capabilities and multi-scale target representation capabilities through the multi-scale convolutional transform block MSCFBlock. The boundary refinement prediction module is used to perform context aggregation, detail refinement, and pixel classification on decoded features through the deep prediction head (DPH). The results output module is used to output the semantic segmentation results of sonar images.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the phased feature-enhanced semantic segmentation method for underwater sonar images as described in any one of claims 1 to 2.

Images