Submarine target detection method and system based on double-frequency side-scan sonar image
By constructing a dual-frequency side-scan sonar image target detection model and combining high- and low-frequency imaging feature complementary modeling, the accuracy and adaptability issues of single-frequency sonar images in seabed target detection were solved, achieving higher accuracy target detection results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG PROVINCIAL MARINE DEV PLANNING RES CENT
- Filing Date
- 2026-01-15
- Publication Date
- 2026-04-28
AI Technical Summary
Existing technologies for detecting seabed targets using side-scan sonar images at a single frequency are difficult to achieve both high-precision detail recognition and wide-area coverage simultaneously. Their effectiveness is limited, especially in complex scenarios, and they are unable to effectively detect small targets and are affected by imaging characteristics.
A target detection method based on dual-frequency side-scan sonar images is adopted. By introducing a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module, a target detection model for dual-frequency side-scan sonar images is constructed. Combined with complementary modeling of high and low frequency imaging features, effective features of high and low frequency images are extracted and fused.
It achieves higher precision seabed target detection, overcomes the limitations of single-frequency range imaging, and improves the detection capability for small targets and adaptability to complex scenarios.
Smart Images
Figure CN121937852A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image target detection technology, and in particular to a method and system for detecting seabed targets based on dual-frequency side-scan sonar images. Background Technology
[0002] Dual-Frequency Side-Scan Sonar (DFSS) is a device that simultaneously emits two different frequencies of sound waves, combining the advantages of high-frequency high resolution and low-frequency wide detection range to achieve precise imaging and comprehensive detection of seabed topography and targets. In marine engineering, its core value lies in balancing "detail recognition" and "wide coverage," overcoming the detection limitations of single-frequency sonar in complex scenarios. It is widely used in all stages of the process, including surveying, construction monitoring, and operation and maintenance. Low-frequency sound waves have strong penetrating power and long propagation distance (a single survey line can cover hundreds of meters), enabling rapid topographic surveys of large sea areas, identifying macroscopic geomorphological features such as seabed uplifts, depressions, sand waves, and trenches, and delineating "no-go zones" for engineering site selection (such as steep slopes and areas with large obstacles). High-frequency sound waves have short wavelengths and high resolution (capable of identifying centimeter-level targets), clearly imaging minute seabed undulations (such as reef outcrops and shell layer distribution) and shallow sedimentary structures (such as sand-mud interfaces and shallowly buried paleochannels). For example, in the survey of submarine pipeline routes, high-frequency detection can identify small obstacles such as piles of gravel and coral reefs in the route area (those with a diameter > 0.5m can be detected), avoiding the pipeline from being punctured by sharp objects during laying; at the same time, it can determine the type of seabed sediment (sand, mud, gravel) to provide a basis for the selection of pipeline anti-corrosion layer.
[0003] In related technologies, target detection algorithms based on side-scan sonar images mainly fall into two categories. Traditional target detection algorithms rely on manually designed features combined with traditional machine learning algorithms to detect and classify specific targets in images. They obtain potential target regions through methods such as sliding windows, feature matching, and candidate region generation, then calculate the effective features (texture, shape, color, etc.) of these potential regions and feed them into a classifier to obtain the final output. Commonly used classifiers include Support Vector Machines (SVM), AdaBoost, and Random Forests. Limited by the generalization ability of manually designed features, traditional target detection algorithms perform poorly in complex scenes (strong noise, weak texture, different lighting intensities) and cannot adaptively learn complex features. With the continuous development of deep learning technology, more and more target detection algorithms based on deep neural networks have emerged in the field of image target detection. Among them, two-stage target detection algorithms, such as Faster-RCNN, separate candidate region selection and target classification into two stages, achieving higher accuracy in target detection. One-stage target detection algorithms, such as YOLO and SSD, strike a balance between target detection accuracy and algorithm complexity, achieving effective target detection with relatively low computational cost. In recent years, with the widespread application of Transformers in natural language processing tasks, researchers have also introduced them into image target detection tasks, with frameworks like ViT and DETR becoming prevalent. While these methods have achieved good results in conventional image detection tasks, their effectiveness is often limited when processing dual-frequency side-scan sonar images. Limited by the imaging characteristics of high- and low-frequency images, relying solely on a single-frequency sonar image often fails to achieve high target detection accuracy. On the one hand, low-frequency imaging results have limited resolution, making it difficult to detect some small targets, but they contain large-scale information such as terrain structure and target contours; on the other hand, high-frequency images have high resolution and can capture detailed information such as small-scale targets on the seabed, but they are often susceptible to the influence of complex seabed structures, resulting in numerous imaging artifacts. Traditional detection algorithms typically rely on single-frequency image input, making them easily affected by imaging characteristics during detection. Summary of the Invention
[0004] To address the aforementioned technical problems, the present invention aims to provide a method and system for detecting seabed targets based on dual-frequency side-scan sonar images, which can improve the target detection results of sonar images by complementary modeling of high and low frequency imaging features.
[0005] The first technical solution adopted in this invention is: a method for detecting seabed targets based on dual-frequency side-scan sonar images, comprising the following steps:
[0006] High- and low-frequency seabed image data are obtained by imaging the target area on the seabed using dual-frequency side-scan sonar.
[0007] A dual-frequency side-scan sonar image target detection model is constructed by introducing a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module.
[0008] The target detection model based on dual-frequency side-scan sonar images is used to detect targets in high- and low-frequency seabed image data, and the seabed target detection results are obtained.
[0009] Furthermore, the dual-frequency side-scan sonar image target detection model specifically includes a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module. The first output terminal of the single-modal modeling module is connected to the input terminal of the cross-modal modeling module, the second output terminal of the single-modal modeling module is connected to the input terminal of the shadow-highlight matching modeling module, the output terminal of the cross-modal modeling module is connected to the first input terminal of the single-modal modeling module, and the output terminal of the shadow-highlight matching modeling module is connected to the second input terminal of the single-modal modeling module, wherein:
[0010] The single-modal modeling module includes a spectral enhancement and prior information extraction module, a CNN backbone network, convolutional layers, and a Transformer encoding / decoding module.
[0011] Both the cross-modal modeling module and the shadow highlight matching modeling module include a multi-head self-attention module and a multi-head cross-attention module.
[0012] Furthermore, the step of performing target detection on high and low frequency seabed image data based on the dual-frequency side-scan sonar image target detection model to obtain seabed target detection results specifically includes:
[0013] High and low frequency seabed image data are input into a dual-frequency side-scan sonar image target detection model;
[0014] The cross-modal modeling module based on the dual-frequency side-scan sonar image target detection model enhances the high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images output by the CNN backbone network to obtain enhanced high- and low-frequency seabed image feature information.
[0015] The shadow highlight matching modeling module based on the dual-frequency side-scan sonar image target detection model enhances the high-dimensional abstract features of the high-frequency subtracted low-frequency seabed image and the high-dimensional abstract features of the low-frequency subtracted high-frequency seabed image output by the CNN backbone network to obtain shadow target region matching features.
[0016] A single-modal modeling module based on a dual-frequency side-scan sonar image target detection model performs target detection on high- and low-frequency seabed image data, enhanced high- and low-frequency seabed image feature information, and shadow target region matching features to obtain seabed target detection results.
[0017] Furthermore, the cross-modal modeling module based on the dual-frequency side-scan sonar image target detection model enhances the high-dimensional abstract features of the high-frequency seabed image output by the CNN backbone network and the high-dimensional abstract features of the low-frequency seabed image to obtain enhanced high- and low-frequency seabed image feature information. This step specifically includes:
[0018] The high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images are input into the cross-modal modeling module of the dual-frequency side-scan sonar image target detection model;
[0019] Based on the multi-head self-attention module of the cross-modal modeling module, complementary information extraction processing is performed on the high-dimensional abstract features of high-frequency seabed images and the high-dimensional abstract features of low-frequency seabed images to obtain complementary information between high-frequency and low-frequency seabed images.
[0020] Based on the multi-head cross-attention module of the cross-modal modeling module, the complementary information of high and low frequency seabed images is adapted across modal information to obtain optimized high and low frequency seabed image feature information.
[0021] Feature fusion and self-attention calculation are performed on the optimized high- and low-frequency seabed image feature information to obtain enhanced high- and low-frequency seabed image feature information.
[0022] Furthermore, the single-modal modeling module based on the dual-frequency side-scan sonar image target detection model performs target detection on high- and low-frequency seabed image data, enhanced high- and low-frequency seabed image feature information, and shadow target region matching features to obtain seabed target detection results. This step specifically includes:
[0023] High and low frequency seabed image data are input into the single-modal modeling module of the dual-frequency side-scan sonar image target detection model;
[0024] The spectral enhancement and prior information extraction module based on the single-modal modeling module calculates high- and low-frequency seabed image data and outputs high-frequency seabed images, low-frequency seabed images, high-frequency subtracted low-frequency seabed images, and low-frequency subtracted high-frequency seabed images.
[0025] Based on the CNN backbone network of the single-modal modeling module, feature extraction processing is performed on high-frequency seabed images, low-frequency seabed images, high-frequency subtracted from low-frequency seabed images, and low-frequency subtracted from high-frequency seabed images, respectively, to obtain high-dimensional abstract features of high-frequency seabed images, high-dimensional abstract features of low-frequency seabed images, high-dimensional abstract features of high-frequency subtracted from low-frequency seabed images, and high-dimensional abstract features of low-frequency subtracted from high-frequency seabed images.
[0026] The high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images are spliced and fused to obtain the high-dimensional abstract features of spliced high- and low-frequency seabed images.
[0027] Based on the convolutional layer of the single-modal modeling module, the high-dimensional abstract features of the stitched high- and low-frequency seabed images, the high-dimensional abstract features of the high-frequency subtracted from the low-frequency seabed images, and the high-dimensional abstract features of the low-frequency subtracted from the high-frequency seabed images are compressed to obtain high- and low-frequency seabed two-dimensional feature maps, high-frequency subtracted from the low-frequency seabed two-dimensional feature maps, and low-frequency subtracted from the high-frequency seabed two-dimensional feature maps.
[0028] The enhanced high and low frequency seabed image feature information, shadow target region matching features, high and low frequency seabed two-dimensional feature maps, high frequency reduced low frequency seabed two-dimensional feature maps, and low frequency reduced high frequency seabed two-dimensional feature maps are stitched together and position coding information is introduced to obtain the stitched high and low frequency seabed image feature information.
[0029] The Transformer encoding / decoding module, based on the single-modal modeling module, encodes and decodes the feature information of the stitched high- and low-frequency seabed images to obtain the seabed target detection results.
[0030] Furthermore, the Transformer encoding / decoding module based on the single-modal modeling module performs encoding / decoding processing on the stitched high- and low-frequency seabed image feature information to obtain the seabed target detection result. This step specifically includes:
[0031] The stitched high and low frequency seabed image feature information is input into the Transformer encoding and decoding module of the single-modal modeling module;
[0032] The encoder module based on the Transformer encoder-decoder module encodes the feature information of the stitched high and low frequency seabed images to obtain the updated high and low frequency seabed image feature information and the target query vector.
[0033] The decoder module, based on the Transformer encoding and decoding module, performs multi-head attention calculation on the updated high and low frequency seabed image feature information and target query vector to obtain seabed target detection results.
[0034] Furthermore, the seabed target detection results specifically include target category labels and the normalized coordinates of the target location box in the image coordinate system.
[0035] The second technical solution adopted in this invention is: a seabed target detection system based on dual-frequency side-scan sonar images, comprising:
[0036] The first module is used to image the seabed target area using dual-frequency side-scan sonar to obtain high- and low-frequency seabed image data.
[0037] The second module is used to introduce a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module to construct a dual-frequency side-scan sonar image target detection model.
[0038] The third module is used to perform target detection on high and low frequency seabed image data based on the dual-frequency side-scan sonar image target detection model, and obtain seabed target detection results.
[0039] The beneficial effects of the method and system of this invention are as follows: This invention images the target area on the seabed using dual-frequency side-scan sonar to obtain high- and low-frequency seabed image data; further, by introducing a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module, a dual-frequency side-scan sonar image target detection model is constructed. Target information and background information in the high- and low-frequency images can complement each other, providing sufficient clues for target detection and improving the limitations of single-frequency range imaging; finally, target detection is performed on the high- and low-frequency seabed image data based on the dual-frequency side-scan sonar image target detection model to obtain seabed target detection results. The imaging characteristics of low- and high-frequency images are mined in a data-driven manner, and effective features are extracted from the data of both low- and high-frequency modalities. Through a designed feature interaction mode, the effective information in the high- and low-frequency images is integrated, thereby achieving higher-precision target detection. Attached Figure Description
[0040] Figure 1 This is a flowchart of the steps of a seabed target detection method based on dual-frequency side-scan sonar images according to the present invention;
[0041] Figure 2 This is a structural block diagram of a seabed target detection system based on dual-frequency side-scan sonar images according to the present invention;
[0042] Figure 3 This is a schematic diagram of the dual-frequency side-scan sonar imaging results provided in a specific embodiment of the present invention;
[0043] Figure 4 This is a schematic diagram of a dual-frequency side-scan sonar image target detection model provided in a specific embodiment of the present invention;
[0044] Figure 5 This is a schematic diagram of the shadow and target area highlighted after subtracting high- and low-frequency images according to a specific embodiment of the present invention;
[0045] Figure 6 This is a schematic diagram of the single-modal modeling module provided in a specific embodiment of the present invention;
[0046] Figure 7This is a schematic diagram of the cross-modal modeling module provided in a specific embodiment of the present invention. Detailed Implementation
[0047] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
[0048] First, it should be noted that when mounted on a ship-borne towed fish or underwater robot platform, the dual-frequency side-scan sonar emits sound waves and receives seabed echoes. After transducer processing, high-frequency and low-frequency grayscale image pairs are generated. High-frequency images often contain more detailed information about seabed targets, but are usually noisier; low-frequency images typically provide clearer imaging of terrain contours, but are less effective at detecting small targets. Therefore, relying solely on imaging results from a single frequency is often insufficient for achieving high-precision target detection. This invention provides a target detection method that integrates low-frequency and high-frequency imaging results from a side-scan sonar. Based on a data-driven approach, it mines the imaging characteristics of low-frequency and high-frequency images, extracts effective features from both low-frequency and high-frequency modal data, and integrates the effective information from both high- and low-frequency images through a designed feature interaction mode, thereby achieving higher-precision target detection. The target detection algorithm relies on the grayscale images output by the dual-frequency side-scan sonar. These grayscale images serve as input to the target detection method, ultimately outputting the target category and the target's location within the image.
[0049] Reference Figure 1 This invention provides a method for detecting seabed targets based on dual-frequency side-scan sonar images, the method comprising the following steps:
[0050] S100: Imaging of the seabed target area using dual-frequency side-scan sonar to obtain high- and low-frequency seabed image data;
[0051] In this embodiment, as Figure 3 As shown, in the deep learning-based side-scan sonar image target detection framework, the model input is typically the image output from the side-scan sonar. Specifically, for dual-frequency side-scan sonar, the high- and low-frequency images output can simultaneously serve as input features for target detection. Target information is extracted through effective modeling, ultimately outputting the target category and its location in the image. In this case, the target information and background information in the high- and low-frequency images can complement each other, providing sufficient clues for target detection and overcoming the limitations of single-frequency range imaging. To improve target recognition in side-scan sonar images, pseudo-color images are typically used to visualize grayscale images and serve as input to the target detection algorithm. As can be seen from the figure, low-frequency imaging results more clearly display large-scale information such as terrain features, while high-frequency images show richer target details.
[0052] Therefore, this embodiment uses a dual-frequency side-scan sonar to image a specific area on the seabed, and obtains single-channel grayscale image data after transducer processing. Each frame of the dual-frequency side-scan sonar image is a low-frequency to high-frequency pair (in the same physical space, high and low frequency images are an organic combination and naturally have alignment characteristics), that is, it simultaneously acquires the sound intensity map obtained from the reflection of high-frequency sound waves and low-frequency sound waves. Considering the defects of directly using single-mode data, high- and low-frequency image pairs are used as input to the detection algorithm.
[0053] S200 introduces a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module to construct a dual-frequency side-scan sonar image target detection model;
[0054] Specifically, the dual-frequency side-scan sonar image target detection model includes a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module. The first output of the single-modal modeling module is connected to the input of the cross-modal modeling module, and the second output of the single-modal modeling module is connected to the input of the shadow-highlight matching modeling module. The output of the cross-modal modeling module is connected to the first input of the single-modal modeling module, and the output of the shadow-highlight matching modeling module is connected to the second input of the single-modal modeling module. The single-modal modeling module includes a spectral enhancement and prior information extraction module, a CNN backbone network, convolutional layers, and a Transformer encoding / decoding module. Both the cross-modal modeling module and the shadow-highlight matching modeling module include a multi-head self-attention module and a multi-head cross-attention module.
[0055] More specifically, such as Figure 4 As shown, the unimodal modeling, cross-modal modeling, and shadow-highlight matching modeling modules share the Transformer's decoder module. The encoder-decoder design references the DETR (DEtection Transformer) encoding and decoding logic. The decoder's input includes the encoder output (unimodal modeling features), the cross-modal modeling module output (cross-modal modeling features), the shadow-highlight matching modeling module output features (the correspondence between shadow and highlight regions), and a learnable query vector. During decoding, the unimodal modeling features are fused with the shadow-highlight matching modeling module output features and the cross-modal modeling features, respectively, as key-value pairs for cross-attention in the decoder to complete the calculation of the attention matrix and obtain the final output.
[0056] The S300 uses a dual-frequency side-scan sonar image target detection model to perform target detection on high and low frequency seabed image data, and obtains seabed target detection results.
[0057] S310. Input high- and low-frequency seabed image data into the dual-frequency side-scan sonar image target detection model;
[0058] S320, a cross-modal modeling module based on a dual-frequency side-scan sonar image target detection model, enhances the high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images output by the CNN backbone network to obtain enhanced high- and low-frequency seabed image feature information.
[0059] Specifically, the high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images are input into the cross-modal modeling module of the dual-frequency side-scan sonar image target detection model. Based on the multi-head self-attention module of the cross-modal modeling module, complementary information extraction processing is performed on the high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images to obtain complementary information of high- and low-frequency seabed images. Based on the multi-head cross-attention module of the cross-modal modeling module, cross-modal information adaptation is performed on the complementary information of high- and low-frequency seabed images to obtain optimized high- and low-frequency seabed image feature information. Feature fusion and self-attention calculation are performed on the optimized high- and low-frequency seabed image feature information to obtain enhanced high- and low-frequency seabed image feature information.
[0060] In this embodiment, this module is responsible for mining and modeling complementary information in high-frequency and low-frequency images. The output features obtained after processing the high-frequency and low-frequency images by the backbone network are sent to the cross-modal modeling module, which then passes through a self-attention module and a cross-attention module. Here, the cross-attention module is mainly used to capture complementary information in high-frequency and low-frequency images, outputting effective cross-modal features to assist in subsequent target detection tasks.
[0061] S330, a shadow highlight matching modeling module based on a dual-frequency side-scan sonar image target detection model, enhances the high-dimensional abstract features of the high-frequency subtracted low-frequency seabed image and the high-dimensional abstract features of the low-frequency subtracted high-frequency seabed image output by the CNN backbone network to obtain shadow target region matching features.
[0062] In this embodiment, in sonar images, areas that sound waves cannot reach often appear around the target region, manifested as shadow areas in the sonar image. In the target detection task based on side-scan sonar images, we introduce a target-shadow region matching modeling module, which models the matching relationship between shadow and highlight regions in the image through an attention mechanism, further providing usable and effective clues for target detection.
[0063] S340, a single-modal modeling module based on a dual-frequency side-scan sonar image target detection model, performs target detection on high and low frequency seabed image data, enhanced high and low frequency seabed image feature information and shadow target region matching features to obtain seabed target detection results.
[0064] Specifically, high- and low-frequency seabed image data are input into the single-modal modeling module of the dual-frequency side-scan sonar image target detection model. Based on the spectral enhancement and prior information extraction module of the single-modal modeling module, calculations are performed on the high- and low-frequency seabed image data to output high-frequency seabed images, low-frequency seabed images, high-frequency minus low-frequency seabed images, and low-frequency minus high-frequency seabed images. Based on the CNN backbone network of the single-modal modeling module, feature extraction processing is performed on the high-frequency seabed images, low-frequency seabed images, high-frequency minus low-frequency seabed images, and low-frequency minus high-frequency seabed images, respectively, to obtain high-dimensional abstract features of the high-frequency seabed images, the low-frequency seabed images, the high-frequency minus low-frequency seabed images, and the low-frequency minus high-frequency seabed images. The high-dimensional abstract features of the high-frequency seabed images and the high-dimensional abstract features of the low-frequency seabed images are then spliced and fused to obtain the spliced high-frequency seabed images. The high-dimensional abstract features of high- and low-frequency seabed images are extracted. Based on the convolutional layer of the single-modal modeling module, the high-dimensional abstract features of the stitched high- and low-frequency seabed images, the high-frequency minus low-frequency seabed images, and the low-frequency minus high-frequency seabed images are compressed to obtain high- and low-frequency two-dimensional feature maps, high-frequency minus low-frequency seabed two-dimensional feature maps, and low-frequency minus high-frequency seabed two-dimensional feature maps. The enhanced high- and low-frequency seabed image feature information, shadow target region matching features, high- and low-frequency seabed two-dimensional feature maps, high-frequency minus low-frequency seabed two-dimensional feature maps, and low-frequency minus high-frequency seabed two-dimensional feature maps are stitched together and positional encoding information is introduced to obtain the stitched high- and low-frequency seabed image feature information. Based on the Transformer encoding and decoding module of the single-modal modeling module, the stitched high- and low-frequency seabed image feature information is encoded and decoded to obtain the seabed target detection results.
[0065] In this embodiment, this module is responsible for mining effective information within a single modality. After the high- and low-frequency image pairs pass through the spectral enhancement and prior information extraction module, the input information will become four parts: low-frequency image, high-frequency image, low-frequency minus high-frequency image, and high-frequency minus low-frequency image. Here, the subtraction of data from two modalities corresponds to different types of prior information, such as... Figure 5As shown, the images obtained by subtracting high frequencies from low frequencies and vice versa are presented. The image obtained by subtracting low frequencies from high frequencies highlights the shadow areas in the image, while the image obtained by subtracting high frequencies from low frequencies highlights the target area more. Therefore, both can provide effective prior information for subsequent object detection, and they are used together with the original image as input to subsequent modules. After enhancement processing, each feature is fed into a CNN backbone network, here using ResNet101. After CNN processing, its output features encode abstract feature information of different spatial regions in the image. ResNet101 has been widely used in various object detection models as an effective feature extraction module. The Transformer encoder-decoder module encodes and decodes the input features to obtain the object detection output. Before being sent to the encoder, positional encoding (PE) information is introduced into the features to help the encoder understand the relative and absolute position information between different targets in the image.
[0066] Finally, the specific implementation process will be described in conjunction with the accompanying drawings of this embodiment:
[0067] First, such as Figure 6 As shown, after acquiring the original sonar image, the grayscale values obtained by subtracting high-frequency from low-frequency (HML) and low-frequency from high-frequency (LMH) are calculated based on the high- and low-frequency grayscale images of the side-scan sonar. Figure 1 And used as input features, the original feature dimension is The original features are processed by a CNN backbone network, which outputs high-dimensional abstract features containing target features and spatial information. Here, a ResNet101 backbone network is used, and the features are output after CNN processing. Dimensions correspond to ,in , , The output features of high- and low-frequency images will be directly fused to obtain... , ,here The operation concatenates the features along the channel dimension. Subsequently, all features are processed... Convolutional layers, channel dimensions from or Compressed to feature dimension The corresponding feature dimension is , , Before being sent to the Transformer codec, all features are flattened into... Two-dimensional feature maps, these feature sequences It will sequentially pass through the encoder self-attention module, the first addition & normalization module, the forward module, and the second addition & normalization module. For example, the calculation process is as follows: ; in, For self-attention function, subscript Indicates position code, This represents the dimension of the input features. After processing by the self-attention module, its output is denoted as... .
[0068] The addition & normalization module consists of a skip connection, a Dropout layer, and a layer normalization module. The first addition & normalization module... and As input, its calculation process can be written as follows: The dropout ratio is 0.1. Subsequently, the feature sequence is passed through a feedforward network consisting of linear layers, dropout layers, and ReLU layers, and then through a second addition and normalization module. The remaining feature processing methods are the same. ,correspond , , Each by , , These features were updated accordingly. Ultimately, these updated features ( ) and target query vector ( The Transformer decoder consists of self-attention and cross-attention modules. The self-attention module is designed similarly to the encoder. After self-attention output, the features pass through residual connections and normalization layers before reaching the cross-attention module. The encoder output... and The key-value pairs used as input for cross-attention, along with the target query vector after adding positional encoding. The updated encoding information is obtained after multi-head attention calculation.
[0069] Furthermore, such as Figure 7 As shown, the Cross Modal Attention (CMA) module also functions as an encoder, with its output features fed into a shared decoder module to aid in the computation of the final output. The CMA part includes self-attention and cross-attention modules, which mine complementary information in low-frequency and high-frequency images through multi-head attention to assist in the object detection task. Its input features are obtained from the output of the backbone CNN network after post-processing, and are denoted as follows: and , Figure 6A flowchart of the cross-modal modeling module is provided.
[0070] The self-attention module in the cross-modal modeling module operates similarly to the self-attention mechanism in the encoder of the single-modal modeling module. Here, the number of heads in the multi-head attention module... All are defined as 16. and Features by and After processing by the self-attention module, the output is then processed by the cross-attention module, which extracts effective information from the features of high- and low-frequency image pairs and extracts cross-modal adaptation information. The calculation process is as follows: ; ; in This is for and Each feature is individually encoded, and the multi-head mechanism in the cross-attention module allows the model to perform joint optimization at different locations within the subspace containing high- and low-frequency features, thus optimizing the output features. Finally, the output of the cross-attention module is fused to obtain the enhanced features. The final output of the cross-modal modeling module is then obtained after processing by the self-attention module. The output is sent to the Transformer decoder section in the unimodal modeling module. Together with the output features of the unimodal modeling module and the shadow-highlight matching modeling module, it serves as the decoder input. After decoding, the final output features are obtained.
[0071] Finally, the processing logic of the shadow-highlight matching modeling module is the same as that of the cross-modal modeling module; the only difference lies in the input features. Considering the enhanced high- and low-frequency images ( , The highlighted and shadow regions of the target are significantly enhanced. When modeling the matching of shadow and highlighted regions, both are used as effective input features, ultimately outputting Shadow-Target Pairing (STP) features, denoted as... The shadow-target matching information, along with the output of the unimodal modeling module, is used as the input features of the decoder.
[0072] The hidden features output by the decoder are passed through a linear layer to obtain the final prediction output, which includes the target class label. and the normalized coordinates of the target bounding box in the image coordinate system The output is used to calculate the generalized IoU loss function with the target label information, and the model parameters are optimized after backpropagation.
[0073] Reference Figure 2 A seabed target detection system based on dual-frequency side-scan sonar images, comprising:
[0074] The first module 201 is used to image the seabed target area using dual-frequency side-scan sonar to obtain high- and low-frequency seabed image data.
[0075] The second module 202 is used to introduce a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module to construct a dual-frequency side-scan sonar image target detection model;
[0076] The third module 203 is used to perform target detection on high and low frequency seabed image data based on the dual-frequency side-scan sonar image target detection model, and obtain seabed target detection results.
[0077] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0078] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A method for detecting seabed targets based on dual-frequency side-scan sonar images, characterized in that, Includes the following steps: High- and low-frequency seabed image data are obtained by imaging the target area on the seabed using dual-frequency side-scan sonar. A dual-frequency side-scan sonar image target detection model is constructed by introducing a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module. The target detection model based on dual-frequency side-scan sonar images is used to detect targets in high- and low-frequency seabed image data, and the seabed target detection results are obtained.
2. The seabed target detection method based on dual-frequency side-scan sonar images according to claim 1, characterized in that, The dual-frequency side-scan sonar image target detection model specifically includes a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module. The first output of the single-modal modeling module is connected to the input of the cross-modal modeling module, and the second output of the single-modal modeling module is connected to the input of the shadow-highlight matching modeling module. The output of the cross-modal modeling module is connected to the first input of the single-modal modeling module, and the output of the shadow-highlight matching modeling module is connected to the second input of the single-modal modeling module. Wherein: The single-modal modeling module includes a spectral enhancement and prior information extraction module, a CNN backbone network, convolutional layers, and a Transformer encoding / decoding module. Both the cross-modal modeling module and the shadow highlight matching modeling module include a multi-head self-attention module and a multi-head cross-attention module.
3. The method for detecting seabed targets based on dual-frequency side-scan sonar images according to claim 2, characterized in that, The step of performing target detection on high and low frequency seabed image data based on the dual-frequency side-scan sonar image target detection model to obtain seabed target detection results specifically includes: High and low frequency seabed image data are input into a dual-frequency side-scan sonar image target detection model; The cross-modal modeling module based on the dual-frequency side-scan sonar image target detection model enhances the high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images output by the CNN backbone network to obtain enhanced high- and low-frequency seabed image feature information. The shadow highlight matching modeling module based on the dual-frequency side-scan sonar image target detection model enhances the high-dimensional abstract features of the high-frequency subtracted low-frequency seabed image and the high-dimensional abstract features of the low-frequency subtracted high-frequency seabed image output by the CNN backbone network to obtain shadow target region matching features. A single-modal modeling module based on a dual-frequency side-scan sonar image target detection model performs target detection on high- and low-frequency seabed image data, enhanced high- and low-frequency seabed image feature information, and shadow target region matching features to obtain seabed target detection results.
4. The seabed target detection method based on dual-frequency side-scan sonar images according to claim 3, characterized in that, The cross-modal modeling module based on the dual-frequency side-scan sonar image target detection model performs feature enhancement on the high-dimensional abstract features of the high-frequency seabed image and the high-dimensional abstract features of the low-frequency seabed image output by the CNN backbone network to obtain enhanced high- and low-frequency seabed image feature information. This step specifically includes: The high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images are input into the cross-modal modeling module of the dual-frequency side-scan sonar image target detection model; Based on the multi-head self-attention module of the cross-modal modeling module, complementary information extraction processing is performed on the high-dimensional abstract features of high-frequency seabed images and the high-dimensional abstract features of low-frequency seabed images to obtain complementary information between high-frequency and low-frequency seabed images. Based on the multi-head cross-attention module of the cross-modal modeling module, the complementary information of high and low frequency seabed images is adapted across modal information to obtain optimized high and low frequency seabed image feature information. Feature fusion and self-attention calculation are performed on the optimized high- and low-frequency seabed image feature information to obtain enhanced high- and low-frequency seabed image feature information.
5. The seabed target detection method based on dual-frequency side-scan sonar images according to claim 4, characterized in that, The single-modal modeling module based on the dual-frequency side-scan sonar image target detection model performs target detection on high- and low-frequency seabed image data, enhanced high- and low-frequency seabed image feature information, and shadow target region matching features to obtain seabed target detection results. This step specifically includes: High and low frequency seabed image data are input into the single-modal modeling module of the dual-frequency side-scan sonar image target detection model; The spectral enhancement and prior information extraction module based on the single-modal modeling module calculates high- and low-frequency seabed image data and outputs high-frequency seabed images, low-frequency seabed images, high-frequency subtracted low-frequency seabed images, and low-frequency subtracted high-frequency seabed images. Based on the CNN backbone network of the single-modal modeling module, feature extraction processing is performed on high-frequency seabed images, low-frequency seabed images, high-frequency subtracted from low-frequency seabed images, and low-frequency subtracted from high-frequency seabed images, respectively, to obtain high-dimensional abstract features of high-frequency seabed images, high-dimensional abstract features of low-frequency seabed images, high-dimensional abstract features of high-frequency subtracted from low-frequency seabed images, and high-dimensional abstract features of low-frequency subtracted from high-frequency seabed images. The high-dimensional abstract features of high-frequency seabed images and low-frequency seabed images are spliced and fused to obtain the high-dimensional abstract features of spliced high- and low-frequency seabed images. Based on the convolutional layer of the single-modal modeling module, the high-dimensional abstract features of the stitched high- and low-frequency seabed images, the high-dimensional abstract features of the high-frequency subtracted from the low-frequency seabed images, and the high-dimensional abstract features of the low-frequency subtracted from the high-frequency seabed images are compressed to obtain high- and low-frequency seabed two-dimensional feature maps, high-frequency subtracted from the low-frequency seabed two-dimensional feature maps, and low-frequency subtracted from the high-frequency seabed two-dimensional feature maps. The enhanced high and low frequency seabed image feature information, shadow target region matching features, high and low frequency seabed two-dimensional feature maps, high frequency reduced low frequency seabed two-dimensional feature maps, and low frequency reduced high frequency seabed two-dimensional feature maps are stitched together and position coding information is introduced to obtain the stitched high and low frequency seabed image feature information. The Transformer encoding / decoding module, based on the single-modal modeling module, encodes and decodes the feature information of the stitched high and low frequency seabed images to obtain the seabed target detection results.
6. The seabed target detection method based on dual-frequency side-scan sonar images according to claim 5, characterized in that, The Transformer encoding / decoding module based on the single-modal modeling module performs encoding / decoding processing on the stitched high- and low-frequency seabed image feature information to obtain the seabed target detection result. This step specifically includes: The stitched high and low frequency seabed image feature information is input into the Transformer encoding and decoding module of the single-modal modeling module; The encoder module based on the Transformer encoder-decoder module encodes the feature information of the stitched high and low frequency seabed images to obtain updated high and low frequency seabed image feature information and target query vector. The decoder module, based on the Transformer encoding and decoding module, performs multi-head attention calculation on the updated high and low frequency seabed image feature information and target query vector to obtain seabed target detection results.
7. The seabed target detection method based on dual-frequency side-scan sonar images according to claim 6, characterized in that, The seabed target detection results specifically include target category labels and the normalized coordinates of the target location box in the image coordinate system.
8. A seabed target detection system based on dual-frequency side-scan sonar images, characterized in that, Includes the following modules: The first module is used to image the seabed target area using dual-frequency side-scan sonar to obtain high- and low-frequency seabed image data. The second module is used to introduce a single-modal modeling module, a cross-modal modeling module, and a shadow-highlight matching modeling module to construct a dual-frequency side-scan sonar image target detection model. The third module is used to perform target detection on high and low frequency seabed image data based on the dual-frequency side-scan sonar image target detection model, and obtain seabed target detection results.