A target shape adaptive method for detecting sunken ships based on side-scan sonar images

Abstract

This paper presents an adaptive target shape detection method for seabed shipwrecks based on side-scan sonar images, belonging to the research field of seabed target detection using side-scan sonar images. First, in the feature extraction stage, we designed a dynamic rotational convolution to extract high-quality rotational features based on the target's orientation information. Next, to address the inconsistency in features required for classification and regression tasks, we designed a feature decoupling head, inputting rotation-dependent and rotation-invariant features into different task branches, thus improving the accuracy of classification and regression. Finally, in the training strategy, we proposed the S-A label allocation strategy, introducing the concept of alignment and comprehensively evaluating sample quality for label allocation by integrating information such as IoU, distance between center points, and angular differences. These three modules are efficiently coupled together, ultimately achieving efficient and accurate detection.

Description

Technical Field

[0001] This invention belongs to the research field of seabed target detection in side-scan sonar images, and particularly relates to a seabed shipwreck detection method with adaptive target shape. Background Technology

[0002] The detection and identification of seabed targets plays a crucial role in underwater search and rescue, marine engineering construction, marine topography and geomorphology surveying, and marine resource investigation. However, due to the complex marine environment, imaging conditions, and measurement methods, detection is more difficult than target detection in natural images, and the required accuracy is often insufficient, making it a current research hotspot and challenge. Acoustic detection utilizes acoustic images formed from echo information from targets, which are then manually interpreted to detect seabed targets. Acoustic detection is widely used in underwater target detection due to its mature technology, intuitiveness, efficiency, and ease of use. Side-scan sonar, as the primary equipment for seabed topography and geomorphology surveying, has become the mainstream equipment for underwater target detection due to its high imaging resolution.

[0003] Currently, with the development of deep convolutional neural networks (DCNNs), many carefully designed structures tailored to the characteristics of sonar images have achieved remarkable results. Most object detection methods primarily detect objects using horizontal bounding boxes (HBBs), whose most prominent feature is that their edges are parallel to the horizontal and vertical axes of the image. Among these, the two-stage detection model R-CNN, based on horizontal bounding boxes, first generates a region proposal network to initially filter out candidate regions that may contain the target; then, a classification and bounding box regression network further determines the category and precise location of the target within the candidate boxes. Single-stage detection models, such as SSD and the YOLO series, simplify the object detection task to a single stage, eliminating the intermediate candidate region generation step and directly predicting the target category and bounding box location on the input image or feature map. All of the above methods perform object detection based on horizontal bounding boxes. However, target entities in side-scan sonar images (such as the most common shipwreck targets) are usually placed in arbitrary orientations and have high aspect ratios. Horizontal bounding box detection cannot accurately represent targets in arbitrary orientations and introduces a large amount of background information, posing a significant challenge to the detection algorithm in accurately locating oriented objects.

[0004] Rotated target detection is an extremely challenging task, more difficult and complex than traditional target detection, mainly in the following three aspects:

[0005] (1) Inconsistency between the target and the anchor frame

[0006] Current convolutional features in sonar networks are typically axis-aligned with fixed receptive fields. However, when dealing with objects distributed in arbitrary directions in sonar images, a misalignment occurs between the anchor boxes and the convolutional features, making it difficult to accurately represent the characteristics of the objects. In other words, existing methods produce low-quality anchor boxes that fail to cover the objects, leading to inconsistencies between the objects and the anchor boxes, and the features within the anchor boxes failing to represent the entire object. This phenomenon is even more pronounced for objects with high aspect ratios; for example, the aspect ratio of underwater shipwrecks is typically between 1 / 3 and 1 / 10. This misalignment exacerbates the imbalance between the target and background information, hindering performance.

[0007] (2) Inconsistent classification features and regression features

[0008] In seabed target detection models, classification and regression tasks rely on features extracted from the backbone network, and these features are typically rotation-invariant. However, in sonar-based seabed target detection, targets are distributed in arbitrary directions. In classification, we need to use fixed features to determine the target category, i.e., rotation-invariant features. Because targets in side-scan sonar images rotate in multiple directions, it is difficult to obtain accurate target location information. Therefore, as the angle changes, we need to extract features from different angles to perceive changes in target position and thus perform precise localization, i.e., rotation-variable features.

[0009] (3) Inconsistency between rotating frame quality and label assignment strategy

[0010] For oriented targets with high aspect ratios, the Intersection over Union (IoU) is highly sensitive to changes in angle. Even small changes in angle can lead to drastic changes in IoU. Furthermore, a high IoU does not necessarily indicate good classification performance. Due to the high aspect ratio of objects, it is difficult to accurately capture various features of the target when setting anchor boxes. Some high-IoU boxes, while summarizing the main location information of the target and potentially performing better in regression, lack key classification features, resulting in poor performance; these are considered low-quality samples but are retained. Conversely, some low-IoU boxes may capture key features and key location nodes, leading to good performance; these high-quality samples are treated as negative samples. Therefore, current label assignment methods that only distinguish between positive and negative samples based on IoU scores can lead to an imbalance between positive and negative samples, thus affecting model performance. Summary of the Invention

[0011] This invention proposes S 3The DR-Det model addresses the inconsistency issue in rotating object detection from three levels. First, in the feature extraction stage, we designed a dynamic rotational convolution that extracts high-quality rotational features based on the target's orientation. Next, to address the inconsistency in features required for classification and regression tasks, we designed a feature decoupling head that inputs rotation-dependent and rotation-invariant features into different task branches, resulting in more accurate classification and regression. Finally, in our training strategy, we proposed the SA label assignment strategy, introducing the concept of alignment and comprehensively evaluating sample quality for label assignment by integrating information such as IoU, distance between center points, and angle differences. These three modules are efficiently coupled together, ultimately achieving efficient and accurate detection.

[0012] To achieve the above objectives, the technical solution of the present invention is as follows:

[0013] A target shape adaptive method for detecting sunken ships based on side-scan sonar images, specifically including the following steps:

[0014] The first step is dataset preprocessing.

[0015] The images in the dataset are converted to grayscale, and the dataset is divided into training set, validation set and test set.

[0016] The second step is the construction and training of the network model.

[0017] The network model includes Backbone, Neck, and Head; the network model is trained and validated using training and validation sets to obtain a well-trained network model.

[0018] (1) Backbone

[0019] The backbone extracts image features through a series of convolutional layers and activation functions, gradually reducing the spatial dimension of the image while increasing the number of channels. The standard convolutions in the ResNet backbone are replaced with DRC modules. The DRC module's specific processing steps are as follows: First, the feature map is input into a depthwise convolution, then layer normalization and ReLU activation are performed. The activated features are then merged through average pooling and max pooling to obtain rich features. The merged feature vector is then passed through linear layers and different activation functions to obtain the predicted rotation angle α = [α1,...,α]. n and weights ω = [ω1,...,ω] n Each convolution kernel is rotated according to the rotation angle and weight, and the rotated convolution kernel is convolved with the feature map. The output features are then added pixel by pixel to obtain the rotated features.

[0020] (2) Neck

[0021] The Neck section uses Feature Pyramid Networks (FPNs), located between the Backbone and Head. The FPNs fuse the rotation features extracted from the Backbone accordingly.

[0022] Furthermore, FPN constructs bottom-up and top-down feature fusion paths to fuse feature maps of different scales, thereby generating a feature pyramid with rich multi-scale information. Each layer of the feature pyramid corresponds to a specific scale range, enabling the model to process targets of different sizes simultaneously.

[0023] (3) Head

[0024] The Head generates the final detection results from the feature maps provided by the Neck, namely the target category, bounding box information, and confidence score. The Head uses an Adaptive Feature Decoupling Head Structure (FDM), which generates rotation-transformed and rotation-invariant features from the fused features. These features are then input into the regression and classification sub-networks respectively to generate the final prediction, thus improving the model's prediction accuracy.

[0025] Furthermore, the adaptive feature decoupling head structure includes an anchor frame optimization module and a dynamic refinement module.

[0026] The anchor box optimization module includes anchor box regression and rotation convolution feature alignment operations. Anchor box regression optimizes horizontal anchor boxes into anchor boxes that closely resemble the target shape and have a certain rotation angle. The rotation convolution feature alignment operation dynamically and adaptively aligns the target features according to the shape, size, and orientation of the corresponding anchor boxes.

[0027] The dynamic refinement module adds a dynamic rotation encoder (DRE) before the classification and regression sub-networks and the classification sub-network. The DRE encodes directional information to generate a feature map with multiple directional channels. The DRE is a k×k×N filter that actively rotates N-1 times during convolution, producing a feature map with N directional channels. For a feature map A and a DRE, the output S in the i-th direction is represented as:

[0028]

[0029] In the formula, α i This represents the angle of rotation of the filter, where n represents the nth direction channel. Let A represent the nth dynamic rotary encoder and A represent the nth feature map.

[0030] Furthermore, during the network model training process, an alignment-based dynamic label allocation strategy is adopted. This strategy uses alignment as an indicator to measure the quality of anchor boxes, and is defined as follows:

[0031]

[0032] Among them, IoU pre The rotational IoU value before regression, IoU post Here are the rotation IoU values ​​after regression, where ad is the alignment, d is the distance between the center point of the rotated predicted bounding box and the center point of the rotated ground truth bounding box, θ is the angular difference between the rotated predicted bounding box and the rotated ground truth bounding box, and max is the rotation IoU value. d With max θ These represent the maximum possible distance and the maximum angle difference, respectively. In the formula, α, β, and γ are weighted hyperparameters used to measure the degree of influence between different terms.

[0033] During the training phase, the adjective (ad) scores of the ground truth (GT) and predicted bounding boxes are first calculated. Then, anchor boxes with ad scores greater than or equal to a certain threshold are selected as positive samples, while those below the threshold are selected as negative samples. For GT boxes that do not match any anchor boxes, the anchor box with the highest ad score is used as a positive candidate box for compensation, thereby achieving dynamic selection of label assignment.

[0034] The third step is model evaluation.

[0035] By feeding the test set into the trained model, target information in side-scan sonar images is obtained. The model performance is evaluated using metrics such as precision (P), recall (R), and mean average precision.

[0036] The beneficial effects of this invention are as follows: This invention proposes an S-type method for detecting shipwreck targets in side-scan sonar images. 3 DR-Det is used to detect multi-directional, high aspect ratio targets in side-scan sonar images. Through our proposed DRC, FDM, and SA label assignment strategies, we address the inconsistencies between target and anchor boxes, classification and regression features, and the quality of rotated boxes compared to existing label assignment strategies at the feature extraction, detection, and training stages, respectively. The modules in our model are highly coupled in function and structure, resolving the problems caused by high aspect ratio targets in arbitrary directions at different stages of the model. Attached Figure Description

[0037] Figure 1 This is a diagram of the overall network structure.

[0038] Figure 2This is a schematic diagram of the Dynamic Rotation Convolution (DRC) module.

[0039] Figure 3 This is a schematic diagram of the feature decoupling detection head structure;

[0040] Figure 4 This is a partial display of the test results. Detailed Implementation

[0041] To make the problem solved by the present invention, the method scheme adopted, and the effect achieved by the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and experiments. It should be understood that the specific experiments described herein are merely for explaining the present invention and not for limiting the present invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the accompanying drawings, not all of them.

[0042] The overall structure of the model of this invention is as follows: Figure 1 As shown, it consists of three parts: Backbone, Neck, and Head. In the Backbone, we designed a DRC convolution module. The dynamically rotated convolutional kernels within this module can dynamically rotate according to different input feature maps to extract rotational features of targets in any orientation, thereby improving the model's ability to represent targets in different orientations. We build upon ResNet by replacing the standard convolutions in the backbone with DRC and integrating the dynamically rotated convolutional DRC module. The specific principle of the DRC module is explained by... Figure 2 As can be seen, we first input the feature map into a depthwise convolution, then perform layer normalization and ReLU activation, and then merge the activated features through average pooling and max pooling to obtain rich features. The merged feature vector is then passed through a linear layer and different activation functions to obtain the predicted rotation angle α = [α1,...,α]. n and weights ω = [ω1,...,ω] n Each convolutional kernel is rotated according to the rotation angle and weight, and the rotated convolutional kernel is convolved with the feature map. The output features are then added pixel by pixel.

[0043] To address the inconsistency in features required for classification and regression tasks, we designed an adaptive feature decoupling header structure (FDM), as shown in the following figure. Figure 3As shown, the adaptive feature decoupling head structure includes an anchor frame optimization module and a dynamic refinement module. The dynamic refinement module adds a dynamic rotation encoder before the classification and regression task branches. First, the anchor frame optimization module generates high-quality anchor frames, and adaptively aligns features with the corresponding anchor frames and dynamic rotation convolution, inputting the features corresponding to the classification and regression tasks into different task branches. The anchor frame optimization module (AFO) includes an anchor frame regression and a rotation convolution feature alignment operation. Anchor frame regression optimizes horizontal anchor frames into high-quality anchor frames that closely resemble the target shape and have a certain rotation angle. The rotation convolution feature alignment operation dynamically and adaptively aligns the target features based on the shape, size, and orientation of the corresponding anchor frames. Next, the dynamic rotation encoder (DRE) in the dynamic refinement module encodes the orientation information to generate a feature map with multiple directional channels. The DRE is a k×k×N filter that actively rotates N-1 times during convolution, producing a feature map with N (default is 8) directional channels. For a feature map A and a DRE (represented by E()), the output S in the i-th direction can be expressed as...

[0044]

[0045] In the formula, α i Let represent the angle of filter rotation, and n represent the nth directional channel. By applying DRE to the convolutional layer, we can obtain rotation-sensitive features encoded with directional information. Furthermore, we can select the directional channel with the strongest response as the input feature for a classification task.

[0046] During the overall model training process, we propose a dynamic label allocation strategy (SA) based on spatial matching prior information (SMPI). This strategy primarily refers to the detection model distinguishing between positive and negative samples during training and matching appropriate supervision targets to different locations on the feature map for loss calculation, thereby completing gradient updates. We comprehensively consider three factors: IoU, center point distance difference, and angle difference. Based on these three factors, we designed the SA label allocation strategy as a basis for distinguishing between positive and negative samples, more comprehensively evaluating anchor box quality, and effectively improving model performance. We introduce alignment as an indicator to measure anchor box quality, defined as follows:

[0047]

[0048] Among them, IoU pre The rotational IoU value before regression, IoU post Here are the rotation IoU values ​​after regression, where ad is the alignment, d is the distance between the center point of the rotated predicted bounding box and the center point of the rotated ground truth bounding box, θ is the angular difference between the rotated predicted bounding box and the rotated ground truth bounding box, and max is the rotation IoU value. d With max θThese represent the maximum possible distance and the maximum angle difference, respectively. In the formula, α, β, and γ are weighted hyperparameters used to measure the degree of influence between different terms.

[0049] During the regression process, effectively suppressing interference can yield higher quality anchor boxes, making training more stable. We construct a penalty term for regression uncertainty consisting of three parts: the IoU part, the distance between center points, and the angle difference. This effectively selects high-quality, highly aligned rotating anchor boxes from three different aspects.

[0050] During the training phase, the adjective (ad) scores of the ground truth (GT) and predicted bounding boxes are first calculated. Then, anchor boxes with ad scores greater than or equal to a certain threshold are selected as positive samples, and those below the threshold are selected as negative samples. For GT boxes that do not match any anchor boxes, the anchor box with the highest ad score is used as a positive candidate box for compensation, thereby achieving dynamic selection of label assignment.

[0051] The first step is dataset preprocessing.

[0052] We selected a dataset of shipwreck targets with high aspect ratios and multiple rotation directions for experimental validation. The experimental dataset was collected by various domestic research institutes and manufacturers using mainstream domestic and international side-scan sonar instruments in different sea areas, as well as online resources. This dataset contains 1691 shipwreck sample images, and the targets in the dataset are placed in arbitrary directions with high aspect ratios, making it suitable for validating the effectiveness of the dynamic rotating target detection model. We divided the entire dataset into training, validation, and test sets in a 5:2:3 ratio.

[0053] Due to the wide availability of training samples, we converted all samples to grayscale before inputting them into the network for training to improve model generalization. During training, only random horizontal flipping was used to avoid overfitting, and a small learning rate was employed to prevent drastic changes in rotation angle. The optimizer used for training was SGD, with an initial learning rate of 0.0025, momentum of 0.9, and weight decay set to 0.0001 to avoid overfitting or underfitting. Training included 500 warm-up iterations before starting training, and no pre-trained weights were used during training; training started from scratch. This model was implemented in Python using the PyTorch deep learning framework, running on Windows 11. The hardware used for the experiment included an Intel Core i7-14650HX CPU, an NVIDIA GeForce RTX 4060 Laptop GPU, and 64GB of RAM.

[0054] The second step is model training.

[0055] The model training process is carried out in accordance with the technical solution.

[0056] The third step is model evaluation.

[0057] In order to comprehensively and objectively evaluate the prediction performance of different models, this invention evaluates the performance of the optimized model through the following coefficients: average precision (AP) and mean average precision (mAP).

[0058] Precision and Recall: In the classification task of predicting whether an image contains a bag, the four elements of precision and recall can be explained as follows: TP (true positive): Positive samples correctly labeled as positive; TN (true negative): Negative samples correctly labeled as negative; FP (false positive): Positive samples incorrectly labeled as negative; FN (false negative): Negative samples incorrectly labeled as positive. The calculation relationship is:

[0059]

[0060]

[0061] Average accuracy: The geometric meaning of average accuracy AP is the area corresponding to the PR curve as shown in equation (5), where the integral can be approximated by interpolation and summation.

[0062]

[0063] We conducted comparative experiments with existing rotating target detection methods, including two-stage and single-stage detection models, and the results are shown in Table 1. From the results in the table, our model achieved an AP result of 89.68%, which is superior to all two-stage and single-stage detection models in the table, resulting in higher Recall and AP values, demonstrating the superiority of our algorithm in rotating target detection.

[0064] Table 1. Experimental results compared with different models

[0065]

[0066]

[0067] To verify the effectiveness of the DRC, FDM, and SA designed in this invention, we conducted ablation experiments, using different module combinations to verify their effects on the model. The results are shown in Table 2.

[0068] Table 2 shows the module ablation experiments in the model.

[0069]

[0070] We will configure our proposed DRC, FDM, and SA modules in the baseline model for ablation experiments to verify the effect of each component on the model, and conduct experiments on the dataset. The results show that each module, individually configured in the baseline model, significantly improves performance. Compared to the baseline model using static convolution, the addition of the DRC module allows the rotated convolution kernel to dynamically align angles, demonstrating the adaptability and effectiveness of the DRC module in capturing rotating targets. The FDM module effectively decouples the extracted rotation features, inputting the corresponding features into a special branch network to achieve fine-grained localization and classification. Therefore, adding the FDM module effectively improves the model's detection accuracy and achieves a higher AP value. Finally, SA introduces a more accurate alignment to measure anchor box quality, incorporating input IoU, center point distance difference, and angle difference for a more comprehensive and realistic selection of high-quality anchor boxes to improve model accuracy.

[0071] Some representative test results, such as Figure 4 As shown, although the targets are distributed in any direction in the image, the detection model can still accurately identify the targets and make accurate bounding boxes according to the target direction, effectively improving the model's localization and recognition accuracy. However, we can also find that the detection boxes for some targets are slightly offset. This is because some shipwreck targets have been submerged for a long time, and some parts are buried, making it difficult to distinguish the edge contours of the shipwreck from the seabed background, resulting in positioning deviations. At the same time, due to observation conditions, instrument equipment, and other factors, the target imaging quality is poor, and the effective echo of the target is obscured by suspended objects in the water, making it difficult to accurately identify the target.

[0072] Finally, it should be noted that the above experiments are only used to illustrate the method of the present invention, and not to limit it; although the present invention has been described in detail, those skilled in the art should understand that modifications to the foregoing method, or equivalent substitutions for some or all of the method features, do not cause the essence of the corresponding method to deviate from the scope of the present invention.

Claims

1. A method for adaptive detection of shipwrecks based on side-scan sonar images, characterized in that, Specifically, the following steps are included: The first step is dataset preprocessing. The images in the dataset are converted to grayscale, and the dataset is divided into training set, validation set and test set. The second step is the construction and training of the network model. The network model includes Backbone, Neck, and Head; the network model is trained and validated using training and validation sets to obtain a well-trained network model. (1) Backbone The backbone extracts image features through a series of convolutional layers and activation functions, gradually reducing the spatial dimension of the image while increasing the number of channels; it replaces the standard convolutions in the ResNet backbone with DRC modules. The specific processing procedure of the DRC module is as follows: First, the feature map is input into a depthwise convolution, then layer normalization and ReLU activation are performed. The activated features are then merged using average pooling and max pooling to obtain rich features. The merged feature vector is then passed through a linear layer and different activation functions to obtain the predicted rotation angle α = [α1,...,α]. n and weights ω = [ω1,...,ω] n ]; Rotate each convolution kernel according to the rotation angle and weight, convolve the rotated convolution kernel with the feature map, and add the output features pixel by pixel to obtain the rotated features; (2) Neck The Neck section uses a Feature Pyramid Network (FPN), located between the Backbone and Head. The FPN fuses the rotation features extracted from the Backbone accordingly. (3) Head The Head generates the final detection result from the feature map provided by the Neck, namely the target category, bounding box information, and confidence score; the Head uses an adaptive feature decoupling head structure (FDM). The adaptive feature decoupling head structure (FDM) generates rotation-transformed and rotation-invariant features from the fused features, which are then input into the regression and classification sub-networks respectively to generate the final prediction, thereby improving the model's prediction accuracy. The adaptive feature decoupling head structure includes an anchor frame optimization module and a dynamic refinement module; The anchor box optimization module includes anchor box regression and rotation convolution feature alignment operations; anchor box regression optimizes horizontal anchor boxes into anchor boxes that closely resemble the target shape and have a certain rotation angle; Rotational convolution feature alignment is a dynamic and adaptive alignment of target features based on the shape, size, and orientation of the corresponding anchor box. The dynamic refinement module adds a dynamic rotary encoder (DRE) before the classification and regression subnetwork and the classification subnetwork. The dynamic rotary encoder encodes the orientation information to generate a feature map with multiple orientation channels. The third step is model evaluation. By feeding the test set into the trained model for detection, target information in side-scan sonar images is obtained. At the same time, the model performance is evaluated by precision P, recall R, and mean precision.

2. The method for adaptive detection of shipwrecks based on side-scan sonar images according to claim 1, characterized in that, FPN fuses feature maps of different scales by constructing bottom-up and top-down feature fusion paths, thereby generating a feature pyramid with rich multi-scale information; each layer of the feature pyramid corresponds to a specific scale range, enabling the model to process targets of different sizes simultaneously.

3. The target shape adaptive seabed shipwreck detection method based on side-scan sonar images according to claim 1, wherein the dynamic rotary encoder is a k×k×N filter that can actively rotate N-1 times during convolution to generate a feature map with N directional channels. For a feature map A and a DRE, the output S in the i-th direction is expressed as: In the formula, α i This represents the angle of rotation of the filter, where n represents the nth direction channel. Let A represent the nth dynamic rotary encoder and A represent the nth feature map.

4. The target shape adaptive seabed shipwreck detection method based on side-scan sonar images according to claim 1, in the network model training process, adopts an alignment degree dynamic label allocation strategy based on spatial matching prior information. The alignment degree dynamic label allocation strategy uses alignment degree as an indicator to measure the quality of the anchor frame, and is defined as follows: in, IoU pre The rotational IoU value before regression, IoU post Here are the rotation IoU values ​​after regression, where ad is the alignment, d is the distance between the center point of the rotated predicted bounding box and the center point of the rotated ground truth bounding box, θ is the angular difference between the rotated predicted bounding box and the rotated ground truth bounding box, and max is the rotation IoU value. d With max θ These represent the maximum possible distance and the maximum angle difference, respectively. In the formula, α, β, and γ are weighted hyperparameters used to measure the degree of influence between different terms. During the training phase, the ad scores of the ground truth bounding boxes (GT) and the predicted bounding boxes are first calculated. Then, anchor boxes with ad scores greater than or equal to a certain threshold are selected as positive samples, and those with ad scores lower than the threshold are selected as negative samples. For GT boxes that do not match any anchor boxes, the anchor box with the highest ad score is used as a positive candidate box for compensation, thereby achieving dynamic selection of label assignment.

Images