An underwater target detection method and system based on dynamic perception area routing

Abstract

The application discloses a kind of underwater target detection method and system based on dynamic perception area routing, specifically includes the following steps: S1, input preprocessing;S2, feature extraction;S3, feature fusion;S4, prediction;The present application relates to the technical field of underwater target identification.This underwater target detection method and system based on dynamic perception area routing first in the feature extraction stage and feature fusion stage by using different hollow convolution to different size target, adaptively enhance the features of large, medium and small targets, improve the accuracy of the model in the target regression prediction stage, then in the feature fusion stage by the regional routing attention mechanism Dynamic perception enhances foreground target features, prevent the interference of background information in the multi-dimensional mixed feature information extracted in the feature extraction stage.Finally, by improving the loss function, the model is associated with the originally independent positioning task and classification task, while reducing the influence of model overfitting caused by the imbalance of positive and negative samples.

Description

Technical Field

[0001] This invention relates to the field of underwater target recognition technology, specifically to an underwater target detection method and system based on dynamic sensing area routing. Background Technology

[0002] Due to the complexity of underwater acoustic channels and the loss and scattering during sound wave propagation, images directly acquired by sonar equipment often have characteristics such as high noise, severe distortion, blurred target edges, and low resolution. Traditional sonar image target detection algorithms mainly include pixel-based, feature-based, and echo-based detection methods and machine learning-based target detection methods. The former's basic idea is to determine the presence of a target based on the brightness and shadow characteristics in the sonar image, while the latter mainly extracts features from the sonar image through prior knowledge and then uses methods such as support vector machines for target detection. However, traditional target detection methods rely on the establishment of accurate mathematical models. With the increasing complexity of underwater channels and underwater targets, these methods can no longer meet the practical application needs of current underwater target identification.

[0003] With breakthroughs in computer performance, deep learning-based target detection techniques, originally designed for optical image processing, have been increasingly applied to underwater target recognition, yielding promising results. McKay et al. proposed a method for multi-instance target detection and recognition using transfer learning on sonar datasets (McKay J. "What's mine is yours: PretrainedCNNs for limited training sonar ATR," OCEANS 2017-Anchorage, IEEE, 2017). Williams et al. built a convolutional neural network model for classifying synthetic aperture sonar images and applied the learned network to several binary classification tasks (William David P., "Underwater target classification in synthetic aperture sonar imagery using deep convolutional neural networks," 2016 23rd international conference on pattern recognition (ICPR), pp.2497-2502, IEEE). In 2016, Chen et al. improved the anchor boxes of YOLOv3 and used the K-Means algorithm to cluster the true width and height of the target boxes in the side-scan sonar dataset. They then designed a hyperparameter mapping relationship to stretch the obtained anchor boxes, thus obtaining a set of better prior anchor boxes. After obtaining the optimized anchor boxes, they set the anchor boxes as the prior parameters of YOLOv3, and then input the dataset into the network for prediction (CHEN Y., MA X. and LI X., “Target detection in sidescan sonar images based on YOLOv3 anchor boxes optimization,” Journal of Signal Processing, vol.38, no.11, pp.2359-2371, 2022). Wang et al. proposed an underwater target detection model based on YOLOv3, YOLOv3F, which solves the problem of shape distortion and aberration in underwater sonar images by inputting the original sonar images and the corresponding generated point cloud data into the YOLOv3 target detection network for feature extraction and target detection (WANG F., WANG X., ZHOU). J. and LIU M., "An Underwater ObjectDetection Method for Sonar Image Based on YOLOv3 Model," Journal of Electronics & Information Technology, vol.44, no.10, pp.3419-3426, 2022). .

[0004] Based on the search of the above materials, it can be seen that the following problems still exist:

[0005] I. Existing deep learning-based underwater sonar image target detection models require a large amount of input data for model training. However, due to the cost of acquisition and environmental factors, it is difficult to obtain a large amount of experimental data for underwater sonar images, and the data of each category of the acquired experimental samples are imbalanced. When using small sample imbalanced data to train existing models, it is very easy to cause model overfitting.

[0006] Second, due to the influence of underwater background noise, biological noise and various underwater vehicle radiated noise, the foreground and background of sonar images are poorly separable, and the feature extraction method using ordinary convolutional neural networks will lose a large number of effective features.

[0007] Third, since sonar images are mainly generated by passive sonar long-distance acquisition of underwater acoustic signals, distortion and scaling will occur during the process of converting acoustic signals into images. Therefore, sonar images mostly contain small and medium-sized targets, and existing target detection models are very likely to lose information about small targets during feature extraction. Summary of the Invention

[0008] (a) Technical problems to be solved

[0009] To address the shortcomings of existing technologies, this invention provides an underwater target detection method and system based on dynamic sensing area routing, which solves the aforementioned problems.

[0010] (II) Technical Solution

[0011] To achieve the above objectives, the present invention provides the following technical solution: an underwater target detection method based on dynamic sensing area routing, specifically comprising the following steps:

[0012] S1. Input preprocessing: Mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling are used to augment and enhance the original input image data.

[0013] S2, Feature Extraction: The output of S1 is input into the feature extraction module. After convolution processing and the cooperation of the multi-residual module based on dynamic feature enhancement, the feature information of the first gradient, the second gradient and the third gradient are obtained from the output of S1 in sequence.

[0014] S3, Feature Fusion: The feature information of the third gradient obtained in S2 is input into the Spatial Pyramid Fusion Structure (SPPF) to obtain a fused feature layer. After convolution, a region routing attention mechanism is introduced to upsample the fused feature layer to obtain a first-level feature layer. After processing the first-level feature layer by the dynamic feature enhancement multi-residual module, it is passed together with the feature information of the second gradient in S2 to the next-level dynamic feature enhancement multi-residual module. After convolution and upsampling operations, a second-level feature layer is obtained. The second-level feature layer and the feature information of the first gradient in S2 are input into the subsequent dynamic feature enhancement multi-residual module to obtain a first-level gradient output feature layer. After convolution, the first-level gradient output feature layer is concatenated with the second-level feature layer without upsampling and then input into the subsequent dynamic feature enhancement multi-residual module to obtain a second-level gradient output feature layer. After convolution, the second-level gradient output feature layer is concatenated with the fused feature layer without the region routing attention mechanism and then input into the subsequent dynamic feature enhancement multi-residual module to obtain a third-level gradient output feature layer.

[0015] S4. Prediction: The features from the first gradient output feature layer, the second gradient output feature layer, and the third gradient output feature layer obtained in S3 are input into the prediction module. Anchor boxes are used to predict the position and size of the bounding box of the target. At the same time, the softmax function is used to calculate the class probability for the prediction result corresponding to each anchor box.

[0016] The present invention is further configured such that the multi-residual module processing method based on dynamic feature enhancement in S2 and S3 is as follows:

[0017] A1. The feature layer input from the upper layer is first processed by a 1×1 convolution and a residual module with a dynamic dilated convolution module to further filter the features extracted from the previous feature layer, retaining multi-size target features.

[0018] B1. Change the channel dimension by performing a 1×1 convolution on the feature layer input from the upper layer;

[0019] C1. Concatenate the features of the output structures of A1 and B1 along the channel dimension, and use a 1×1 convolution to change the channel dimension.

[0020] The present invention is further configured such that the dynamic dilated convolution module is used as follows:

[0021] A2. Enhance the original feature information through a global average pooling residual layer;

[0022] B2. Full-dimensional target feature filtering is performed through dynamic dilated convolutional layers. The detailed calculation process is as follows:

[0023] When using The formula for the dynamic dilated convolution module operation, which takes x as input, weights w, a dilated convolution rate of r, and outputs y, is as follows:

[0024]

[0025] Where r is a hyperparameter of the dynamically dilated convolution module. These are trainable parameters. It is implemented by an average pooling operation with a kernel size of 5 and a convolution operation with a kernel size of 1. S is a 0-1 probability matrix. SAC extracts a portion of the values ​​from the feature maps with Atrous=1 and Atrous=3 respectively to obtain new combined version feature maps. For example, if S>0.5, the new combined version feature maps mostly come from the Atrous=1 branch; if S<0.5, they mostly come from the Atrous=3 branch.

[0026] C2. The feature information of each size target after screening is further enhanced by the global average pooling residual layer.

[0027] The present invention is further configured such that: after the feature information of the third gradient is input into the spatial pyramid fusion structure SPPF, it first goes through a convolution and three global max pooling layers, and then the four output results are concatenated in the channel dimension.

[0028] The present invention is further configured such that the area routing attention mechanism in S3 specifically includes the following three parts:

[0029] The first part is region partitioning and input projection. For a feature map of H×W×C, it is divided into S×S non-overlapping regions, and then the projections of query Q, key K, and value V are derived as follows:

[0030]

[0031] in , , These are the projection weights for the query, key, and value, respectively.

[0032] The second part involves region-to-region routing with directed graphs, identifying participation relationships by constructing directed graphs. Specifically, region-level routing is derived by applying the average value of each region to Q and K respectively. and Then through and Matrix multiplication between regions to derive the adjacency matrix between regions. :

[0033] ;

[0034] The third part retains the k connections with the highest similarity to other regions for each region. The matrix composed of all these regions and their indices is as follows:

[0035] ;

[0036] For each region, the key K and value V are gathered together, and token-to-token attention is used to obtain the final output of the region routing attention mechanism:

[0037] .

[0038] The present invention is further configured such that: the first gradient output feature layer, the second gradient output feature layer and the third gradient output feature layer in S4 correspond to prediction boxes of different scales, and each prediction box includes a confidence score, a class probability, a bounding box position and a size.

[0039] The present invention is further configured such that: after S4 performs result prediction, a joint loss function for localization and classification based on weight sequence is used as the classification loss and localization loss, and the derivation of the loss function is as follows:

[0040]

[0041]

[0042] Where P is the set of positive samples, i.e., TP, and N is the set of negative samples. This indicates the rank of sample i among the negative samples. This indicates the rank of sample i among all negative samples. This refers to the positioning error of the TP. By hyperparameters Sure.

[0043] The present invention also discloses an underwater target detection system based on dynamic sensing area routing, including an input preprocessing module, a feature extraction module, a feature fusion module and a prediction module;

[0044] The input preprocessing module is used to perform data augmentation and enhancement operations on the original input image data using Mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling.

[0045] Both the feature extraction module and the feature fusion module include a multi-residual module based on dynamic feature enhancement. The multi-residual module based on dynamic feature enhancement is used to retain target information of various sizes in the sonar image while acquiring multi-gradient feature information and accelerating feature extraction efficiency.

[0046] The feature fusion module also includes a spatial pyramid fusion structure SPPF and a regional routing attention mechanism, wherein the spatial pyramid fusion structure SPPF is used to adaptively fuse multi-scale feature information from different feature layers;

[0047] The region routing attention mechanism is used to dynamically perceive and enhance foreground target features;

[0048] The prediction module is used to predict the target based on the output of the feature fusion module.

[0049] (III) Beneficial Effects

[0050] This invention provides an underwater target detection method and system based on dynamic sensing area routing. It has the following beneficial effects:

[0051] (1) By setting up a multi-residual module based on dynamic feature enhancement, the present invention uses different dilated convolutions for targets of different sizes, so as to quickly obtain gradient flow information while retaining target information of various sizes in the sonar image, especially the feature information of small targets.

[0052] (2) This invention uses a regional routing attention mechanism to dynamically perceive and enhance the features of the foreground target, prevents interference from background information in the multi-dimensional mixed feature information extracted during the feature extraction stage, and increases the accuracy of target regression detection.

[0053] (3) This invention achieves a balance between classification and localization in the target detection process by using a joint loss function based on weighted sequence localization and classification, and implements high-quality localization for high-precision classification, effectively solving the problems of overfitting of some categories and loss of features of small target samples caused by unbalanced samples. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the process of the present invention;

[0055] Figure 2 This is a schematic diagram of the model architecture of the present invention;

[0056] Figure 3 This is a schematic diagram of the multi-residual module architecture based on dynamic feature enhancement of the present invention;

[0057] Figure 4 This is a schematic diagram of the operation process of the dynamic dilated convolution module of the present invention;

[0058] Figure 5 This is a schematic diagram of the operation process of the regional routing attention mechanism of the present invention;

[0059] Figure 6 This is a table of mAP values ​​for the detection box at different IoUs in an embodiment of the present invention;

[0060] Figure 7 This is a table of mAP for different categories of targets in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0062] Please see Figure 1-7 The present invention provides the following technical solution: an underwater target detection method based on dynamic sensing area routing, specifically including the following steps:

[0063] Dataset preparation:

[0064] The public dataset UATD used consists of 7,600 sonar images captured by Pengcheng Lab using multibeam forward-looking sonar in lakes and shallow waters. The dataset mainly includes 10 object categories: Human Body, Ball, Circle Cage, Square Cage, Tyre, Metal, Bucket, Cube, Cyclinder, Plane, and Rov.

[0065] Data preprocessing:

[0066] The 10 categories in UATD are divided into training set, validation set and test set, with a division ratio of 7:1:2;

[0067] Model building:

[0068] A non-equilibrium underwater sonar image target detection model based on a dynamic sensing feature enhancement network was constructed. The activation function of the multi-gradient flow global feature enhancement network was set as FReLU, the loss function parameters of the multi-gradient flow global feature enhancement network were set as well as the training parameters of the multi-gradient flow global feature enhancement network. The adaptive moment estimation optimizer Adam was used to suppress sample noise interference. The first-order momentum factor, second-order momentum factor and ambiguity factor of Adam were set to 0.9, 0.999 and 0.0000001, respectively.

[0069] To accelerate the training process, the initial learning rate is set to 0.001, and the learning rate during training is the initial learning rate multiplied by a weight decay coefficient of 0.0005. The batch size is set to 330, and the classification loss function and localization loss function are set to a joint localization and classification loss function based on the weight sequence.

[0070] Model training:

[0071] S1. Input Preprocessing: The training set is input into the input preprocessing module, where Mosaic image augmentation, adaptive anchor box calculation, and adaptive image scaling are used to augment and enhance the dataset.

[0072] S2, Feature Extraction: The output of S1 is input into the feature extraction module. After convolution processing and the cooperation of the multi-residual module based on dynamic feature enhancement, the feature information of the first gradient, the second gradient and the third gradient are obtained from the output of S1 in sequence.

[0073] S3, Feature Fusion: The feature information of the third gradient obtained in S2 is input into the Spatial Pyramid Fusion Structure (SPPF) to obtain a fused feature layer. After convolution, a region routing attention mechanism is introduced to upsample the fused feature layer to obtain a first-level feature layer. After processing the first-level feature layer by the dynamic feature enhancement multi-residual module, it is passed together with the feature information of the second gradient in S2 to the next-level dynamic feature enhancement multi-residual module. After convolution and upsampling operations, a second-level feature layer is obtained. The second-level feature layer and the feature information of the first gradient in S2 are input into the subsequent dynamic feature enhancement multi-residual module to obtain a first-level gradient output feature layer. After convolution, the first-level gradient output feature layer is concatenated with the second-level feature layer without upsampling and then input into the subsequent dynamic feature enhancement multi-residual module to obtain a second-level gradient output feature layer. After convolution, the second-level gradient output feature layer is concatenated with the fused feature layer without the region routing attention mechanism and then input into the subsequent dynamic feature enhancement multi-residual module to obtain a third-level gradient output feature layer.

[0074] S4. Prediction: The features from the first gradient output feature layer, the second gradient output feature layer, and the third gradient output feature layer obtained in S3 are input into the prediction module. Anchor boxes are used to predict the position and size of the bounding box of the target. At the same time, the softmax function is used to calculate the class probability for the prediction result corresponding to each anchor box.

[0075] After training on the training and validation sets of the UATD dataset for 330 epochs, it can be seen that the model gradually stabilizes after 100 epochs, converges quickly, and has good overall stability as the epochs progress. This effectively solves the problem of overfitting in some underwater sonar image target detection models due to insufficient sample size and imbalance of samples in different categories.

[0076] The model weights converged as described above were used to test the test set in the UATD dataset. The mAP detection results of the proposed model and the YOLOv5 and YOLOv8 object detection models on the UATD dataset when different IoU values ​​were used for the detection boxes are shown in the appendix. Figure 6 As shown.

[0077] As can be seen, when the IoU threshold of the detection box is set to 0.5, the improved model proposed in this invention improves the mAP value by 2.4% compared to the YOLOv8 model, and by 2.7% compared to the baseline YOLOv5 model;

[0078] When the IoU threshold of the detection box is in the range of [0.5, 0.95], the improved model proposed in this invention improves upon the baseline model YOLOv5 by 3.9%.

[0079] It can be seen that the improved target detection model proposed in this invention outperforms existing mainstream target detection models on the underwater sonar image dataset.

[0080] To further verify the performance of this model, the AP values ​​of this invention were compared with those of YOLOv5 and YOLOv8 object detection models for different categories. Detailed experimental results are attached. Figure 7 As shown.

[0081] As can be seen, the AP values ​​of the proposed model in this invention exceed those of the two baseline models in all 10 categories, with an AP improvement of at least 1% in each category. Meanwhile, the AP values ​​of Square Cage, Metal Bucket, Cyclinder, and Rov are improved by 3.4%, 9.2%, 4.6%, and 3% respectively compared to the original YOLOv5 model. This is mainly due to the adaptive feature enhancement based on the target scale in the feature extraction stage and the dynamic attention and neglect of target and background information before feature fusion.

Claims

1. An underwater target detection method based on dynamic sensing area routing, The feature includes the following steps: S1. Input preprocessing: Mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling are used to augment and enhance the original input image data. S2, Feature Extraction: The output of S1 is input into the feature extraction module. After convolution processing and the cooperation of the multi-residual module based on dynamic feature enhancement, the feature information of the first gradient, the second gradient and the third gradient are obtained from the output of S1 in sequence. S3, Feature Fusion: The feature information of the third gradient obtained in S2 is input into the Spatial Pyramid Fusion Structure (SPPF) to obtain a fused feature layer. After convolution, a region routing attention mechanism is introduced to upsample the fused feature layer to obtain a first-level feature layer. After processing the first-level feature layer by the dynamic feature enhancement multi-residual module, it is passed together with the feature information of the second gradient in S2 to the next-level dynamic feature enhancement multi-residual module. After convolution and upsampling operations, a second-level feature layer is obtained. The second-level feature layer and the feature information of the first gradient in S2 are input into the subsequent dynamic feature enhancement multi-residual module to obtain a first-level gradient output feature layer. After convolution, the first-level gradient output feature layer is concatenated with the second-level feature layer without upsampling and then input into the subsequent dynamic feature enhancement multi-residual module to obtain a second-level gradient output feature layer. After convolution, the second-level gradient output feature layer is concatenated with the fused feature layer without the region routing attention mechanism and then input into the subsequent dynamic feature enhancement multi-residual module to obtain a third-level gradient output feature layer. S4. Prediction: The features from the first gradient output feature layer, the second gradient output feature layer, and the third gradient output feature layer obtained in S3 are input into the prediction module. Anchor boxes are used to predict the position and size of the bounding box of the target. At the same time, the softmax function is used to calculate the class probability for the prediction result corresponding to each anchor box.

2. The underwater target detection method based on dynamic sensing area routing according to claim 1, characterized in that: The processing methods of the multi-residual module based on dynamic feature enhancement in S2 and S3 are as follows: A1. The feature layer input from the upper layer is first processed by a 1×1 convolution and a residual module with a dynamic dilated convolution module to further filter the features extracted from the previous feature layer, retaining multi-size target features. B1. Change the channel dimension by performing a 1×1 convolution on the feature layer input from the upper layer; C1. Concatenate the features of the output structures of A1 and B1 along the channel dimension, and use a 1×1 convolution to change the channel dimension.

3. The underwater target detection method based on dynamic sensing area routing according to claim 2, characterized in that: The dynamic dilated convolution module is used as follows: A2. Enhance the original feature information through a global average pooling residual layer; B2. Full-dimensional target feature filtering is performed through dynamic dilated convolutional layers. The detailed calculation process is as follows: When using The formula for the dynamic dilated convolution module operation, which takes x as input, weights w, a dilated convolution rate of r, and outputs y, is as follows: ； Where r is a hyperparameter of the dynamically dilated convolution module. These are trainable parameters. It is implemented by an average pooling operation with a kernel size of 5 and a convolution operation with a kernel size of 1, where S is a 0-1 probability matrix; C2. The feature information of each size target after screening is further enhanced by the global average pooling residual layer.

4. The underwater target detection method based on dynamic sensing area routing according to claim 1, characterized in that: The feature information of the third gradient is input into the Spatial Pyramid Fusion Structure (SPPF). After passing through a convolutional layer and three global max pooling layers, the four outputs are then concatenated along the channel dimension.

5. The underwater target detection method based on dynamic sensing area routing according to claim 1, characterized in that: The area routing attention mechanism in S3 specifically includes the following three parts: The first part is region partitioning and input projection. For a feature map of H×W×C, it is divided into S×S non-overlapping regions, and then the projections of query Q, key K, and value V are derived as follows: ； in , , These are the projection weights for the query, key, and value, respectively. The second part involves region-to-region routing with directed graphs. This is achieved by constructing directed graphs to identify participation relationships. Specifically, region-level routing is derived by applying the average value of each region to Q and K respectively. and Then through and Matrix multiplication between regions to derive the adjacency matrix between regions. : ； The third part retains the k connections with the highest similarity to other regions for each region. The matrix composed of all these regions and their indices is as follows: ； For each region, the key K and value V are gathered together, and token-to-token attention is used to obtain the final output of the region routing attention mechanism: 。 6. The underwater target detection method based on dynamic sensing area routing according to claim 1, characterized in that: The first gradient output feature layer, the second gradient output feature layer, and the third gradient output feature layer in S4 correspond to prediction boxes of different scales. Each prediction box includes a confidence score, class probability, bounding box position, and size.

7. The underwater target detection method based on dynamic sensing area routing according to claim 1, characterized in that: After S4 performs result prediction, it uses a joint loss function based on weighted sequences for localization and classification as the classification loss and localization loss. The derivation of the loss function is as follows: ； ； ； Where P is the set of positive samples, i.e., TP, and N is the set of negative samples. This indicates the rank of sample i among the negative samples. This indicates the rank of sample i among all negative samples. This refers to the positioning error of the TP. By hyperparameters Sure.

8. An underwater target detection system based on dynamic sensing area routing, characterized in that: It includes an input preprocessing module, a feature extraction module, a feature fusion module, and a prediction module; The input preprocessing module is used to perform data augmentation and enhancement operations on the original input image data using Mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling. Both the feature extraction module and the feature fusion module include a multi-residual module based on dynamic feature enhancement. The multi-residual module based on dynamic feature enhancement is used to retain target information of various sizes in the sonar image while acquiring multi-gradient feature information and accelerating feature extraction efficiency. The feature fusion module also includes a spatial pyramid fusion structure SPPF and a regional routing attention mechanism, wherein the spatial pyramid fusion structure SPPF is used to adaptively fuse multi-scale feature information from different feature layers; The region routing attention mechanism is used to dynamically perceive and enhance foreground target features; The prediction module is used to predict the target based on the output of the feature fusion module.

Images