A side scan sonar image seabed cable zero sample rapid detection method

Abstract

The application discloses a kind of side-scan sonar image seabed cable zero-sample rapid detection method, belong to seabed cable detection technical field, comprising the following steps: step one, establish neural network model;Step two, data fusion enhancement: to existing public side-scan sonar image data data fusion enhancement, generate experimental data set;Step three, data set annotation: to data set is marked;Step four, model training: neural network is trained;Step five, model test: using measured data to the network model after training is tested;The application, by establishing neural network model, and to existing public side-scan sonar image data data fusion enhancement, generate experimental data set, to data set is marked and neural network is trained, using measured data to the network model after training is tested, this method combines data enhancement method, realizes the zero-sample training and rapid detection of neural network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of submarine cable detection, and particularly relates to a side-scan sonar image submarine cable zero-sample rapid detection method. BACKGROUND

[0002] In modern marine industry, submarine cables undertake important work such as resource transportation and information transmission. At present, side-scan sonar is the most important tool for human beings to perceive the seabed, and accurate detection of cables on the side-scan sonar image is the basis for tracking, maintaining and repairing the cables.

[0003] In the field of sonar image cable detection, the current mainstream method is based on deep learning, such as: 1. DuX, Sun Y, Song Y, Dong L, Zhao X. “Revealing the Potential of Deep Learning for Detecting Submarine Pipelines in Side-Scan Sonar Images: An Investigation of Pre-Training Datasets,” Measurement Science and Technology, vol. 15, no. 19, p. 4873, 2023. Using GoogleNet to automatically detect and identify pipelines or cables from side-scan sonar images. 2. H. Feng, Y. Huang, J. Qiao, Z. Wang, F. Hu, and J. Yu, “Prediction-Based Submarine Cable-Tracking Strategy for Autonomous Underwater Vehicles with Side-Scan Sonar,” Journal of Marine Science and Engineering, vol. 12, no. 10, p. 1725, 2024. Using LSTM network for underwater cable detection and predicting its future state. 3. J. Wang, T. Shan, M. Chandrasekaran, T. Osedach, and B. Englot, “Deep learning for detection and tracking of underwater pipelines using multibeam imaging sonar,” in Proceedings of the IEEE International Conference on Robotics and Automation Workshop, 2019. Using U-Net to realize real-time detection of cables on sonar images.

[0004] However, deep learning-based methods require a large amount of data to fully train the network, and the acquisition of submarine cable side-scan sonar images requires reliance on sea trials for collection, which requires a lot of time and money, so a zero-shot submarine cable detection method for side-scan sonar images is needed to solve the above problems. SUMMARY

[0005] The present application aims to provide a side-scan sonar image submarine cable zero-sample rapid detection method to solve the problems in the background art.

[0006] To achieve the above object, the present application provides the following technical solution: a side-scan sonar image submarine cable zero-sample rapid detection method, comprising the following steps:

[0007] Step one, establish a neural network model;

[0008] Step two, data fusion enhancement: data fusion enhancement is performed on the existing public side-scan sonar image data to generate an experimental data set;

[0009] Step three, data set annotation: annotate the data set;

[0010] Step four, model training: train the neural network;

[0011] Step five, model testing: test the trained network model using measured data.

[0012] As a preferred scheme, in "step one", the network is composed of four Conv layers, three C2f layers, one CBAM (Convolutional Block Attention Module) attention layer and one OBB-Head.

[0013] As a preferred scheme, in "step one", the Conv is an integrated convolution layer, which is composed of a two-dimensional convolution layer (Conv2d), a batch normalization layer and a loss function layer in sequence, which can extract features from the image and provide nonlinear mapping, wherein the two-dimensional convolution layer has a convolution kernel size of 3x3, a step of 2 and 0 padding with a padding width of 1.

[0014] As a preferred scheme, in "step one", the activation function uses SiLU activation function, and its mathematical expression is: .

[0015] As a preferred scheme, in "step one", the C2f layer is subjected to a channel dimension transformation of the input feature map through a Conv layer, the Split layer divides the channels of the feature map into two parts, one part is directly output, and the other part continues to enter multiple bottleneck structures (Bottleneck) for further feature extraction, and finally all channels are spliced in the Concat layer and integrated and output through the Conv layer.

[0016] As a preferred scheme, in "step one", the CBAM layer comprises two parts of channel attention (Channel Attention) and spatial attention (Spatial Attention), the channel attention performs maximum pooling and average pooling on each channel of the feature map to obtain two one-dimensional vectors, then the two one-dimensional vectors are input into a multilayer perceptron (MLP) to be added element by element to obtain a one-dimensional vector, and finally the one-dimensional vector is output after an activation function, and the number of elements of the final one-dimensional vector is the same as the number of channels of the feature map input into the CBAM.

[0017] The spatial attention performs maximum pooling and average pooling on elements at each spatial position of all channels of the feature map to obtain two two-dimensional tensors, and then a two-dimensional tensor is obtained through convolution and an activation function.

[0018] As a preferred scheme, in "step one", the OBB-Head is a decoupled detection head, which is divided into three branches, each branch is composed of several consecutive convolution layers, and the output of the feature map after being processed by the OBB-Head includes the probability of the cable, the size, angle and position information of the bounding box, which are used to calculate the classification (Classification, CLS) loss, the Probabilistic Intersection-over-Union (ProbIoU) loss and the Distribution Focal Loss (DFL).

[0019] As a preferred scheme, in "step one", the three-dimensional tensor input and output by the first to eighth layers of the network is: width x height x channel number in each dimension, and the ninth layer outputs a three-dimensional tensor with a size of 66x40x40, wherein the first dimension includes the probability that the target is a cable and the size, angle and position information of the detection box, and the last two dimensions are the feature map size.

[0020] As a preferred scheme, in "step two", the existing public side-scan sonar image data is fused and enhanced: there are a large number of public data sets of side-scan sonar seabed terrain images, a seabed cable side-scan sonar image is intercepted, rotated and smoothed to obtain a side-scan sonar cable image with smooth edges, and is fused with a public side-scan sonar seabed terrain image to obtain more simulated seabed cable side-scan sonar images similar to the actual measured images.

[0021] As a preferred scheme, in "step three", the data set is labeled with a rotated box: X-AnyLabeling is used to label the self-made data set, and the format of the data label after labeling is: Wherein, Class is the target category, x1, y1 is the right upper corner coordinate of the marking box, x2, y2 is the right lower corner coordinate of the marking box, x3, y3 is the left upper corner coordinate of the marking box, and x4, y4 is the left lower corner coordinate of the marking box.

[0022] Compared with the prior art, the beneficial effects of the present application are:

[0023] The present application, by establishing a neural network model, and carrying out data fusion enhancement on the existing public side-scan sonar image data, generating experimental data set, marking the data set and training the neural network, using the measured data to test the trained network model, this method combines data enhancement method, realizes the zero sample training and rapid detection of neural network. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 It is the whole process schematic diagram of the present application;

[0025] Figure 2 It is the network overall structure schematic diagram of the present application;

[0026] Figure 3 It is the network each part internal structure schematic diagram of the present application;

[0027] Figure 4 It is the SiLU activation function schematic diagram of the present application;

[0028] Figure 5 It is the CBAM internal structure schematic diagram of the present application;

[0029] Figure 6 It is the network each layer input and output data size diagram of the present application;

[0030] Figure 7 It is the data fusion enhancement process schematic diagram of the present application;

[0031] Figure 8 It is the marking data example diagram of the present application;

[0032] Figure 9 It is the loss function convergence condition schematic diagram in the training process of the present application;

[0033] Figure 10 It is the detection effect comparison diagram of different models on the measured data of the present application;

[0034] Figure 11 It is the index comparison diagram of different models on the verification set of the present application. DETAILED DESCRIPTION

[0035] The present application will be further described below in combination with embodiments.

[0036] The following examples are used to illustrate the present application, but cannot be used to limit the scope of protection of the present application. The conditions in the examples can be further adjusted according to specific conditions, and simple improvements to the method of the present application under the concept of the present application are within the scope of the present application.

[0037] Referring to Figures 1-11 , the present application provides a side-scan sonar image seabed cable zero-sample rapid detection method, comprising the following steps:

[0038] Step one, establish a neural network model:

[0039] As shown in Figure 2 and Figure 3 , the network is composed of four Conv layers, three C2f layers, one CBAM (Convolutional Block Attention Module) attention layer and one OBB-Head, Conv is an integrated convolution layer, which is composed of a two-dimensional convolution layer (Conv2d), a batch normalization layer and a loss function layer in turn, which can extract features from images and provide nonlinear mapping;

[0040] Among them, the two-dimensional convolution layer has a convolution kernel size of 3x3, a step of 2, and is filled with 0 with a padding width of 1;

[0041] The activation function uses the SiLU activation function, and its mathematical expression is: ;

[0042] The C2f layer first passes through a Conv layer (built-in convolution kernel size 1x1, does not change the feature map size) to transform the channel dimension of the input feature map, the Split layer divides the channels of the feature map into two parts, one part is directly output, and the other part continues to enter multiple bottleneck structures (Bottleneck) for further feature extraction, and finally all channels are spliced in the Concat layer and integrated and output through the Conv layer. The CBAM layer includes channel attention (Channel Attention) and spatial attention (Spatial Attention) two parts;

[0043] Among them, the channel attention first performs maximum pooling and average pooling on each channel of the feature map to obtain two one-dimensional vectors, then inputs a multilayer perceptron (Multilayer Perceptron, MLP), element-wise addition to obtain a one-dimensional vector, and finally outputs after the activation function. The number of elements in the one-dimensional vector obtained is the same as the number of channels of the feature map input to the CBAM, which is equivalent to assigning a weight to each channel;

[0044] The spatial attention first performs maximum pooling and average pooling on the elements of each spatial position of all channels of the feature map to obtain two two-dimensional tensors, and then performs convolution and an activation function to obtain a two-dimensional tensor, the size of which is the same as that of the feature map, which is equivalent to assigning a weight to each spatial position of all channels of the feature map;

[0045] The OBB-Head is divided into three branches, each branch being composed of several continuous convolution layers. The feature map after being processed by the OBB-Head is used to calculate a loss function. The three loss functions are a classification (CLS) loss, a Probabilistic Intersection-over-Union (ProbIoU) loss, and a Distribution Focal Loss (DFL).

[0046] The input and output feature map sizes of each layer of the network are shown in the table. The three-dimensional tensor of the input and output of the first to eighth layers has three dimensions: width x height x channel number. The output of the ninth layer is a three-dimensional tensor with a size of 66 x 40 x 40. The first dimension contains the probability that the target is a cable as well as the size, angle, and position information of the detection box. The last two dimensions are the feature map sizes.

[0047] Step two, data fusion enhancement: existing public side-scan sonar image data is enhanced through data fusion to generate an experimental data set.

[0048] There are a large number of public data sets of side-scan sonar seabed terrain images, but there are currently no public data sets of submarine cable side-scan sonar images, and the collection requires a high cost. We cut, rotate, and smooth a submarine cable side-scan sonar image in our hands to obtain a side-scan sonar cable image with smooth edges, and then fuse it with a public side-scan sonar seabed terrain image to obtain more simulated submarine cable side-scan sonar images similar to the actual measured images. The data fusion enhancement process is shown in FIG. 4. Using this method, 789 simulated submarine cable side-scan sonar images are generated, of which 671 are used for network training and 118 are used for network verification. In order to make the results more convincing, the test set uses actual measured submarine cable side-scan sonar images.

[0049] Step three, data set labeling: label the data set.

[0050] The self-made data set is labeled using X-AnyLabeling. The format of the data label after labeling is:

[0051] ;

[0052] The meanings are as follows: Class is the target category, x1, y1 are the coordinates of the upper right corner of the label box, x2, y2 are the coordinates of the lower right corner of the label box, x3, y3 are the coordinates of the upper left corner of the label box, and x4, y4 are the coordinates of the lower left corner of the label box. The figure is the labeled image and label, wherein the label coordinates are normalized;

[0053] Step four, model training: training the neural network;

[0054] In the zero-shot training side-scan sonar image cable detection task, better results are achieved compared with existing methods;

[0055] Step five, model testing: testing the trained network model by using the measured data;

[0056] Compared with the current more advanced Yolov8-obb series model, the model proposed by the method has a higher cable completeness than any model in the Yolov8-obb series;

[0057] In terms of training time, our model only takes 510.68 seconds, which is 38.73% shorter than the minimum time of 833.5 seconds of the Yolov8-obb series model;

[0058] In terms of single image detection time, our model only takes 0.06 seconds, which is 17.81% shorter than the minimum time of 0.073 seconds of the Yolov8-obb series model;

[0059] In terms of mAP50~95, our model reaches 0.795, which is 16.4% higher than the highest score of 0.683 of the Yolov8-obb series model.

[0060] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A side-scan sonar image seabed cable zero-sample quick detection method, characterized in that, Comprise the following steps: Step one, the neural network model is established, and the neural network model is composed of four Conv layers, three C2f layers, one CBAM attention layer and one OBB-Head; Conv is an integrated convolution layer, which is composed of a two-dimensional convolution layer, a batch normalization layer and a loss function layer in turn, which can extract features from images and provide nonlinear mapping; the C2f layer includes Conv, Split layer, Bottleneck layer and Concat layer; Step two, data fusion enhancement: the existing public side scan sonar image data is fused and enhanced to generate experimental data set, specifically, a single submarine cable side scan sonar image is intercepted, rotated and smoothed to obtain a side scan sonar cable image with smooth edge, and then fused with the public side scan sonar submarine topographic map to obtain more simulated submarine cable side scan sonar images similar to the measured images; Step three, data set labeling: use X-AnyLabeling to rotate frame labeling on self-made data set, and the data label format after labeling is: wherein, Class is the target category, x1, y1 is the right upper corner coordinate of the labeling frame, x2, y2 is the right lower corner coordinate of the labeling frame, x3, y3 is the left upper corner coordinate of the labeling frame, and x4, y4 is the left lower corner coordinate of the labeling frame. Step four, model training: the neural network is trained; Step five, model test: the measured data is used to test the trained network model.

2. The method of claim 1, wherein: In "step one", the C2f layer is transformed in channel dimension through a Conv layer, the Split layer divides the channel of the feature map into two parts, one part is directly output, and the other part continues to enter multiple bottleneck structures for further feature extraction, and finally all channels are spliced in the Concat layer and integrated and output through the Conv layer.

3. The method of claim 2, wherein: In "step one", the CBAM layer includes channel attention and spatial attention, the channel attention performs maximum and average pooling on each channel of the feature map to obtain two one-dimensional vectors, which are then input into a multilayer perceptron to be added element by element to obtain a one-dimensional vector, which is output after activation function, and finally the number of elements in the one-dimensional vector is the same as the number of channels of the input CBAM feature map; The spatial attention performs maximum and average pooling on the elements of each spatial position of all channels of the feature map to obtain two two-dimensional tensors, which are then convolved and activated to obtain a two-dimensional tensor.

4. The method of claim 2, wherein: In "step one", the OBB-Head is divided into three branches, each branch is composed of several consecutive convolution layers, and the feature map is processed by each branch of the OBB-Head and then merged in the channel dimension to calculate the loss function; the three loss functions used are classification loss, joint intersection over union loss and distribution focal loss.

5. The method of claim 4, wherein: In "step one", the three-dimensional tensor input and output of the network 1~8 layers is: width x height x channel number, and the output of the 9th layer is a three-dimensional tensor with a size of 66x40x40, wherein the first dimension contains the probability that the target is a cable and the size, angle and position information of the detection box, and the last two dimensions are the feature map size.

Images