Side-scan sonar image submarine cable zero sample rapid detection method

Abstract

The invention discloses a side-scan sonar image submarine cable zero sample rapid detection method, and belongs to the technical field of submarine cable detection, and the method comprises the following steps: 1, building a neural network model; step 2, data fusion enhancement: carrying out data fusion enhancement on the existing public side-scan sonar image data to generate a data set for an experiment; step 3, data set labeling: labeling the data set; 4, model training: training the neural network; step 5, model testing: testing the trained network model by using actual measurement data; according to the method, the neural network model is established, data fusion enhancement is performed on the existing open side-scan sonar image data, the data set for experiment is generated, the data set is labeled, the neural network is trained, and the trained network model is tested by using the actually measured data. And zero sample training and rapid detection of the neural network are realized.

Description

Technical Field

[0001] The invention belongs to the technical field of submarine cable detection, and in particular relates to a zero-sample rapid detection method for submarine cables using side-scan sonar images. Background Art

[0002] In modern marine affairs, submarine cables undertake important tasks such as resource transportation and information transmission. At present, side-scan sonar is the most important tool for humans to perceive the seabed, and accurate detection of cables on side-scan sonar images is the basis for tracking, maintaining and repairing them.

[0003] In the field of sonar image cable detection, the current mainstream method is based on deep learning, such as: 1. DuX, Sun Y, Song Y, Dong L, Zhao X. "Revealing the Potential of Deep Learningfor Detecting Submarine Pipelines in Side-Scan Sonar Images: An Investigationof Pre-Training Datasets," Measurement Science and Technology, vol. 15, no.19, p. 4873, 2023. Automatic detection and identification of pipelines or cables from side-scan sonar images using GoogleNet. 2. H. Feng, Y. Huang, J. Qiao, Z. Wang, F. Hu, and J. Yu, "Prediction-BasedSubmarine Cable-Tracking Strategy for Autonomous Underwater Vehicles withSide-Scan Sonar," Journal of Marine Science and Engineering, vol. 12, no. 10, p. 1725, 2024. Use LSTM network to detect underwater cables and predict their future status. 3. J. Wang, T.Shan, M. Chandrasekaran, T. Osedach, and B. Englot, "Deep learning for detection and tracking of underwater pipelines using multibeam imagingsonar," in Proceedings of the IEEE International Conference on Robotics andAutomation Workshop, 2019. Real-time detection of cables on sonar images was achieved using U-Net.

[0004] However, deep learning-based methods require a large amount of data to fully train the network, and the acquisition of side-scan sonar images of submarine cables requires collection through offshore trials, which takes a lot of time and money. Therefore, a zero-sample rapid detection method for submarine cables in side-scan sonar images is needed to solve the above problems. Summary of the invention

[0005] The object of the present invention is to provide a method for rapid zero-sample detection of submarine cables in side-scan sonar images to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a method for rapid zero-sample detection of submarine cables in side-scan sonar images, comprising the following steps: Step 1: Establish a neural network model; Step 2: Data fusion enhancement: Perform data fusion enhancement on the existing public side-scan sonar image data to generate an experimental data set; Step 3: Dataset annotation: Annotate the dataset; Step 4: Model training: training the neural network; Step 5: Model testing: Use measured data to test the trained network model.

[0007] As a preferred solution, in "Step 1", the network consists of four Conv layers, three C2f layers, a CBAM (Convolutional Block Attention Module) attention layer and an OBB-Head.

[0008] As a preferred solution, in "Step 1", Conv is an integrated convolution layer, which is composed of a two-dimensional convolution layer (Conv2d), a batch normalization layer and a loss function layer in sequence. It can extract features from images and provide nonlinear mapping. The convolution kernel size of the two-dimensional convolution layer is 3×3, the step size is 2, and it is padded with 0 and the padding width is 1.

[0009] As a preferred solution, in "Step 1", the activation function uses the SiLU activation function, and its mathematical expression is: .

[0010] As a preferred solution, in "Step 1", the C2f layer transforms the channel dimension of the input feature map through a Conv layer, and the Split layer divides the channel of the feature map into two parts, one part is directly output, and the other part continues to enter multiple bottleneck structures (Bottleneck) for further feature extraction. Finally, all channels are spliced ​​in the Concat layer and then integrated and output through the Conv layer.

[0011] As a preferred solution, in "Step 1", the CBAM layer includes two parts: Channel Attention and Spatial Attention. Channel Attention performs maximum pooling and average pooling on each channel of the feature map to obtain two one-dimensional vectors, which are then input into a multilayer perceptron (MLP) to add each element one-dimensionally and then output after the activation function. The number of elements in the final one-dimensional vector is the same as the number of channels of the feature map input into the CBAM. Spatial attention performs maximum pooling and average pooling on the elements of each spatial position of all channels of the feature map to obtain two two-dimensional tensors, which are then convolved and activated to obtain a two-dimensional tensor.

[0012] As a preferred solution, in "Step 1", OBB-Head is a decoupled detection head, which is divided into three branches. Each branch consists of several consecutive convolutional layers. The output of the feature map after OBB-Head processing contains the probability, bounding box size, angle and position information of the cable, which is used to calculate the classification (CLS) loss, probabilistic intersection-over-union (ProbIoU) loss and distribution focal loss (DFL).

[0013] As a preferred solution, in "Step 1", each dimension of the three-dimensional tensors input and output by the 1st to 8th layers of the network is: width × height × number of channels, and the 9th layer outputs a three-dimensional tensor of size 66 × 40 × 40, where the first dimension contains the probability that the target is a cable and the size, angle and position information of the detection box, and the last two dimensions are the feature map sizes.

[0014] As a preferred solution, in "Step 2", data fusion enhancement is performed on the existing public side-scan sonar image data: there are a large number of public data sets of side-scan sonar seabed topography images. A side-scan sonar image of a submarine cable is intercepted, rotated and smoothed to obtain a side-scan sonar cable map with smooth edges, which is fused with the public side-scan sonar seabed topography map to obtain more simulated submarine cable side-scan sonar images that are similar to the actual measured images.

[0015] As a preferred solution, in "Step 3", the data set is labeled with a spin box: X-AnyLabeling is used to label the self-made data set. The data label format after labeling is: , where Class is the target category, x1 and y1 are the coordinates of the upper right corner of the annotation box, x2 and y2 are the coordinates of the lower right corner of the annotation box, x3 and y3 are the coordinates of the upper left corner of the annotation box, and x4 and y4 are the coordinates of the lower left corner of the annotation box.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention establishes a neural network model and performs data fusion enhancement on existing public side-scan sonar image data to generate an experimental data set, annotates the data set and trains the neural network, and uses measured data to test the trained network model. This method combines the data enhancement method to achieve zero-sample training and rapid detection of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the overall structure of the network of the present invention; Figure 3 It is a schematic diagram of the internal structure of each part of the network of the present invention; Figure 4 Schematic diagram of SiLU activation function of the present invention; Figure 5 This is a schematic diagram of the internal structure of the CBAM of the present invention; Figure 6 A size diagram of the input and output data of each layer of the network of the present invention; Figure 7 It is a schematic diagram of the data fusion enhancement process of the present invention; Figure 8 This is an example diagram of the data labeled by the present invention; Fig. 9 It is a schematic diagram of the convergence of the loss function during the training process of the present invention; Fig.10 It is a comparison chart of the detection effects of different models of the present invention on measured data; Fig.11 This is a comparison chart of indicators of different models of the present invention on the validation set. DETAILED DESCRIPTION

[0018] The present invention will be further described below in conjunction with the embodiments.

[0019] The following examples are used to illustrate the present invention, but they cannot be used to limit the scope of protection of the present invention. The conditions in the examples can be further adjusted according to specific conditions. Simple improvements to the method of the present invention under the premise of the concept of the present invention belong to the scope of protection claimed in the present invention.

[0020] See also Figure 1-11The present invention provides a method for rapid zero-sample detection of submarine cables in side-scan sonar images, comprising the following steps: Step 1: Build a neural network model: like Figure 2 and Figure 3 As shown in the figure, the network consists of four Conv layers, three C2f layers, a CBAM (Convolutional Block Attention Module) attention layer and an OBB-Head. Conv is an integrated convolution layer, which is composed of a two-dimensional convolution layer (Conv2d), a batch normalization layer and a loss function layer in sequence. It can extract features from images and provide nonlinear mapping. Among them, the convolution kernel size of the two-dimensional convolution layer is 3×3, the step size is 2, and it is padded with 0 and the padding width is 1; The activation function uses the SiLU activation function, whose mathematical expression is: ; The C2f layer first passes through a Conv layer (with a built-in convolution kernel size of 1×1, which does not change the size of the feature map) to transform the channel dimension of the input feature map. The Split layer divides the channel of the feature map into two parts, one of which is directly output, and the other continues to enter multiple bottleneck structures (Bottleneck) for further feature extraction. Finally, all channels are spliced ​​in the Concat layer and then integrated and output through the Conv layer. The CBAM layer consists of two parts: channel attention and spatial attention; Among them, channel attention first performs maximum pooling and average pooling on each channel of the feature map to obtain two one-dimensional vectors, which are then input into a multilayer perceptron (MLP) and added element by element to obtain a one-dimensional vector. After the activation function is passed, the output is finally obtained. The number of elements in the one-dimensional vector is the same as the number of channels of the feature map input to CBAM, which is equivalent to assigning a weight to each channel. Spatial attention first performs maximum pooling and average pooling on the elements of each spatial position of all channels in the feature map to obtain two two-dimensional tensors, and then obtains a two-dimensional tensor after convolution and activation function. The size of the tensor is the same as the size of the feature map, which is equivalent to assigning a weight to each spatial position of all channels in the feature map. OBB-Head is divided into three branches, each of which consists of several consecutive convolutional layers. The feature map is processed by OBB-Head and used to calculate the loss function. The three loss functions are classification (CLS) loss, probabilistic intersection-over-union (ProbIoU) loss and distribution focal loss (DFL).

[0021] The input and output of each layer of the network and the output feature map size are shown in the table, where the 3D tensors of the 1st to 8th layers have the dimensions of width × height × number of channels. The 9th layer outputs a 3D tensor of size 66 × 40 × 40, where the first dimension contains the probability that the target is a cable and the size, angle, and position information of the detection box, and the last two dimensions are the feature map sizes; Step 2: Data fusion enhancement: Perform data fusion enhancement on the existing public side-scan sonar image data to generate an experimental data set; There are a large number of public data sets for side-scan sonar seabed topography images, but there is currently no public data set for side-scan sonar images of submarine cables, and the acquisition requires a high cost. We intercepted, rotated and smoothed a side-scan sonar image of a submarine cable to obtain a side-scan sonar cable map with smooth edges, and then fused it with the public side-scan sonar seabed topography map to obtain more simulated submarine cable side-scan sonar images that are similar to the measured images. The data fusion enhancement process is shown in Figure 4. Using this method, 789 simulated submarine cable side-scan sonar images were generated, of which 671 were used for network training and 118 were used for network verification. In order to make the results more convincing, the test set uses measured submarine cable side-scan sonar images; Step 3: Dataset annotation: Annotate the dataset; Use X-AnyLabeling to label the self-made data set. The data label format after labeling is: ; The meanings are: Class is the target category, x1, y1 are the coordinates of the upper right corner of the annotation box, x2, y2 are the coordinates of the lower right corner of the annotation box, x3, y3 are the coordinates of the upper left corner of the annotation box, and x4, y4 are the coordinates of the lower left corner of the annotation box. The figure shows the annotated image and label, where the label coordinates are normalized; Step 4: Model training: training the neural network; In the zero-shot training side-scan sonar image cable detection task, better results were achieved compared with existing methods; Step 5: Model testing: Use measured data to test the trained network model; This method is compared with the currently more advanced Yolov8-obb ​​series models. The cable integrity detected by the model proposed by this method is stronger than any model in the Yolov8-obb ​​series. In terms of training time, our model only takes 510.68 seconds, which is 38.73% shorter than the minimum time of 833.5 seconds of the Yolov8-obb ​​series model; In terms of single image detection time, our model only takes 0.06 seconds, which is 17.81% shorter than the minimum time of 0.073 seconds of the Yolov8-obb ​​series model; In terms of mAP50~95, our model reached 0.795, which is 16.4% higher than the highest score of 0.683 of the Yolov8-obb ​​series models.

[0022] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A zero-sample rapid detection method for submarine cables in side-scan sonar images, characterized in that: The following steps are involved: Step 1: Establish a neural network model; Step 2: Data fusion enhancement: Perform data fusion enhancement on the existing public side-scan sonar image data to generate an experimental data set; Step 3: Dataset annotation: Annotate the dataset; Step 4: Model training: training the neural network; Step 5: Model testing: Use measured data to test the trained network model.

2. The method for rapid zero-sample detection of submarine cables using side-scan sonar images according to claim 1, characterized in that: In "Step 1", the network consists of four Conv layers, three C2f layers, a CBAM (Convolutional BlockAttention Module) attention layer, and an OBB-Head.

3. The method for rapid zero-sample detection of submarine cables using side-scan sonar images according to claim 2, characterized in that: In "Step 1", Conv is an integrated convolutional layer, which consists of a two-dimensional convolutional layer (Conv2d), a batch normalization layer, and a loss function layer in sequence. It can extract features from images and provide nonlinear mapping.

4. The method for rapid zero-sample detection of submarine cables using side-scan sonar images according to claim 3 is characterized by: In "Step 1", the activation function uses the SiLU activation function, whose mathematical expression is: .

5. The method for rapid zero-sample detection of submarine cables in side-scan sonar images according to claim 4 is characterized by: In "Step 1", the C2f layer transforms the channel dimension of the input feature map through a Conv layer. The Split layer divides the channels of the feature map into two parts, one part is directly output, and the other part continues to enter multiple bottleneck structures (Bottleneck) for further feature extraction. Finally, all channels are spliced ​​in the Concat layer and then integrated and output through the Conv layer.

6. The method for rapid zero-sample detection of submarine cables in side-scan sonar images according to claim 5 is characterized by: In "Step 1", the CBAM layer contains two parts: Channel Attention and Spatial Attention. Channel Attention performs maximum pooling and average pooling on each channel of the feature map to obtain two one-dimensional vectors, which are then input into a multilayer perceptron (MLP) to add each element to obtain a one-dimensional vector. After the activation function is passed, the output is finally obtained. The number of elements in the one-dimensional vector is the same as the number of channels of the feature map input into the CBAM. Spatial attention performs maximum pooling and average pooling on the elements of each spatial position of all channels of the feature map to obtain two two-dimensional tensors, which are then convolved and activated to obtain a two-dimensional tensor.

7. The method for rapid zero-sample detection of submarine cables using side-scan sonar images according to claim 6 is characterized by: In "Step 1", OBB-Head is divided into three branches, each of which consists of several consecutive convolutional layers. After being processed by each branch of OBB-Head, the feature maps are merged in the channel dimension to calculate the loss function; the three loss functions used are classification (CLS) loss, probabilistic intersection-over-union (ProbIoU) loss and distribution focal loss (DFL).

8. The method for rapid zero-sample detection of submarine cables using side-scan sonar images according to claim 7, characterized in that: In "Step 1", the three-dimensional tensors input and output by the 1st to 8th layers of the network have dimensions of width × height × number of channels. The 9th layer outputs a three-dimensional tensor of size 66 × 40 × 40, where the first dimension contains the probability that the target is a cable and the size, angle, and position information of the detection box, and the last two dimensions are the feature map sizes.

9. The method for rapid zero-sample detection of submarine cables in side-scan sonar images according to claim 1, characterized in that: In "Step 2", data fusion enhancement is performed on the existing public side-scan sonar image data: there are a large number of public data sets of side-scan sonar seabed topography images. A side-scan sonar image of a submarine cable is intercepted, rotated and smoothed to obtain a side-scan sonar cable map with smooth edges, which is then fused with the public side-scan sonar seabed topography map to obtain more simulated submarine cable side-scan sonar images that are similar to the measured images.

10. The method for rapid zero-sample detection of submarine cables in side-scan sonar images according to claim 1, characterized in that: In "Step 3", the data set is labeled with a spin box: X-AnyLabeling is used to label the self-made data set. The data label format after labeling is: , where Class is the target category, x1 and y1 are the coordinates of the upper right corner of the annotation box, x2 and y2 are the coordinates of the lower right corner of the annotation box, x3 and y3 are the coordinates of the upper left corner of the annotation box, and x4 and y4 are the coordinates of the lower left corner of the annotation box.

Images