Automatic detection method and system for small target sonar image

Abstract

The invention discloses an automatic detection method and system for a small target sonar image, and relates to the technical field of underwater target detection, and the method comprises the steps: collecting a small target sonar image with accurate target information; based on a YOLO11 model, a three-channel cross-dimensional interactive attention mechanism is added to the front end of a detection head, an improved YOLO11 model is generated, model training is carried out through the small target sonar image, a detection model is constructed, and the detection model is used for detecting the underwater small target through the sonar image. The automatic detection performance of the three-channel cross-dimensional interactive-YOLO11 model for the small-target side-scan sonar image is optimal in similar models, and the automatic detection requirement of the small-target sonar image can be met.

Description

Technical Field

[0001] The invention relates to the technical field of underwater target detection, and in particular to an automatic detection method for small target sonar images. Background Art

[0002] At present, underwater unmanned submersibles are developing rapidly, and the demand for search and reconnaissance based on underwater unmanned platforms is increasing. The main search and reconnaissance equipment includes side-scan sonar, synthetic aperture sonar, forward-looking sonar, and optical imaging equipment. In order to give full play to the search and reconnaissance efficiency of unmanned submersibles, the search and reconnaissance equipment carried by them needs to perform automatic detection after obtaining the target image, otherwise it will not be able to carry out further work in an unmanned state. At present, the research on automatic detection of optical images is in-depth, and the detection effect is good, especially in the fields of face recognition and autonomous driving, which has basically met the application requirements. There are few studies on automatic detection of sonar images, and most of the main methods are to directly apply optical image detection models to sonar images, and no further research has been carried out in combination with the characteristics of sonar images and the actual needs of unmanned submersibles.

[0003] At present, the research on automatic detection of sonar images based on the YOLO model mainly faces two problems: First, the sonar images used in the research are mostly large targets such as shipwrecks, aircraft, seabeds, and bottoms, which are not difficult to identify. In addition, most of the acoustic atlases are collected online, and most of them have been flipped, rotated, and cropped to expand the quantity. The quality and authenticity cannot be guaranteed, and the detection accuracy lacks persuasiveness. Second, it is difficult to balance the improvement of the model between improving the detection accuracy and reducing the complexity of the model. After the detection accuracy is improved, the parameters of the model will often increase, and the structure will become bloated. When the model is pruned or distilled, the detection accuracy will often decrease. In response to the above problems, it is of great significance to carry out special offshore tests, fully mine the test data, and study the automatic detection method of small-sample underwater small-target sonar images based on transfer learning. Summary of the invention

[0004] In order to solve the problems of sonar image automatic detection due to the lack of professional and standardized acoustic atlases and systematic and engineered detection models, resulting in detection indicators fluctuating, a variety of detection models and thus difficult to promote and apply, the purpose of the present invention is to propose an automatic detection technology for small target sonar images based on an improved YOLO11, aiming to improve the detection performance of the automatic detection model.

[0005] In order to achieve the above technical objectives, the present application provides an automatic detection method for small target sonar images, comprising the following steps:

[0006] Collect small target sonar images with accurate target information;

[0007] Based on the YOLO11 model, an improved YOLO11 model is generated by adding a three-channel cross-dimensional interactive attention mechanism at the front end of the detection head. The model is trained through small target sonar images to build a detection model for detecting underwater small targets through sonar images.

[0008] Preferably, in the process of improving the YOLO11 model, the three-channel cross-dimensional interactive attention mechanism includes three parallel branches, each branch is designed to process feature maps, focusing on the information of the image in different dimensions of channel, height, and width respectively; in the three branches, the attention weights are calculated in different ways, and finally the outputs of the three branches are averaged and aggregated.

[0009] Preferably, in the process of calculating the attention weights of the three branches in different ways, one of the branches extracts the spatial dimension information through the CBAM mechanism, and the other two branches calculate the attention weights by extracting the interaction features between the spatial dimension and the channel dimension.

[0010] Preferably, in the process of model training, based on the small target sonar image, the target is marked using the labelImg label, and the training set, validation set, and test set are set according to 7:1:2 to construct an acoustic image set for model training.

[0011] The present invention also discloses an automatic detection system for small target sonar images, which is used to implement the above-mentioned automatic detection method for small target sonar images, comprising:

[0012] Data acquisition module, used to collect small target sonar images with accurate target information;

[0013] The data processing module is used to label the small target sonar images using the labelImg label, and to construct the acoustic image set by setting the training set, validation set, and test set in a ratio of 7:1:2;

[0014] The model building module is used to generate an improved YOLO11 model based on the YOLO11 model by adding a three-channel cross-dimensional interactive attention mechanism to the front end of the detection head, train the model through the sound atlas, and build a detection model;

[0015] The monitoring module is used to detect small underwater targets by collecting sonar images based on the detection model.

[0016] Preferably, the three-channel cross-dimensional interactive attention mechanism includes three parallel branches, each of which is designed to process feature maps, focusing on the information of the image in different dimensions of channel, height, and width respectively; among the three branches, one of them extracts spatial dimension information through the CBAM mechanism, and the other two calculate the attention weights by extracting the interactive features between the spatial dimension and the channel dimension, and finally the outputs of the three branches are averaged and aggregated.

[0017] The present invention further discloses an automatic detection device for small target sonar images, comprising:

[0018] A data acquisition unit, arranged at at least one position on the water surface, in the water and under the water, for carrying the automatic detection system for small target sonar images mentioned above;

[0019] A driving unit, used to carry and drive the data acquisition unit, to execute the above-mentioned automatic detection method for small target sonar images, and to obtain the detection result of underwater small targets;

[0020] A positioning unit, used to locate small underwater targets based on the detection results;

[0021] A communication unit is used to establish a data exchange channel between data acquisition units at different locations and transmit the positioning result of the underwater small target to any data acquisition unit;

[0022] A power supply unit is used to supply power.

[0023] Preferably, there are several data acquisition units.

[0024] Preferably, the device also includes a motion unit, which is used to control the data acquisition unit to execute the motion instructions of the driving unit, approach the small underwater target, and perform target detection. When only one data acquisition unit obtains the detection result, a motion instruction is generated based on the detection result and the positioning result to control other data acquisition units to perform target detection on the small underwater target.

[0025] The present invention discloses the following technical effects:

[0026] The three-channel cross-dimensional interaction-YOLO11 model proposed in the present invention has the best automatic detection performance for small target side-scan sonar images among similar models and can meet the needs of automatic detection of small target sonar images. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 It is the overall structure diagram of the three-channel cross-dimensional interactive attention mechanism described in the present invention;

[0029] Figure 2 It is the improved YOLO11 network structure diagram of the present invention;

[0030] Figure 3 It is the Permute1 operation described in the present invention;

[0031] Figure 4 It is the Permute2 operation described in the present invention;

[0032] Figure 5 It is the Permute3 operation described in the present invention;

[0033] Figure 6 This is the Permute4 operation described in the present invention. DETAILED DESCRIPTION

[0034] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0035] like Figure 1-6 As shown, the present invention provides an automatic detection method for small target sonar images, which belongs to an improved YOLO11 detection algorithm to improve the detection performance of the automatic detection model, including the following processes:

[0036] By conducting scanning tests, selecting basic models, and improving model structures, an improved YOLO11 detection algorithm is proposed. The specific implementation process is as follows:

[0037] Step 1: By placing special targets on the seabed and conducting a large number of scanning activities, we can obtain small target sonar images with accurate target information, use labelImg label mapping software to mark the targets, and set the training set, validation set, and test set in a ratio of 7:1:2; construct the acoustic map set;

[0038] Step 2: Use the six classic basic models of the YOLO series to train and detect the set sound atlas, and use 10 indicators such as training time, number of parameters, amount of calculation, and frame rate per second to evaluate the detection performance of each model to verify whether the YOLO11 model is the optimal model;

[0039] Step 3: By adding a three-channel cross-dimensional interactive attention mechanism to the front end of the detection head, the backbone network of the YOLO11 model is transformed, and a three-channel cross-dimensional interactive detection model based on YOLO11 is constructed. Experimental comparison is performed to verify the effectiveness of the model.

[0040] The test is carried out on the corresponding data set, and the detection data is shown in Table 1.

[0041] Table 1

[0042] Model version Time layers parameters GFLOPs FPS P R mAP@0.5 mAP@0.5-0.95 F1 YOLO11n 2.78 238 2582542 6.3 344.8 0.913 0.917 0.950 0.469 0.914 Three-channel cross-dimensional interaction model 2.99 249 2582742 6.3 344.8 0.915 0.920 0.954 0.483 0.917

[0043] As can be seen from the table, from the perspective of lightweight, the new model constructed based on YOLO11n has little difference in the number of parameters, amount of calculation, and frame rate per second. From the perspective of detection accuracy, the performance of the three-channel cross-dimensional interaction model's P, R, mAP@0.5, mAP@0.5-0.95, and F1 indicators exceeds the original model, reaching 0.915, 0.920, 0.954, 0.483, and 0.917 respectively, which are all improved compared with the YOLO11n model.

[0044] Specific meaning of each evaluation index in the table

[0045] (1) time refers to the time taken to complete the training under the experimental environment;

[0046] (2) layers indicates the number of convolutional layers of the model;

[0047] (3) parameters represents the total number of all trainable parameters in the model, indicating the complexity of the model;

[0048] (4) GFLOPs is the number of billion floating-point operations performed by the model, which represents the computational workload of the model;

[0049] (5) FPS is the number of frames per second, which indicates the number of images that can be detected per second;

[0050] (6) P is the precision rate, which refers to the probability that a sample is actually positive among all samples predicted to be positive. In the model, it refers to the ratio of the number of detection frames in which the model correctly detects the target to the number of all detection frames, which is used to measure the accuracy of the detection.

[0051] (7) R is the recall rate, which refers to the probability of a sample being positive among all samples whose true values ​​are positive. In the model, it refers to the ratio of the number of detection boxes that the model correctly detects the target to the number of all targets, which is used to measure the completeness of the detection.

[0052] (8) mAP@0.5 is the area enclosed by plotting precision and recall as the coordinate axes. m represents the average, and the number after @ represents the threshold for determining positive and negative samples. mAP@.5 represents the average mAP of detection accuracy (IoU) greater than 0.5.

[0053] (9) mAP@0.5-0.95 is the area enclosed by plotting precision and recall as the coordinate axes. m represents the average, and the number after @ represents the threshold for determining positive and negative samples. mAP@0.5-0.95 represents the average mAP at different IoU thresholds (from 0.5 to 0.95, with a step size of 0.05), namely 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, and 0.95.

[0054] (10) The F1 value is the harmonic mean of precision and recall, ranging from 0 to 1. The F1 value is usually used as an important indicator for detecting model performance, and is particularly suitable for unbalanced datasets or tasks that require balancing precision and recall.

[0055] The data set used in the present invention is self-built data, which is obtained by towing a side-scan sonar on a surface vessel to scan small targets at sea. In order to improve the scanning efficiency, the side-scan sonar adopts a high- and low-frequency synchronous working mode to synchronously display high- and low-frequency images. At the same time, in order to increase the receptive field, the full-screen image of the waterfall chart is intercepted as the detection image. After standardization, 3120 acoustic images containing targets are obtained, and the targets are marked using labelImg label mapping software to construct an acoustic image set. In order to detect the generalization ability of the model, the training set, validation set, and test set are set according to 7:1:2, among which there are 2182 training sets, 312 validation sets, and 626 test sets. After the improved YOLO11 algorithm is implemented, the detection data is as shown in Table 2.

[0056] Table 2

[0057]

[0058]

[0059] As can be seen from Table 2, from the perspective of lightweight, the 11 new models constructed based on YOLO11 as the basic model have little difference in the number of parameters, calculation amount, and frame rate per second; from the perspective of detection accuracy, the P, R, mAP@0.5, mAP@0.5-0.95, and F1 performance of the three-channel cross-dimensional interaction-YOLO11n are all the best, reaching 0.915, 0.920, 0.954, 0.483, and 0.917 respectively. Moreover, on the basis of the three-channel cross-dimensional interaction-YOLO11 structure, the AFGC attention mechanism is further added, and it is found that the detection performance has decreased, proving that the model modification is not the more attention layers added, the better. In summary, the automatic detection performance of the three-channel cross-dimensional interaction-YOLO11 model for small target side scan sonar images is the best among similar models, which can meet the needs of automatic detection of small target sonar images, verifying the effectiveness of this method.

[0060] For the three-channel cross-dimensional interactive attention mechanism proposed in this invention:

[0061] Traditional attention mechanisms usually calculate channel attention and spatial attention separately. In particular, when calculating channel attention, the spatial information of the input tensor is converted into a pixel through global average pooling, resulting in a large loss of spatial information. The three-channel cross-dimensional interactive attention mechanism calculates the attention weight by using three parallel structures to build cross-dimensional interactions. The basic idea is to analyze the input image information from three different perspectives, and then jointly decide which parts are the most valuable feature information. It aims to enhance the model's ability to understand data by capturing interactive information in three dimensions: width, height, and channel.

[0062] 1. Overall structure:

[0063] The overall structure of the three-channel cross-dimensional interactive attention mechanism contains three parallel branches, each of which is designed to process feature maps (tensor dimensions are C×H×W, where C is the number of channels, H is the height, and W is the width), focusing on the information of the image in different dimensions of channel, height, and width. The three branches calculate the attention weights in different ways. One branch extracts spatial dimension information through the CBAM mechanism, and the other two branches calculate the attention weights by extracting the interactive features between the spatial dimension and the channel dimension. Finally, the outputs of the three branches are averaged and aggregated (Avg) structure.

[0064] 2. Upper branches:

[0065] The upper branch of the three-channel cross-dimensional interactive attention mechanism processes the input tensor χ∈R through the CBAM attention mechanism C ×H×W The main process is to perform Z pooling and convolution operations on the input respectively, and finally generate the attention weight χ1 through the Sigmoid function and residual transformation.

[0066] Z-Pool is a special pooling operation, which is mainly used for feature extraction and dimensionality reduction. It is responsible for reducing the channel dimension tensor to 2 dimensions in the three-channel cross-dimensional interactive attention mechanism. The specific method is to connect the average pooled features and the maximum pooled features on the dimension, which can not only retain the rich feature information of the channel, but also reduce its depth and reduce the amount of model calculation. The calculation formula is:

[0067] Z-Pool(χ)=[MaxPool(χ),AvgPool(χ)];

[0068] The calculation formula of CBAM attention mechanism is as follows:

[0069] ω=σ(f(w0,w1)g(χ)+f(w0,w1)δ(χ))

[0070] Among them, ω∈R C×1×1 , the function f(w0,w1) represents the following changes

[0071] ω=σ(w1ReLU(w0g(χ))+w1ReLU(w0δ(χ)))

[0072] ReLU(x)=max(0,x)

[0073] σ(x) is the Sigmoid function, and its expression is:

[0074] g(χ) is the global average pooling (Gap) function, and δ(χ) is the global maximum pooling (GMP) function, and the formula is as follows:

[0075]

[0076] 3. Central branch:

[0077] The middle branch is responsible for calculating the attention weights of the channel dimension C and the spatial dimension W. First, the input tensor is Permute1-operated, that is, the input tensor is rotated along the dimensions of width (W) and channel (C). Then, the rotated tensor is Z-pooled and convolved. Finally, the attention weights are generated through the Sigmoid function and residual transformation.

[0078] Input tensor χ∈R C×H×W , rotate χ 90° counterclockwise along the H axis to obtain the tensor χ2, whose dimension is W×H×C. After the Z-Pool operation, the tensor is obtained Its dimension becomes 2×H×C, and then after the convolution operation with a convolution kernel size of 7×7, it becomes 1×H×C, and then after the normalization operation and the Sigmoid function, it is combined with the input tensor χ∈R C×H×W Perform residual transformation, generate attention weights, and finally perform Permute2 operation on the input tensor, that is, rotate it 90° clockwise along the H axis to keep it consistent with the input.

[0079] 4. Lower branches:

[0080] The middle branch is responsible for calculating the attention weights of the channel dimension C and the spatial dimension H. First, the input tensor is Permute3-operated, that is, the input tensor is rotated along the height (H) and channel (C) dimensions, and then the rotated tensor is Z-pooled and convolved. Finally, the attention weights are generated through the Sigmoid function and residual transformation.

[0081] Input tensor χ∈R C×H×W , rotate χ 90° counterclockwise along the W axis to obtain the tensor χ3, whose dimension is H×C×W. After the Z-Pool operation, the tensor is obtained Its dimension becomes 2×C×W, and then after the convolution operation with a convolution kernel size of 7×7, it becomes 1×C×W, and then after the normalization operation and the Sigmoid function, it is combined with the input tensor χ∈R C×H×W Perform residual transformation, generate attention weights, and finally perform Permute4 operation on the input tensor, that is, rotate it 90° clockwise along the W axis to keep it consistent with the input.

[0082] 5. Average Aggregation (Avg)

[0083] After generating the attention weights for each branch, the outputs of the three branches are averaged and aggregated (Avg), and finally the triple attention output χ is obtained. out .

[0084]

[0085] Where: σ represents the sigmoid activation function;

[0086] ψ1, ψ2, and ψ3 represent the three branches obtained after the convolution operation with a convolution kernel size of 7×7;

[0087] ω1, ω2, ω3 represent the attention weights in the three branches

[0088] This structure can integrate information from different dimensions through different rotation and permutation operations to better capture the intrinsic characteristics of the data. At the same time, this method is computationally efficient and can be added as a module to the existing network architecture to enhance the network's understanding and processing capabilities of complex data structures. Through this design, the triple attention model can effectively capture the interactive relationship between the spatial and channel dimensions in the input tensor. This method enables the model to build interdependencies between channels and spatial positions, improving the model's ability to understand features.

[0089] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0091] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for automatic detection of small target sonar images, characterized in that: The following steps are involved: Collect small target sonar images with accurate target information; Based on the YOLO11 model, an improved YOLO11 model is generated by adding a three-channel cross-dimensional interactive attention mechanism at the front end of the detection head. The model is trained through the small target sonar image to construct a detection model for detecting underwater small targets through sonar images.

2. The automatic detection method for small target sonar images according to claim 1, characterized in that: In the process of improving the YOLO11 model, the three-channel cross-dimensional interactive attention mechanism includes three parallel branches, each of which is designed to process feature maps, focusing on the information of the image in different dimensions of channel, height, and width respectively; in the three branches, the attention weights are calculated in different ways, and finally the outputs of the three branches are averaged and aggregated.

3. The automatic detection method for small target sonar images according to claim 2 is characterized by: In the process of calculating the attention weights of the three branches in different ways, one of them extracts the spatial dimension information through the CBAM mechanism, and the other two branches calculate the attention weights by extracting the interaction features between the spatial dimension and the channel dimension.

4. The automatic detection method for small target sonar images according to claim 3 is characterized by: In the process of model training, based on the small target sonar image, the labelImg label is used to mark the target, and the training set, validation set, and test set are set according to 7:1:2 to construct an acoustic image set for model training.

5. An automatic detection system for small target sonar images, used to implement an automatic detection method for small target sonar images as claimed in any one of claims 1 to 4, characterized in that: include: Data acquisition module, used to collect small target sonar images with accurate target information; A data processing module is used to label the small target sonar image using the labelImg label, and to set the training set, the validation set, and the test set in a ratio of 7:1:2 to construct an acoustic image set; A model building module is used to generate an improved YOLO11 model based on the YOLO11 model by adding a three-channel cross-dimensional interactive attention mechanism at the front end of the detection head, and to train the model through the acoustic atlas to build a detection model; The monitoring module is used to detect small underwater targets by collecting sonar images based on the detection model.

6. The automatic detection method for small target sonar images according to claim 5, characterized in that: The three-channel cross-dimensional interactive attention mechanism includes three parallel branches, each of which is designed to process feature maps, focusing on the information of the image in different dimensions of channel, height, and width respectively; among the three branches, one of them extracts spatial dimension information through the CBAM mechanism, and the other two calculate the attention weights by extracting the interactive features between the spatial dimension and the channel dimension, and finally the outputs of the three branches are averaged and aggregated.

7. An automatic detection device for small target sonar images, characterized in that: include: A data acquisition unit, arranged at at least one position on the water surface, in the water and under the water, for carrying an automatic detection system for small target sonar images as described in any one of claims 5-6; A driving unit, used to carry and drive the data acquisition unit to execute the automatic detection method for small target sonar images according to any one of claims 1 to 4 to obtain the detection result of the underwater small target; A positioning unit, used for positioning the small underwater target according to the detection result; A communication unit, used to establish a data exchange channel between data acquisition units at different locations, and transmit the positioning result of the underwater small target to any data acquisition unit; A power supply unit is used to supply power.

8. The automatic detection device for small target sonar images according to claim 7, characterized in that: The device also includes a motion unit for executing the motion instructions of the driving unit, controlling the data acquisition unit to approach the small underwater target, and performing target detection. When only one data acquisition unit obtains the detection result, the motion instruction is generated according to the detection result and the positioning result to control other data acquisition units to perform target detection on the small underwater target.

Images