A side-scan sonar target detection method combining accurate image segmentation and target shadow information

Abstract

The present invention discloses a side-scan sonar target detection method that combines precise image segmentation and target shadow information. The method first addresses the problems of excessive grayscale differences and different physical resolutions in side-scan sonar images, and uses prior information such as the height and angle during imaging to autonomously perform gain compensation and resolution correction on the image. The corrected sonar image is then subjected to image segmentation based on an improved DeepLabV3+ network model. After segmentation, the imaging principle and geometric relationship of the side-scan sonar are used to match the highlight area and shadow area of the same target, and the target of interest is selected. After the data set is prepared through image correction and precise segmentation, a sonar target detection model based on the YOLOv5s network is constructed, and the model is trained. Finally, the trained results are used to perform target detection on the segmented sonar image to be detected. The present invention simultaneously introduces target acoustic shadow area information and precise image segmentation, effectively improving the detection and recognition accuracy of side-scan sonar targets.

Description

Technical Field

[0001] The present invention belongs to the field of underwater detection and identification, and specifically relates to a side-scan sonar target detection method combining precise image segmentation and target shadow information. Background Art

[0002] In marine operations, underwater target detection and identification is the most common application. Underwater targets range from small targets such as schools of fish, reefs, shipwrecks, mines, submarines, and underwater robots to large targets such as submarine volcanoes, trenches, and large areas of seabed sediments. In marine resource development, side-scan sonar is a very important means of underwater precision detection. It is usually towed behind an unmanned boat or mounted on both sides of an underwater vehicle. It uses echolocation and beamforming to shape the echo intensity at different distances and directions to either side of the vehicle. Side-scan sonar is the most popular type of imaging sonar due to its high usage, large imaging range, low cost, and moderate image resolution. It has been widely used in many fields such as topographic mapping, underwater search and rescue, and marine exploration.

[0003] Side-scan sonar imaging characteristics are closely related to seafloor topography. Sonar images can be broadly divided into three components: target, shadow, and background. The background region is characterized by seafloor reverberation and contains numerous noise points. The shadow region is formed when sound waves are blocked by targets or hillocks and cannot reach them. Sonar images are characterized by high noise, severe distortion, blurred target edges, low resolution, and poor texture, resulting in low accuracy in underwater target classification and recognition. Improving classification accuracy and speed while reducing model complexity are key issues that urgently need to be addressed. Given these challenges, deep learning algorithms, due to their ability to extract richer features, strong robustness, and practicality, offer superior performance and are well-suited for sonar target detection and recognition. YOLOv5s, among the YOLO detection algorithm family, boasts the smallest network model, the smallest feature map width, and the fastest recognition speed. Therefore, using this network as the primary model for sonar target detection and recognition can significantly improve classification accuracy, efficiency, and model complexity.

[0004] The shadow areas in side-scan sonar images contain information such as the shape and height of the sonar target. While these shadow areas are often ignored in conventional sonar image recognition, they contain as much information as the target's highlight areas. Therefore, it's necessary to consider both the target and shadow areas together to obtain more effective target information. Simultaneously extracting both the highlight and shadow areas of a sonar target and using both information for target recognition is highly effective in improving sonar target recognition.

[0005] Based on these considerations, this method investigates side-scan sonar target detection and recognition, proposing a method that combines precise image segmentation with target shadow information. Based on the characteristics of side-scan sonar imaging, the image is simultaneously segmented into target highlight and shadow regions. Finally, the information from both regions is used as feature information for subsequent detection and recognition, thereby completing the recognition of side-scan sonar targets. Summary of the Invention

[0006] In response to the above-mentioned technical problems, the present invention discloses a side-scan sonar target detection method that combines precise image segmentation and target shadow information. Starting from the imaging principle of side-scan sonar images, this method fully considers the information of the acoustic shadow area. First, to address the problems of excessive grayscale differences and different physical resolutions at different distances in side-scan sonar images, the method uses prior information such as the height and angle of the side-scan sonar imaging to autonomously perform gain compensation and resolution correction on the image, thereby realizing autonomous gain compensation and correction technology. Next, the corrected sonar image is segmented, and based on the improved DeepLabV3+ network model, feature extraction and precise image segmentation are performed on the highlight area and shadow area of the target of interest in the side-scan sonar image. After segmentation, the imaging principle and geometric relationship of the side-scan sonar are used to match the highlight area and shadow area of the same target to select the target of interest. After the above preprocessing operations, the segmented image is finally detected and identified by the YOLOv5s network model. At the same time, the target acoustic shadow area information and precise image segmentation are introduced, effectively improving the recognition accuracy of the sonar target.

[0007] The present invention aims to provide a side-scan sonar target detection method that combines accurate image segmentation and target shadow information. The method comprises the following steps:

[0008] S1: Autonomous gain compensation is performed to address the grayscale differences in side-scan sonar images caused by energy attenuation due to distance, as well as the large grayscale differences between the port and starboard sides caused by the instability of the mobile platform due to the harsh underwater environment.

[0009] S2: Geometric distortion correction is performed to address the problem of different physical resolutions at different distances in side-scan sonar images.

[0010] S3: Perform image segmentation based on the improved DeepLabV3+ network on the corrected side-scan sonar image to extract the highlight and shadow areas of the target of interest in the side-scan sonar image.

[0011] S4: After obtaining the segmented areas, the imaging principle and geometric relationship of the side scan sonar are used to match the highlight area and shadow area of the same target.

[0012] S5: Build a sonar target detection model based on the YOLOv5s network, feed the rectified and segmented images into the network for training, and use the trained model for detection and recognition of real sonar targets.

[0013] S6: Ablation experiment setup to verify the effectiveness of shadow areas in sonar images and the effectiveness of sonar image segmentation.

[0014] Furthermore, the step S1 includes the following steps:

[0015] S11: Finding the seabed line:

[0016] The bottom line is found by referring to the attitude information of the mobile platform, and then accurately searching for the bottom line position left and right. When collecting sonar data, the height information and attitude information of the underwater mobile platform are stored. Then the sonar data is parsed according to the number of pings to obtain sound intensity data and height information. Based on the pre-stored height information, a preliminary rough calculation of the bottom line is performed. Since the side scan sonar is divided into port and starboard, the conversion from height to image pixel position is performed as follows:

[0017] line orig =N s -(altitude*N s / range) (1)

[0018] Where line orig Indicates the initial seabed position point in the sonar image, altitude indicates the height information, range indicates the working range of the sonar, N s The number of sampling points for a single-sided sound intensity data line (ping(n)). The pixel with the maximum grayscale value within 50 pixels around the initial value is the sonar seabed line location.

[0019] S12: Calculate the width of the image area corresponding to the maximum towed fish depth in the image:

[0020] The height of the vehicle is constantly changing when it moves underwater. If each ping data is processed, the image will appear jagged. Since the final image data needs to be neatly arranged, it is necessary to find the minimum area range. When the towed fish is at its deepest, the range of underwater information collected is the smallest. Starting from the starboard side, N data are used to form a complete side-scan sonar image. The grayscale value converted from the sound intensity value of each point is defined as a sequence s(n,i), where n is the serial number of the ping and i is the serial number of each point in each data. The seabed line position of the nth data found by the above algorithm is a(n), so the width N of the image area corresponding to the maximum height of the towed fish in the image can be calculated according to the following formula: min :

[0021] N min =min(N s -a(n)),n=1...N (2)

[0022] S13: Calculate the average grayscale value of each ping section:

[0023] After obtaining the seabed line position and the width of the image area corresponding to the maximum towed fish depth in the image, it is necessary to perform grayscale correction on all pixels in the area. First, calculate the mean grayscale value of each ping section in the longitudinal direction of the image (the sonar image is fixed horizontally, and the vertical direction is calculated):

[0024]

[0025] S14: Get the image grayscale correction factor sequence:

[0026] After obtaining the grayscale mean of the longitudinal ping section, we then perform the grayscale mean in the transverse direction of the sonar image, thereby finally obtaining the grayscale correction factor sequence for all pixels in the image area:

[0027]

[0028] Furthermore, step S2 includes the following steps:

[0029] S21: Get the relationship between slope distance, horizontal distance and depth:

[0030] The geometric relationship between slant range, horizontal distance and depth can be obtained based on the position of the sonar transducer and the direction of the sound wave, as follows:

[0031]

[0032] Where PlantRange represents the horizontal range, SlantRange represents the slant range, and TowfishAlt represents the height of the transducer from the seafloor. TowfishAlt can be obtained by detecting the seafloor line in the sonar image.

[0033] S22: Obtain the corresponding point on the corrected horizontal range image to a point on the original slant range image based on the geometric relationship:

[0034] Side-scan sonar is divided into port and starboard, and sonar data is stored in the order in which the sonar signals are returned, so resolution correction must take port and starboard into account. Assume that a point P(x2, y2) on the original slant-range image corresponds to a point P(x1, y1) on the corrected horizontal-range image. Based on the above geometric relationship, the specific correspondence between horizontal-range points and slant-range points is as follows:

[0035] Port resolution correction factor:

[0036] Starboard resolution correction factor:

[0037] Where Res represents the image resolution, and width represents the image width. Based on the prior information about the vehicle's motion parameters during imaging, the resolution correction module automatically obtains the pixel coordinates of each point on the corrected image. These coordinates are then mapped one-to-one to the pixel coordinates, ultimately resulting in a resolution-corrected side-scan sonar image.

[0038] Furthermore, step S3 includes the following steps:

[0039] S31: Build segmentation network model:

[0040] A sonar image segmentation model based on an improved DeepLabv3+ network was constructed, replacing the original Xception series network with MobilenetV2 as the backbone feature extraction network. Feature network extraction was enhanced in the encoder and decoder, and cross-entropy loss was used for loss calculation. The Dice loss metric was introduced to evaluate semantic segmentation results.

[0041] Furthermore, the step S31 includes the following steps:

[0042] S311: Modify the backbone feature extraction network:

[0043] For the DeepLabv3+ network model used for side-scan sonar image segmentation, its original structural network uses the Xception series as the backbone extraction network, which has a large number of parameters and leads to slow training speed. Therefore, the present invention replaces it with the lightweight mobilenetV2 as the backbone extraction network. MobilenetV2 uses the Inverted Residuals structure to first expand, then compress, and finally add a residual block to directly connect the input and output.

[0044] S312: Strengthen feature network extraction:

[0045] After completing feature extraction for mobilenetV2, we obtain two valid feature layers. Next, we perform enhanced feature extraction on the initial valid features. The enhanced feature extraction network consists of two parts: the encoder and the decoder. In the encoder, for the initial valid feature layer compressed four times, we first perform feature extraction using parallel dilated convolutions with different rates. For the input x and convolution kernel w, the output feature y of the dilated convolution at the i-th position is calculated as follows:

[0046]

[0047] Where r represents the dilation rate in the dilated convolution, and kernel-size represents the size of the convolution kernel. After extraction, features are merged and finally compressed using a 1x1 convolution.

[0048] In the decoder, for the preliminary effective feature layer that has been compressed twice, the number of channels is first adjusted using 1x1 convolution. After adjustment, the result of upsampling with the above-mentioned dilated convolution feature is stacked, and the stacked result is then subjected to two depthwise separable convolutions to obtain a final effective feature layer.

[0049] S313: Improved loss function:

[0050] During model training, the Focal loss function and the Dice loss function are used to evaluate the training effect of the network model. Since the difficulty of classifying sonar image samples is inconsistent, in order to solve the model training problem caused by sample imbalance, an improvement is made based on the cross entropy loss function. The Focal loss function is proposed. It adds weight to the loss corresponding to the sample according to the difficulty of sample discrimination, reducing the weight of a large number of simple negative samples in training. The specific expression is written as:

[0051] FL(p t )=-α t (1-p t ) γ log(p t ) (9)

[0052] where p t To predict the probability of this category, in multi-classification, it is the probability of Softmax output, α t is the weight factor of each class, γ is the adjustment factor, and when γ = 0, it is the cross entropy loss function.

[0053] The Dice coefficient is a set similarity measurement function with a value range of [0,1]. The larger the coefficient value, the greater the overlap between the predicted result and the true result. However, during model training, the smaller the loss value, the better. Therefore, Dice loss is defined as follows:

[0054]

[0055] Where |X∩Y| represents the intersection of the predicted result and the true result, |X| and |Y| represent the number of elements in the predicted result and the true result, respectively. In the DeepLabV3+ semantic segmentation model adopted by the present invention, X represents the ground truth and Y represents the predicted segmented image.

[0056] S32: Create a sonar image dataset and perform model training:

[0057] After the model was modified, a sonar image dataset was created and processed accordingly. The purpose of the sonar image segmentation component is to segment target highlights (formed by active acoustic echoes) from shadows (formed by obstructions). It does not distinguish between the target types represented by each region. Therefore, for the segmentation task, only two types of labels are used for the dataset: highlights (light) and shadows (dark). Through experimental collection, a training set of 488 side-scan sonar images, a validation set of 71 images, and a test set of 136 images were compiled. Each image in the test and validation sets was annotated with target highlights and shadows. After these labels were fed into the network, the network's category parameters, backbone model, and pre-trained weights were modified to complete the training of the sonar image segmentation model.

[0058] S33: Using the trained model, the sonar image to be segmented is segmented into the target bright area and the acoustic shadow area to obtain the final image segmentation result:

[0059] The weight file in the test code was modified to match the weights that yielded the best training results, along with the category parameters. After these modifications, the sonar images in the test set were segmented into target bright and shadow areas to obtain the final segmentation results. After segmentation, the original sonar image was converted into a mask consisting of only three pixel values: target bright area, shadow segmentation, and background. This effectively addresses the issues of blurred sonar target edges, high noise levels, and low resolution in sonar images.

[0060] The sonar image segmentation part does not classify the target, which is equivalent to further preprocessing the original sonar image. After the above preprocessing, the image is accurately segmented and the sonar target shadow information is extracted. Therefore, the segmented image needs to be used for target detection and recognition.

[0061] Furthermore, step S4 includes the following steps:

[0062] S41: Use the imaging principle of side-scan sonar to match the bright and shadow areas of the same target.

[0063] After obtaining the segmented regions, according to the imaging principle of side-scan sonar imaging, since the transmission and detection echoes of side-scan sonar are perpendicular to the heading, after the sound wave is transmitted, it propagates through the acoustic channel and, upon reaching the seabed, the reflected wave returns along the original path to the transducer, forming the corresponding target echo. The target shadow area is where the sound wave is blocked by the target and is not illuminated. Therefore, the target shadow area and the target highlight area should be aligned with the sound line, that is, the highlight area and the shadow are on the same horizontal line, and the area formed by the shadow and the highlight area should be at the same height (in the direction of the vehicle's movement). For a target located on the seabed, the shadow is immediately behind the target's strong echo.

[0064] Since there may be multiple target highlight areas and shadow areas obtained by the segmentation network, the highlight areas and shadow areas of the same target are matched according to the imaging principle of the above-mentioned side-scan sonar image to select the target of interest.

[0065] Furthermore, step S5 includes the following steps:

[0066] S51: Building network models and algorithms:

[0067] YOLOv5s is the network with the smallest network model, the smallest feature map width, and the fastest recognition speed in the YOLOv5 detection algorithm series. Therefore, the present invention uses this network as the main model for sonar target detection and recognition. The YOLOv5s network structure consists of four main parts: Input, Backbone, Neck (multi-scale feature fusion), and Output. The Input includes Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling; the Backbone uses the Focus structure and the CSP structure; the Neck adopts the FPN+PAN structure; and the Output includes the Bounding box loss function calculation and NMS non-maximum suppression.

[0068] S52: Data collection, annotation and dataset construction:

[0069] After the above preprocessing operations, image correction and precise segmentation are used to improve problems such as blurred sonar target edges and achieve accurate extraction of target shadow information. This section performs side-scan sonar target detection based on precise image segmentation and target shadow information, and thus uses the segmentation results obtained by the DeepLabV3+ network as the raw data for the sonar target detection network.

[0070] First, the highlighted and shaded areas of the segmented target are annotated simultaneously, and the bounding boxes are placed into the same target box. A total of four data types are annotated: drowning people, mines, aircraft, and shipwrecks. The annotated data is then divided into training, validation, and test sets. The training set consists of 488 images, including 98 images of drowning people, 119 images of mines, 87 images of aircraft, and 184 images of shipwrecks; the validation set consists of 71 images, including 17 images of drowning people, 18 images of mines, 13 images of aircraft, and 23 images of shipwrecks; and the test set consists of 136 images, including 13 images of drowning people, 35 images of mines, 23 images of aircraft, and 65 images of shipwrecks.

[0071] S53: Experimental setup and model training:

[0072] Experimental environment configuration and parameter setting: The models of this invention are trained and tested on the NVIDIA GeForce RTX 3080 graphics card based on the PyTorch deep learning framework, using Python as the programming language, modifying the target type, pre-training weight path, and network structure configuration file, setting epochs, batch-size, image size, initial learning rate, cyclic learning rate, learning rate momentum, weight decay coefficient, IoU loss coefficient, cls loss coefficient, and cls BCELoss positive sample weight and other parameters to complete the construction and setting of the overall model.

[0073] Model Training: After modifying and setting various parameters, the sonar image segmentation results obtained above were used as a dataset and fed into the network for model training. The model training results were visualized using the Tensorboard tool. The mean values of the three loss functions (prediction box regression loss function, object detection loss function, and classification loss function) were analyzed over the number of iterations. The precision and recall rates were also analyzed over the number of iterations. The mean average precision at an Intersection over Union (IoU) threshold of 0.5 and at IoU thresholds of 0.5:0.95 were also analyzed over the number of iterations.

[0074] S54: Model testing and result analysis:

[0075] After the network model training is completed, the weight file with the best training effect is selected to perform position regression on the test set data to identify sonar targets, thus completing the model testing. Finally, the model is evaluated from multiple aspects using evaluation indicators such as precision, recall, PR curve, F1 score, and average precision. TP (True Positive) means that the true category of the sample is positive and the model predicts it as a positive example; TN (True Negative) means that the true category of the sample is negative and the model predicts it as a negative example; FP (False Positive) means that the true category of the sample is negative, but the model predicts it as a positive example; FN (False Negative) means that the true category of the sample is positive, but the model predicts it as a negative example.

[0076] Precision is a measure of accuracy, indicating the proportion of examples classified as positive that are actually positive. Its calculation formula is defined as follows:

[0077]

[0078] Recall is a measure of coverage, measuring how many positive examples are classified as positive examples. Its calculation formula is defined as follows:

[0079]

[0080] The PR curve is a curve with Recall as the horizontal axis and Precision as the vertical axis. The larger the area on the lower left, the better the model performs on the dataset. The shaded area on the lower left of the PR curve is the average precision (AP), which is calculated as follows:

[0081]

[0082] F1-score (F1 score) represents the harmonic mean of precision and recall, and is a comprehensive evaluation indicator used to evaluate the detection ability of the model. Its value is [0,1]. Its calculation formula is defined as follows:

[0083]

[0084] Furthermore, step S6 includes the following steps:

[0085] S61: Verify the effectiveness of sonar target shadow areas:

[0086] First, only the highlighted areas of the target are annotated in the original sonar image. Note that image segmentation is not performed here; the original sonar image is directly annotated. The annotated dataset is then divided into training, validation, and test sets. The training set data is fed into the YOLOv5s network for model training. The validation set verifies the model's performance, and the test set predicts the trained model's results. Finally, graphs of metrics such as Precision, Recall, PR curves, and F1-score, as well as a confusion matrix, are obtained.

[0087] Next, the original sonar image was annotated with both the target highlight and shadow areas, and a series of indicator curves and confusion matrix results were obtained. Finally, the test results based on the sonar target highlight area were compared with the test results based on the combined sonar target highlight and shadow areas to verify the effectiveness of the shadow area information in the acoustic image.

[0088] S62: Verify the effectiveness of accurate image segmentation:

[0089] Based on the results obtained in the above steps, the indicator curve results and confusion matrix obtained by simultaneously detecting the highlight and shadow areas of the target in the original sonar image are compared with the test results obtained by pre-segmenting the bright and shadow areas of the sonar target using the DeepLabV3+ network and then detecting them, thereby verifying the effectiveness of the pre-precise segmentation of the side-scan sonar image.

[0090] S63: Verify the effectiveness of joint image segmentation and target shadow information:

[0091] Based on the test results obtained above, the detection and recognition effect of the original target bright area without segmentation is compared with the index curve results and confusion matrix obtained by simultaneously combining the shadow area and image precise segmentation, thereby verifying the effectiveness of the combined image precise segmentation and target shadow information. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is a flowchart of the present invention.

[0093] Figure 2 This is a schematic diagram of the side-scan sonar imaging principle of the present invention.

[0094] Figure 3 This is a schematic diagram of the side scan sonar image preprocessing results of the present invention.

[0095] Figure 4 This is a schematic diagram of the training process curve based on the YOLOv5s model detection of the present invention.

[0096] Figure 5 It is the index value of all categories in the test results of the present invention.

[0097] Figure 6 This is the curve showing the change of precision with confidence in the test results of the present invention.

[0098] Figure 7 This is the curve of the recall rate (Recall) versus confidence in the test results of the present invention.

[0099] Figure 8 Schematic diagram of the PR curve in the test results of the present invention.

[0100] Figure 9 This is the curve showing the F1-score changing with the confidence level in the test results of the present invention.

[0101] Figure 10 Schematic diagram of the confusion matrix results in the test results of the present invention.

[0102] Figure 11 This is an example diagram of some detection results in the test results of the present invention.

[0103] Figure 12 This is a schematic diagram of the comparison results for verifying the effectiveness of the shadow area of the present invention.

[0104] Figure 13 Schematic diagram of comparison results for verifying the effectiveness of image segmentation of the present invention.

[0105] Figure 14 This is a comparison chart of the verification of the joint target shadow area and image accurate segmentation effect of the present invention. DETAILED DESCRIPTION

[0106] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0107] refer to Figure 1 , Figure 1 The present invention provides a flow chart of a side-scan sonar target detection method that combines precise image segmentation and target shadow information, including the following steps:

[0108] S1: Autonomous gain compensation is performed to address the grayscale differences in side-scan sonar images caused by energy attenuation due to distance, as well as the large grayscale differences between the port and starboard sides caused by the instability of the mobile platform due to the harsh underwater environment.

[0109] S2: Geometric distortion correction is performed to address the problem of different physical resolutions at different distances in side-scan sonar images.

[0110] S3: Perform image segmentation based on the improved DeepLabV3+ network on the corrected side-scan sonar image to extract the highlight and shadow areas of the target of interest in the side-scan sonar image.

[0111] S4: After obtaining the segmented areas, the imaging principle and geometric relationship of the side scan sonar are used to match the highlight area and shadow area of the same target.

[0112] S5: Build a sonar target detection model based on the YOLOv5s network, feed the rectified and segmented images into the network for training, and use the trained model for detection and recognition of real sonar targets.

[0113] S6: Ablation experiment setup to verify the effectiveness of shadow areas in sonar images and the effectiveness of sonar image segmentation.

[0114] Furthermore, the step S1 includes the following steps:

[0115] S11: Find the seabed line. The seabed line is found by referring to the posture information of the mobile platform, and then accurately searching for the seabed line position left and right:

[0116] When collecting sonar data, the height and attitude information of the underwater mobile platform are stored. Then, the sonar data is parsed according to the number of pings to obtain sound intensity data and height information. Based on the pre-stored height information, a preliminary rough calculation of the seabed line is performed. Since the side scan sonar is divided into port and starboard, the conversion from height to image pixel position is performed as follows:

[0117] line orig =N s -(altitude*N s / range) (15)

[0118] Where line orig Indicates the initial seabed position point in the sonar image, altitude indicates the height information, range indicates the working range of the sonar, N s It represents the number of sampling points of a certain sound intensity data (ping(n)) obtained by a single side. Then, within 50 pixels around the initial value, find the point with the maximum gray value, which is the sonar seabed line position point a(n):

[0119] a(n)=max(line orig ±50) (16)

[0120] In the sonar style adopted by the present invention, N s =1000, range=20m, and altitude is the altitude information collected in real time by the altitude sensor carried by the underwater mobile platform.

[0121] S12: Calculate the width of the image area corresponding to the maximum towed fish depth in the image:

[0122] The height of the vehicle is constantly changing when it moves underwater. If each ping data is processed, the image will appear jagged. Since the final image data needs to be neatly arranged, it is necessary to find the minimum area range. When the towed fish is at its deepest, the range of underwater information collected is the smallest. Starting from the starboard side, N data are used to form a complete side-scan sonar image. The grayscale value converted from the sound intensity value of each point is defined as a sequence s(n,i), where n is the serial number of the ping and i is the serial number of each point in each data. The seabed line position of the nth data found by the above algorithm is a(n), so the width N of the image area corresponding to the maximum height of the towed fish in the image can be calculated according to the following formula: min :

[0123] N min =min(N s -a(n)),n=1...N (17)

[0124] S13: Calculate the average grayscale value of each ping section:

[0125] After obtaining the seabed line position and the width of the image area corresponding to the maximum towed fish depth in the image, it is necessary to perform grayscale correction on all pixels in the area. First, calculate the mean grayscale value of each ping section in the longitudinal direction of the image (the sonar image is fixed horizontally, and the vertical direction is calculated):

[0126]

[0127] In order to observe the side scan sonar image beautifully, N=500 is adopted in the present invention.

[0128] S14: Get the image grayscale correction factor sequence:

[0129] After obtaining the grayscale mean of the longitudinal ping section, we then perform the grayscale mean in the transverse direction of the sonar image, thereby finally obtaining the grayscale correction factor sequence for all pixels in the image area:

[0130]

[0131] Furthermore, step S2 includes the following steps:

[0132] S21: Get the relationship between slope distance, horizontal distance and depth:

[0133] The geometric relationship between slant range, horizontal distance and depth can be obtained based on the position of the sonar transducer and the direction of the sound wave, as follows:

[0134]

[0135] Where PlantRange represents the horizontal range, SlantRange represents the slant range, and TowfishAlt represents the height of the transducer from the seafloor. TowfishAlt can be obtained by detecting the seafloor line in the sonar image.

[0136] S22: Obtain the corresponding point on the corrected horizontal range image to a point on the original slant range image based on the geometric relationship:

[0137] Side-scan sonar is divided into port and starboard, and sonar data is stored in the order in which the sonar signals are returned, so resolution correction must take port and starboard into account. Assume that a point P(x2, y2) on the original slant-range image corresponds to a point P(x1, y1) on the corrected horizontal-range image. Based on the above geometric relationship, the specific correspondence between horizontal-range points and slant-range points is as follows:

[0138] Port resolution correction factor:

[0139] Starboard resolution correction factor:

[0140] Wherein, Res represents the resolution of the image, and width represents the image width. In the present invention, width = 2000, TowfishAlt = a, and Res = 1. During the calculation, since x2 is an integer, the x1 obtained after a series of calculations is a non-integer. The non-integer value cannot find the corresponding pixel position point in the image. Therefore, after completing the correspondence calculation, the bilinear interpolation principle is required to calculate the pixel value at the non-integer image coordinate position point. Since y1 = y2, the image coordinates are only interpolated in the horizontal x-axis direction, so the pixel value at the non-integer x1 can be obtained according to the bilinear interpolation calculation formula:

[0141]

[0142] Based on the prior information of the vehicle's motion parameters during imaging, the pixel coordinates of each point on the corrected image are automatically obtained in the resolution correction module, and then they are mapped one by one to the pixel coordinates, ultimately obtaining a resolution-corrected side-scan sonar image.

[0143] Furthermore, step S3 includes the following steps:

[0144] S31: Build segmentation network model:

[0145] A sonar image segmentation model based on an improved DeepLabV3+ network was constructed, replacing the original Xception series network with MobilenetV2 as the backbone feature extraction network. Feature network extraction was enhanced in the encoder and decoder, and Focal loss was used for loss calculation, with the Dice loss metric introduced to evaluate semantic segmentation results.

[0146] Furthermore, the step S31 includes the following steps:

[0147] S311: Modify the backbone feature extraction network:

[0148] The DeepLabV3+ network model used for side-scan sonar image segmentation originally used the Xception series as the backbone extraction network, which has a large number of parameters and slow training speed. Therefore, this paper replaces it with the lightweight mobilenetV2 as the backbone extraction network. MobilenetV2 uses an inverted residuals structure for expansion and then compression. Specifically, it first uses 1x1 convolution for dimensionality increase, then uses 3x3 depthwise separable convolution for feature extraction, and finally uses 1x1 convolution for dimensionality reduction. The new activation function ReLU6 is used in the mobilenetV2 structure:

[0149] y=ReLU6(x)=min(max(x,0),6) (24)

[0150] In the last layer of 1x1 convolution dimensionality reduction, in order to prevent the ReLU activation function from causing a large loss of low-dimensional information while causing little loss of high-dimensional information, a linear activation function is used:

[0151] y=linear(x)=x (25)

[0152] Finally, the entire structure is added with a residual block to directly connect the input and output.

[0153] S312: Strengthen feature network extraction:

[0154] After completing feature extraction for mobilenetV2, we obtain two valid feature layers. Next, we perform enhanced feature extraction on the initial valid features. The enhanced feature extraction network consists of two parts: the encoder and the decoder. In the encoder, for the initial valid feature layer compressed four times, we first perform feature extraction using parallel dilated convolutions with different rates. For the input x and convolution kernel w, the output feature y of the dilated convolution at the i-th position is calculated as follows:

[0155]

[0156] Where r represents the dilation rate in the dilated convolution, and kernel-size represents the size of the convolution kernel. After extraction, features are merged and finally compressed by 1x1 convolution. Using dilated convolution at different rates can obtain different ranges of receptive field r:

[0157]

[0158] Where S kernal-i is the size of the convolution kernel of the i-th layer, V i is the receptive field size of the i-th layer, step i-1 is the kernel size of the i-1th layer.

[0159] In the decoder, for the preliminary effective feature layer that has been compressed twice, the number of channels is first adjusted using 1x1 convolution. After adjustment, the result of upsampling with the above-mentioned dilated convolution feature is stacked, and the stacked result is then subjected to two depthwise separable convolutions to obtain a final effective feature layer.

[0160] S313: Improved loss function:

[0161] During model training, the cross entropy loss function and the Dice loss function are used to evaluate the training effect of the network model. The cross entropy loss function is specifically expressed as follows:

[0162]

[0163] Where θ represents the weight parameter, x represents the batch training sample size, p1 represents the expected category probability, and p2 represents the predicted category probability.

[0164] Since the difficulty of classifying sonar image samples is inconsistent, in order to solve the model training problem caused by sample imbalance, an improvement is made on the basis of the cross entropy loss function. The Focal loss function is proposed, which adds weights to the loss corresponding to the sample according to the difficulty of sample discrimination, reducing the weight of a large number of simple negative samples in training. The specific expression is written as:

[0165] FL(p t )=-α t (1-p t ) γ log(p t ) (29)

[0166] where p t To predict the probability of this category, in multi-classification, it is the probability of Softmax output, α t is the weight factor of each class, γ is the adjustment factor, and when γ = 0, it is the cross entropy loss function.

[0167] The Dice coefficient is a set similarity measurement function, usually used to calculate the similarity between two samples. Its value range is [0,1]. The calculation formula is as follows:

[0168]

[0169] Where |X∩Y| represents the intersection of the predicted result and the true result, and |X| and |Y| represent the number of elements in the predicted result and the true result, respectively. In the DeepLabV3+ semantic segmentation model adopted by this invention, X represents the ground truth and Y represents the predicted segmented image. A larger coefficient value indicates a greater overlap between the predicted result and the true result. However, during model training, the smaller the loss value, the better. Therefore, the Dice loss as a semantic segmentation loss is defined as follows:

[0170]

[0171] S32: Create a sonar image dataset and perform model training:

[0172] After the model was modified, a sonar image dataset was created and processed accordingly. The purpose of the sonar image segmentation component is to segment target highlights (formed by active acoustic echoes) from shadows (formed by obstructions). The target type represented by each region is not distinguished. Therefore, for the segmentation task, only two types of labels are used for the dataset: highlight (light) and shadow (dark). Through experimental collection, a training set of 488 side-scan sonar images, a validation set of 71 images, and a test set of 136 images were compiled. Each image in the test and validation sets was annotated with the target highlight and shadow regions. After this labeling, the images were fed into the network, and the network's category parameters, backbone model, and pre-trained weights were modified to complete the training of the sonar image segmentation model.

[0173] S33: Using the trained model, the sonar image to be segmented is segmented into the target bright area and the acoustic shadow area to obtain the final image segmentation result:

[0174] The weight file in the test code was modified to match the weights that yielded the best training results, along with the category parameters. After these modifications, the sonar images in the test set were segmented into target bright and shadow areas to obtain the final segmentation results. After segmentation, the original sonar image was converted into a mask consisting of only three pixel values: target bright area, shadow segmentation, and background. This effectively addresses the issues of blurred sonar target edges, high noise levels, and low resolution in sonar images.

[0175] refer to Figure 2 The following figure shows the side-scan sonar image after the correction and pre-segmentation described above. The sonar image segmentation process does not classify targets, but rather performs further preprocessing on the original sonar image. This preprocessing simultaneously achieves accurate image segmentation and extracts sonar target shadow information. Therefore, the segmented image must then be used for target detection and recognition.

[0176] Furthermore, step S4 includes the following steps:

[0177] S41: Use the imaging principle of side-scan sonar to match the bright and shadow areas of the same target:

[0178] After obtaining the segmented regions, according to the imaging principle of side-scan sonar images, since the emission and detection echoes of side-scan sonar are perpendicular to the heading, after the sound wave is transmitted, it propagates through the acoustic channel, reaches the seabed, and the reflected wave returns along the original path to the transducer to form the corresponding target echo. The target shadow area is the area where the sound wave is blocked by the target and is not illuminated, resulting in very few sound wave echoes. Therefore, the target shadow area and the target highlight area should be in the same direction as the sound line, that is, the highlight area and the shadow area should be on the same horizontal line, and the area formed by the shadow and the highlight area should be at the same height (in the direction of vehicle movement).

[0179] refer to Figure 3 This image shows the imaging principle of side-scan sonar images and the shadows cast by targets. The image shows that the bright area created by the sonar target and the acoustic shadow are on the same horizontal line and at the same height. In the image, the shadow of a target on the seafloor is directly behind the target's strong echo.

[0180] Since there may be multiple target highlight areas and shadow areas obtained by the segmentation network, the highlight areas and shadow areas of the same target are matched according to the imaging principle of the above-mentioned side-scan sonar image to select the target of interest.

[0181] Furthermore, step S5 includes the following steps:

[0182] S51: Building network models and algorithms:

[0183] YOLOv5s is the network with the smallest network model, the smallest feature map width, and the fastest recognition speed in the YOLOv5 detection algorithm series. Therefore, this paper uses this network as the main model for sonar target detection and recognition. It consists of four main parts: Input, Backbone, Neck (multi-scale feature fusion), and Output. The Input includes Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling; the Backbone uses the Focus structure and CSP structure; the Neck uses the FPN+PAN structure; and the Output includes the Bounding box loss function calculation and NMS non-maximum suppression. The general process of using the YOLOv5s network for side-scan sonar target detection is as follows:

[0184] (1) Image preprocessing: First, the side-scan sonar image is enhanced with mosaic data, then adaptively scaled to a pre-specified image size (640x640), and finally fed into the network in batches according to a pre-set minibatch size.

[0185] (2) Data forward propagation. Forward propagation includes feature extraction based on Backbone, feature fusion and prediction based on Neck, and finally obtains the position, size and type of the sonar target prediction box.

[0186] (3) Calculate the error. Calculate the error between the predicted box and the ground truth based on the loss function.

[0187] (4) Parameter update. The coefficient matrix and bias in the forward propagation are updated using the gradient descent method to reduce the error between the predicted box and the ground truth. Finally, the coefficient matrix and bias corresponding to the minimum loss value are selected.

[0188] (5) Target prediction. The coefficient matrix and bias selected at the end of the iteration are replaced in the forward propagation to solve the target prediction information for the side-scan sonar image to be detected.

[0189] S52: Data collection, annotation and dataset construction:

[0190] After the above preprocessing operations, image correction and precise segmentation are used to improve problems such as blurred sonar target edges and achieve accurate extraction of target shadow information. This section performs side-scan sonar target detection based on precise image segmentation and target shadow information, and thus uses the segmentation results obtained by the DeepLabV3+ network as the raw data for the sonar target detection network.

[0191] First, the target highlight area and shadow area of the segmentation result are annotated with borders at the same time, and they are annotated into the same target frame. A total of 4 types of data are annotated, namely drowning people, mines, aircraft and shipwrecks. Then the annotated data are decomposed into training set, validation set and test set. The training set has 488 images, including 98 drowning person target images, 119 mine target images, 87 aircraft target images, and 184 shipwreck target images; the validation set has 71 images, including 17 drowning person target images, 18 mine target images, 13 aircraft target images, and 23 shipwreck target images; the test set has 136 images, including 13 drowning person target images, 35 mine target images, 23 aircraft target images, and 65 shipwreck target images. The details are shown in the following table:

[0192] category Training set / sheet Validation set / sheet Test set / sheet Drowning 98 17 13 mines 119 18 35 airplane 87 13 23 Shipwreck 184 23 65

[0193] S53: Experimental setup and model training:

[0194] Experimental environment configuration: The training and testing of the model of this invention are completed under the Ubuntu system, using Python as the programming language, using the PyTorch deep learning framework, and the CPU model is silver 4110 CPU@2.10Ghz, memory is 64G, GPU model is NVIDIA GeForce RTX 3080, and GPU acceleration library is CUDA 11.4.

[0195] Parameter settings: change the target type (number of classes) to 4, change the pre-training weight path to yolov5s.pt, change the network structure configuration file to yolov5s.yaml, change the dataset path to the self-created sonar dataset data file sonar.yaml, set epochs to 150, batch-size to 16, image size to 640*640, initial learning rate to 0.01, cyclic learning rate to 0.1, learning rate momentum to 0.937, weight decay coefficient to 0.0005, warm-up learning to 3.0, warm-up learning momentum to 0.8, warm-up initial learning rate to 0.1, IoU loss coefficient to 0.05, cls loss coefficient to 0.5, cls BCELoss positive sample weight to 1.0, IoU training threshold to 0.2, anchor aspect ratio to 4.0.

[0196] Model Training: After modifying and setting various parameters, the sonar image segmentation results obtained above were used as a dataset and fed into the network for model training. The model training results were visualized using the Tensorboard tool. The mean values of the prediction box regression loss function, the object detection loss function, and the classification loss function were analyzed over the number of iterations. The precision and recall rates were also analyzed over the number of iterations. The mean average precision at an Intersection over Union (IoU) threshold of 0.5 and at IoU thresholds of 0.5:0.95 were also analyzed over the number of iterations.

[0197] refer to Figure 4 It shows the changing trends of various types of function values during the training process.

[0198] The first figure in the first row shows the mean value of the prediction box regression loss function in the training set as a function of the number of iterations. Smaller values indicate more accurate box predictions. The training results show that box predictions are below 0.02, indicating a relatively accurate result. The second figure in the first row shows the mean value of the object detection loss function in the training set as a function of the number of iterations. Smaller values indicate more accurate object detection. The training results show that the loss function ultimately decreases to 0.01, indicating a relatively accurate object detection result. The third figure in the first row shows the mean value of the classification loss function in the training set as a function of the number of iterations. Smaller values indicate more accurate classifications. The training results show that the loss function value decreases to 0.01, indicating a relatively accurate classification result. The mean values of the above three loss functions decrease rapidly with the number of iterations and stabilize around 100 iterations in the later stages of training.

[0199] The fourth figure in the first row shows how precision changes with the number of iterations in the training set. A larger value indicates higher prediction accuracy. The fifth figure in the first row shows how recall changes with the number of iterations in the training set. A larger value indicates higher prediction accuracy. The figures show that both precision and recall increase rapidly with the number of iterations and then stabilize.

[0200] The first, second, and third figures in the second row are the changes in the mean of the prediction box regression loss function, the mean of the target detection loss function, and the mean of the classification loss function in the validation set with the number of iterations. They also decrease rapidly with the increase in the number of iterations, and finally tend to stabilize at around 100 times. However, compared with the training set, their curves fluctuate more and are not smooth.

[0201] The fourth image in the second row shows the mean average precision (mAP@0.5) when the intersection over union (IoU) threshold is 0.5. The fifth image in the second row shows the mean average precision (mAP@0.5:0.95) when the intersection over union (IoU) threshold is 0.5:0.95. Both mAP@0.5 and mAP@0.5:0.95 values are gradually increasing and stabilizing.

[0202] S54: Model testing and result analysis:

[0203] After the network model training is completed, the weight file with the best training effect is selected to perform position regression on the test set data to identify sonar targets, thus completing the model testing. Finally, the model is evaluated from multiple aspects using evaluation indicators such as precision, recall, PR curve, F1 score, and average precision. TP (True Positive) means that the true category of the sample is positive and the model predicts it as a positive example; TN (True Negative) means that the true category of the sample is negative and the model predicts it as a negative example; FP (False Positive) means that the true category of the sample is negative, but the model predicts it as a positive example; FN (False Negative) means that the true category of the sample is positive, but the model predicts it as a negative example.

[0204] Precision is a measure of accuracy, indicating the proportion of examples classified as positive that are actually positive. Its calculation formula is defined as follows:

[0205]

[0206] Recall is a measure of coverage, measuring how many positive examples are classified as positive examples. Its calculation formula is defined as follows:

[0207]

[0208] The PR curve is a curve with Recall as the horizontal axis and Precision as the vertical axis. The larger the area on the lower left, the better the model performs on the dataset. The shaded area on the lower left of the PR curve is the average precision (AP), which is calculated as follows:

[0209]

[0210] F1-score (F1 score) represents the harmonic mean of precision and recall, and is a comprehensive evaluation indicator used to evaluate the detection ability of the model. Its value is [0,1]. Its calculation formula is defined as follows:

[0211]

[0212] refer to Figure 5 The results for the test set show the precision, recall, mAP@0.5, and mAP@0.5:0.95 values for all categories and classes. When the Intersection over Union (IoU) ratio is 0.5, the overall prediction values are: P = 0.944, R = 0.925, mAP@0.5 = 0.974, and mAP@0.5:0.95 = 0.698. The recognition accuracies for the four underwater sonar targets are: 1 for human, 91.2% for mine, 92.7% for plane, and 93.9% for ship. The prediction results show that the prediction values for the test set are very high, the prediction effect is good, and high recognition accuracy is achieved.

[0213] refer to Figure 6 The following graph shows how the accuracy of the test set changes with confidence. The graph shows that for all categories with a confidence level of 0.753 or higher, the prediction accuracy reaches 1. For the human category, the prediction accuracy reaches 1 at a confidence level of 0.4.

[0214] refer to Figure 7 The following graph shows how recall changes with confidence in the test set. The confidence level is typically set to 0.5. The graph shows that at a confidence level of 0.5, the recall for the human class is 0.915, the recall for the mine class is 0.886, the recall for the plane class is 0.886, and the recall for the ship class is 0.941. These results demonstrate that the recall values are all high, indicating good prediction performance.

[0215] refer to Figure 8The figure shows a schematic representation of the PR curve for the test set results. The figure shows the AP value, the area enclosed by the PR curve and the coordinate axes. The average AP value for all categories, mAP@0.5, is 0.974. The AP values for the individual categories are: human (0.982), mine (0.959), plane (0.977), and ship (0.978). These high mAP values demonstrate the excellent performance of our model.

[0216] refer to Figure 9 The graph shows the curve of the F1-score of the test set results as the confidence level changes. The figure shows that the F1 score of all categories reaches 0.93 at a confidence level of 0.393, which is close to 1, verifying the strong detection ability of the proposed model.

[0217] refer to Figure 10 The following figure shows the confusion matrix results for the test set. The confusion matrix indicates which parts of the model are confused when making predictions. The values along the main diagonal from the upper left to the lower right represent the correct classification probabilities. The figure shows that the correct classification probabilities are: 0.85 for human, 0.89 for mine, 0.96 for plane, and 0.94 for ship. Starting from the horizontal axis, the human class is easily misidentified as the plane class, the mine class is easily misidentified as the ship class, the plane class is easily misidentified as background, and the ship class is easily misidentified as the mine class.

[0218] refer to Figure 11 The following figure shows a schematic diagram of some detection results from the test set. The detection and recognition results of the four categories are all displayed, and all of them can correctly detect the target and accurately identify the sonar target type.

[0219] Furthermore, step S6 includes the following steps:

[0220] S61: Verify the effectiveness of sonar target shadow areas:

[0221] First, only the highlighted areas of the target are annotated in the original sonar image. Note that image segmentation is not performed here; the original sonar image is directly annotated. The annotated dataset is then divided into training, validation, and test sets. The training set data is fed into the YOLOv5s network for model training. The validation set verifies the model's performance, and the test set predicts the trained model's results. Finally, graphs of metrics such as Precision, Recall, PR curves, and F1-score, as well as a confusion matrix, are obtained.

[0222] Next, the original sonar image was annotated with both the target highlight and shadow areas, and a series of indicator curves and confusion matrix results were obtained. Finally, the test results based on the sonar target highlight area were compared with the test results based on the combined sonar target highlight and shadow areas to verify the effectiveness of the shadow area information in the acoustic image.

[0223] refer to Figure 12 A schematic diagram shows the comparison results for validating the effectiveness of shadow regions. Comparing the various metrics in the data table shows that target detection using both shadow and highlight information outperforms detection using only highlight information in all metrics. The mAP@0.5 value for the shadow-rich approach is 3.8% higher than that for the shadow-free approach, the mAP@0.5:0.95 is 11.6% higher, the R-value is 3.9% higher, and the P-value is 0.2% higher. Comparing the F1 score vs. confidence curve, the P-value vs. confidence curve, the R-value vs. confidence curve, the PR curve, and the confusion matrix further validates the correctness and effectiveness of the proposed algorithm's integration of sonar target shadow information.

[0224] S62: Verify the effectiveness of accurate image segmentation:

[0225] Based on the results obtained in the above steps, the indicator curve results and confusion matrix obtained by simultaneously detecting the highlight and shadow areas of the target in the original sonar image are compared with the test results obtained by pre-segmenting the bright and shadow areas of the sonar target using the DeepLabV3+ network and then detecting them, thereby verifying the effectiveness of the pre-precise segmentation of the side-scan sonar image.

[0226] refer to Figure 13 The figure shows a schematic diagram of the comparison results for validating image segmentation effectiveness. Comparing the various metrics in the data table shows that the target detection results using the DeepLabV3+ network to segment the bright and shadow areas of sonar targets are superior to those without segmentation preprocessing (although target shadow information is still included). The mAP@0.5 value after image segmentation is 0.8% higher than that without segmentation, the mAP@0.5:0.95 is 6.3% higher, the R value is 1.3% higher, and the P value is 4.3% higher. This demonstrates the correctness and effectiveness of the algorithm's proposed preprocessing of segmenting the bright and shadow areas of sonar targets before target detection.

[0227] contrast Figure 12 and Figure 13The results show that adding sonar target shadow information significantly improves mAP, indicating that shadow information enhances the model's detection and recognition performance across all categories. Pre-processing the sonar image by segmenting the target's bright areas from its shadows before target detection also significantly improves mAP, demonstrating that pre-segmentation effectively enhances the model's target classification performance. This means that shadow regions effectively enhance the overall model's detection performance, while image segmentation effectively enhances the model's target recognition performance.

[0228] S63: Verify the effectiveness of joint image segmentation and target shadow information:

[0229] Based on the test results obtained above, the detection and recognition effect of the original target bright area without segmentation is compared with the index curve results and confusion matrix obtained by simultaneously combining the shadow area and image precise segmentation, thereby verifying the effectiveness of the combined image precise segmentation and target shadow information.

[0230] refer to Figure 14 The comparison shows the effect. From the comparison of various indicators in the data table, it can be seen that the target detection effect of the joint shadow area and image segmentation is better than the original detection result without segmentation and only including the target bright area in all indicators. The mAP@0.5 value of the former is 4.6% higher than that without segmentation, mAP@0.5:0.95 is 17.9% higher, R value is 5.2% higher, and P value is 4.5% higher, which verifies the correctness and effectiveness of the proposed joint shadow area and image segmentation in this algorithm.

[0231] The preferred embodiments and principles of the present invention are described in detail above. For those skilled in the art, there may be changes in the specific implementation methods based on the ideas provided by the present invention, and these changes should also be considered as the scope of protection of the present invention.

Claims

1. A side-scan sonar target detection method combining accurate image segmentation and target shadow information, characterized in that: The following steps are involved: S1: Autonomous gain compensation is performed to address the grayscale differences in side-scan sonar images caused by energy attenuation due to distance, as well as the large grayscale differences between the port and starboard sides caused by the instability of the mobile platform due to the harsh underwater environment. S2: Correct the geometric distortion of the side-scan sonar image due to the different physical resolutions at different distances. S3: Perform image segmentation based on the improved DeepLabV3+ network on the corrected side-scan sonar image to extract the highlight and shadow areas of the target of interest in the side-scan sonar image; Build a sonar image segmentation model based on the improved DeepLabv3+ network, replacing the original Xception series network with MobilenetV2 as the backbone feature extraction network; Strengthen feature network extraction in Encoder and Decoder, use cross entropy to calculate loss, and introduce Dice loss indicator to evaluate semantic segmentation results; S4: After obtaining the segmented areas, the imaging principle and geometric relationship of the side scan sonar are used to match the highlight area and shadow area of the same target; S5: Build a sonar target detection model based on the YOLOv5s network, feed the rectified and segmented images into the network for training, and use the trained model for detection and recognition of real sonar targets; S6: Ablation experiment setup to verify the effectiveness of shadow areas in sonar images and the effectiveness of sonar image segmentation.

2. The side-scan sonar target detection method combining accurate image segmentation and target shadow information according to claim 1, characterized in that: The step S1 comprises the following steps: S11: Finding the seabed line: The bottom line is found by referring to the attitude information of the mobile platform, and then accurately searching the bottom line left and right. Since the side scan sonar is divided into port and starboard, the following rules are used when converting from height to image pixel position: line orig =N s -(altitude*N s / range) (1) Where line orig Indicates the initial seabed position point in the sonar image, altitude indicates the height information, range indicates the working range of the sonar, N s Indicates the number of sampling points of a certain sound intensity data (ping(n)) obtained on a single side; then, within 50 pixels around the initial value, find the pixel corresponding to the maximum grayscale value, which is the sonar seabed line position point; S12: Calculate the width of the image area corresponding to the maximum towed fish depth in the image: From the starboard side, a complete side-scan sonar image is formed using N data points. The grayscale values converted from the sound intensity values at each point are defined as a sequence s(n,i), where n is the ping sequence number and i is the sequence number of each point in each data point. The seabed line position of the nth data point found by the above algorithm is a(n), so the width N of the image area corresponding to the maximum towed fish height in the image can be calculated according to the following formula: min : N min =min(N s -a(n)),n=1…N (2) S13: Calculate the average grayscale value of each ping section: After obtaining the seabed line position and the width of the image area corresponding to the maximum towed fish depth in the image, it is necessary to perform grayscale correction on all pixels in the area. First, the mean grayscale value of each ping section in the longitudinal direction of the image is calculated (the sonar image is fixed horizontally, and the vertical direction is calculated): S14: Get the image grayscale correction factor sequence: After obtaining the grayscale mean of the longitudinal ping section, we then perform the grayscale mean in the transverse direction of the sonar image, thereby finally obtaining the grayscale correction factor sequence for all pixels in the image area:

3. The side-scan sonar target detection method combining accurate image segmentation and target shadow information according to claim 1, characterized in that: The step S2 comprises the following steps: S21: Get the relationship between slope distance, horizontal distance and depth: The geometric relationship between slant range, horizontal distance and depth can be obtained based on the position of the sonar transducer and the direction of the sound wave, as follows: Where PlantRange represents the horizontal distance, SlantRange represents the slant distance, and TowfishAlt represents the height of the transducer from the seabed. TowfishAlt can be obtained by detecting the seabed line in the sonar image. S22: Obtain the corresponding point on the corrected horizontal range image to a point on the original slant range image based on the geometric relationship: The side scan sonar is divided into port and starboard. Assume that a point P(x2, y2) on the original slant range image corresponds to a point P(x1, y1) on the corrected horizontal range image. Based on the above geometric relationship, the specific correspondence between the horizontal range point and the slant range point can be obtained as follows: Port resolution correction factor: Starboard resolution correction factor: Res represents the image resolution, and width represents the image width.

4. The side-scan sonar target detection method combining accurate image segmentation and target shadow information according to claim 1, characterized in that: The step S3 comprises the following steps: S31: Build segmentation network model: S32: Create a sonar image dataset and perform model training: After the model was modified, a sonar image dataset was created and processed accordingly. The purpose of the sonar image segmentation part is to segment the target highlight area formed by the active acoustic wave echo and the shadow area formed by the object occlusion, without distinguishing the target type represented by each area. Therefore, in the segmentation task, only two types of data sets are labeled: highlight area (light) and shadow area (dark). After experimental collection, 488 side-scan sonar images were collected for training, 71 for validation, and 136 for test. Each image in the test and validation sets was annotated with the target bright area and shadow area, and then sent to the network. Then, the category parameters, backbone model, and pre-trained weights in the network were modified to finally complete the training of the sonar image segmentation model. S33: Using the trained model, the sonar image to be segmented is segmented into the target bright area and the acoustic shadow area to obtain the final image segmentation result: The weight file in the test code is modified to the weight with the best training results, and the category parameters are modified at the same time. After the modification, the target bright area and shadow area of the sonar image to be segmented in the test set are segmented to obtain the final segmentation result; after segmentation, the original sonar image is converted into a mask containing only three pixel values. The three values represent: target bright area, shadow segmentation and background, which effectively solves the problems of blurred sonar target edges, large noise and low resolution of sonar images.

5. The side-scan sonar target detection method combining accurate image segmentation and target shadow information according to claim 1, characterized in that: The step S4 is specifically as follows: Since the target highlight and shadow areas obtained by the segmentation network may have multiple highlights and shadow areas, it is necessary to use the imaging principle of the side-scan sonar to match the bright and shadow areas of the same target and select the target of interest; After obtaining the segmented area, according to the imaging principle of side-scan sonar images, it can be seen that the target shadow area is formed because the sound waves are blocked by the target and are not illuminated. Therefore, the target shadow area and the target highlight area should be in the same direction as the sound line, that is, the bright area and the shadow are on the same horizontal line, and the height of the area formed by the shadow and the bright area is consistent in the direction of vehicle movement; for targets located on the seabed, their shadows are immediately connected to the strong echo of the target.

6. The side-scan sonar target detection method combining accurate image segmentation and target shadow information according to claim 1, characterized in that: The step S5 comprises the following steps: S51: Constructing network model and algorithm: The YOLOv5s network structure consists of four main parts: Input, Backbone, Neck (multi-scale feature fusion), and Output. The Input includes Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling; the Backbone uses the Focus structure and CSP structure; the Neck adopts the FPN+PAN structure; the Output includes the Bounding Box loss function calculation and NMS non-maximum suppression. S52: Data collection, annotation, and dataset construction: First, the target highlight area and shadow area of the segmentation result are annotated with bounding boxes at the same time and annotated into the same target box. A total of 4 types of data are annotated, namely drowning people, mines, aircraft and shipwrecks. Then the annotated data are decomposed into training set, validation set and test set; the training set consists of 488 images, including 98 drowning people target images, 119 mine target images, 87 aircraft target images, and 184 shipwreck target images; the validation set consists of 71 images, including 17 drowning people target images, 18 mine target images, 13 aircraft target images, and 23 shipwreck target images; the test set consists of 136 images, including 13 drowning people target images, 35 mine target images, 23 aircraft target images, and 65 shipwreck target images; S53: Experimental setup and model training: Experimental environment configuration and parameter setting: The models of the present invention are trained and tested on the NVIDIA GeForce RTX 3080 graphics card based on the PyTorch deep learning framework, using Python as the programming language, modifying the target type, pre-training weight path, and network structure configuration file, setting epochs, batch-size, image size, initial learning rate, cyclic learning rate, learning rate momentum, weight decay coefficient, IoU loss coefficient, cls loss coefficient, and cls BCELoss positive sample weight and other parameters, thereby completing the construction and setting of the overall model; Model training: After completing the modification and setting of various parameters, the sonar image segmentation results obtained above are used as a dataset and fed into the network for model training. The model training results are viewed using the Tensorboard visualization tool. The mean values of the three loss functions (prediction box regression loss function, target detection loss function, and classification loss function) are obtained as a function of the number of iterations. The precision and recall rates are also plotted as a function of the number of iterations. The mean values of the average precision when the intersection over union (IoU) threshold is 0.5 and when the intersection over union (IoU) threshold is 0.5:0.95 are plotted as a function of the number of iterations. S54: Model testing and result analysis: After the network model training is completed, the weight file with the best training effect is selected to perform position regression of sonar targets on the test set data to complete the model testing. Finally, the model is evaluated from multiple aspects based on evaluation indicators such as precision, recall, PR curve, F1 score and average precision.

7. The side-scan sonar target detection method combining accurate image segmentation and target shadow information according to claim 1, characterized in that: The step S6 comprises the following steps: S61: Verify the effectiveness of sonar target shadow areas: First, only the target highlight area is annotated in the original sonar image. Note that image segmentation is not performed here, and the original sonar image is directly annotated. The annotated dataset is decomposed into a training set, a validation set, and a test set. The training set data is fed into the YOLOv5s network for model training. The validation set verifies the quality of the model. The test set predicts the results of the trained model. Finally, the Precision, Recall, PR curve, F1-score and other indicator curves and confusion matrix results are obtained. Secondly, the original sonar image is annotated with both the target highlight and shadow areas, and a series of index curves and confusion matrix results are obtained. Finally, the test results obtained based on the sonar target highlight area are compared with the test results based on the combined sonar target highlight and shadow areas to verify the effectiveness of the shadow area information in the acoustic image. S62: Verify the effectiveness of accurate image segmentation: Based on the results obtained in the above steps, the indicator curve results and confusion matrix obtained by simultaneously detecting the highlight and shadow areas of the target in the original sonar image are compared with the test results obtained by pre-segmenting the bright and shadow areas of the sonar target using the DeepLabv3+ network and then detecting them, thereby verifying the effectiveness of the pre-precise segmentation of the side-scan sonar image. S63: Verify the effectiveness of joint image segmentation and target shadow information: Based on the test results obtained above, the detection and recognition effect of the original target bright area without segmentation is compared with the index curve results and confusion matrix obtained by simultaneously combining the shadow area and image precise segmentation, thereby verifying the effectiveness of the combined image precise segmentation and target shadow information.

Images