An unmanned ship underwater target intelligent identification method based on an imaging sonar

Abstract

The application discloses an unmanned ship underwater target intelligent identification method based on an imaging sonar, and relates to the fields of unmanned operation platforms and target identification. The application is aimed at the requirements of unmanned ship navigation and tasks, considers the task characteristics of the unmanned ship, and specifically adopts a forward-looking sonar to perform target ranging positioning and accurate identification, and based on machine learning and, realizes thorough water area perception, target detection and accurate identification for underwater targets. The application can improve the underwater target identification performance of the unmanned ship.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned ship operation platform, digital image processing of target identification and classification, underwater target identification, etc. BACKGROUND

[0002] Unmanned Surface Vessels (USV) refers to those intelligent unmanned platforms that can be deployed or recovered by shore-based or surface vessels, and can realize autonomous or semi-autonomous navigation. In the field of marine environment detection, unmanned surface vessels have become a new detection means in the field of environmental detection integration and intelligentization. Therefore, unmanned surface vessels equipped with sonar are commonly used for real-time detection of underwater environment, which also makes them widely used in military and environmental exploration fields, such as mine detection, intelligence collection, monitoring, reconnaissance, offshore defense, and marine hydrological and meteorological information collection, seabed topography scanning and mapping, etc.

[0003] Unmanned surface vessels are generally small in size and shallow in draft, and the tasks they perform are mainly concentrated in shallow water areas. Underwater detection technology mainly includes optical imaging and sonar imaging. The underwater target recognition technology of unmanned surface vessels is different from the target recognition method on land or in the air. Due to the influence of environmental factors in shallow water areas, such as constantly changing water flow, uneven light, turbid water, etc., it is difficult to collect target image information. At the same time, it is greatly affected by the scattering effect of water medium on target information, resulting in blurred or distorted target information. The traditional underwater target recognition method of unmanned surface vessels is optical recognition, and the distance of optical imaging is generally a few meters to tens of meters, which is relatively close, and it is basically ineffective in turbid water. Sonar imaging has the advantages of long distance and strong penetration, and is particularly suitable for turbid water.

[0004] In recent years, deep learning has made great progress in target recognition and classification of digital image processing. Through the training of a large number of sample models, the accuracy of recognition can be improved. Due to the high cost of sonar target imaging experiments, and the difficulty in obtaining a large number of available actual images, direct training of CNN may cause overfitting, which may result in low sonar recognition accuracy. SUMMARY

[0005] To address the limitations of traditional underwater target optical recognition methods for unmanned surface vessels (USVs), such as ineffectiveness in near-field or murky waters, and the difficulty in obtaining training samples for sonar target imaging recognition leading to overfitting and low accuracy, this invention provides an intelligent underwater target recognition method for USVs based on imaging sonar. This method utilizes a multi-beam forward-looking imaging sonar mounted on the USV and combines a sonar image enhancement algorithm based on Curvelet-enhance transform with a convolutional neural network deep learning algorithm to highlight details such as edges and textures in the sonar image, improving sonar imaging quality and meeting the target recognition needs of common USV scenarios, thereby enhancing the accuracy of underwater target recognition.

[0006] The technical solution of this invention includes: utilizing a multi-beam forward-looking imaging sonar transmitting array mounted on an unmanned surface vessel to simultaneously transmit multiple acoustic beams with the same horizontal and vertical opening angles, scanning a three-dimensional fan-shaped region in front of the sonar to acquire underwater target sonar images from multiple environments and angles; performing Curvelet-enhance decomposition on the acquired original sonar images to improve the overall contrast of the sonar images and enhance the overall imaging quality; establishing a deep learning neural network, constructing convolutional layers, extracting local features of the images, alternately constructing convolutional and pooling layers to prevent data overfitting, and iterative training to improve sample recognition accuracy.

[0007] The advantages of this invention compared to the prior art are as follows:

[0008] Traditional target recognition methods for unmanned surface vessels (USVs) rely on optical recognition. However, traditional optical imaging is largely ineffective in murky waters due to the effects of waves and sunlight scattering. Images acquired by ordinary imaging sonar often suffer from high noise, severe distortion, blurred target edges, and low resolution. This invention employs an underwater intelligent target recognition method combining a sonar image enhancement algorithm based on Curvelet-enhance transform with a convolutional neural network deep learning algorithm. This improves the display effect of sonar images. Leveraging the multi-scale characteristics and good directionality of Curvelet transform, a piecewise nonlinear enhancement method based on Curvelet-enhance transform is proposed. This method effectively separates noise and edge information, improving contrast, suppressing noise, and highlighting details such as edges and textures in the sonar image. For applications involving sonar images such as rail fortifications, fences, and fishing nets, the overall and local characteristics of the sonar image are purposefully enhanced, amplifying the differences between features of different objects in the image, thus meeting the needs of intelligent underwater target recognition for USVs. This improves target recognition accuracy and environmental adaptability, and reduces the probability of false alarms and missed detections. Attached Figure Description

[0009] Figure 1This is the flow chart of the present invention. Detailed implementation manners

[0010] The present invention proposes an intelligent recognition method for underwater targets of an unmanned surface vehicle based on an imaging sonar, and the implementation process is as Figure 1 shown. The preferred implementation process can be described as follows:

[0011] Step 1: Use the forward-looking imaging sonar carried by the unmanned surface vehicle to collect sonar images, and then transmit the collected data to the information processing computer carried inside the unmanned surface vehicle to build an image and video database for training and verification.

[0012] Step 2: Perform Curvelet-enhance wavelet decomposition on the collected original sonar images to obtain low-frequency sub-band coefficients F0 and high-frequency sub-band coefficients F s,n , s represents the scale, and n represents the sub-band direction.

[0013] Step 3: Apply the S-shaped function algorithm to the normalized low-frequency sub-band coefficients to improve the overall contrast of the sonar images:

[0014]

[0015] In the formula, F0 and F0' are the low-frequency sub-band coefficients before and after enhancement respectively; M0 is the maximum value of the low-frequency coefficients; k1 is a constant (k1>1), and the S-shaped function algorithm is in the form of: y = vx / (x + exp(A - Bx)), where v is the maximum gray value.

[0016] Step 4: To avoid amplification of the noise coefficient, perform non-linear enhancement on each high-frequency sub-band coefficient and perform thresholding according to the set threshold. When abs(F s,n )≥T:

[0017]

[0018] When abs(F s,n )<T, F s ' ,n =0; in the formula, F s,n and F s ' ,n are the high-frequency sub-band coefficients before and after enhancement respectively; M s,n is the maximum value of the coefficients of this layer; k2 is a constant (k2>1); the non-linear gain function is:

[0019] f(x)=A[omicron(C(x - B)-omicron(-C(x + B))]×e (x-1)×D

[0020] In the formula C takes a value between 20 and 50, D generally takes a value between 1 and 0.05, parameter B is used to control the enhancement range, and parameters C and D are used to control the gain intensity.

[0021] Step 5: Perform Curvelet-enhance inverse transform on all subband coefficients to obtain the enhanced sonar image.

[0022] Step 6: Build a deep learning convolutional neural network on the enhanced sonar images obtained from Steps 1-5 using Curvelet-enhance transformation. Manually select and mark the desired feature regions in the enhanced sonar images.

[0023] Step 7: Create a convolutional layer. When the image is passed to the convolutional layer, set an m×n kernel. Each weight in the convolution kernel W is multiplied by the corresponding pixel x in the enhanced sonar image X, and then summed. The calculation formula is as follows:

[0024]

[0025] The output is generated by adding a scalar bias to the calculation result z. Convolutional layers can extract local features from the image through kernel filtering, thereby reducing the data size and computational cost.

[0026] Step 8: Perform max pooling on the region of interest (ROI). This transforms the image of the selected region into a feature vector of fixed size W×H. Here, the ROI refers to a rectangular window in the convolutional feature map. A search algorithm is used to calculate the segmented regions, each represented by a four-element vector (x, y, w, h), where (x, y) represents the top-left corner coordinates, and (w, h) represents the height and width of the rectangular window. The ROI pooling layer divides the region of size (w, h) into W×H grid sub-windows, each approximately (w / W)×(h / H). Then, the feature values ​​in each sub-window are max-pooled into the corresponding output network. After applying this operation to each feature channel, h is the same as in standard max pooling.

[0027] Step 9: Perform multi-task training, simultaneously using object classification and candidate region bounding box regression calculations as two parallel output layers. The first task outputs the probability distribution of each region of interest across K+1 classes, where K is the number of object classes plus the background class, and calculates the probability using the softmax function. The second task calculates the regression offset of the bounding box. The loss function for multi-task calculation is the joint training classification and candidate region bounding box regression calculation:

[0028] L(p,u,t u ,v)=Lcls (p,u)+λ[u≥1]+L loc (t u ,v)

[0029] Wherein, parameter u marks the candidate region content as the true category of the target, usually u≥1, if u=0 indicates that the region content is background; L cis (p,u)=-logP u L is the loss function corresponding to category u. loc (t u v) The loss function at the bounding box position is calculated using the smoothed L1 loss function, t u This is the bounding box predicted for category u; the square brackets [u≥1] indicate that if the condition within the box is u≥1, it is 1, otherwise it is 0; the parameter λ controls the balance between the two loss functions; since both loss functions are equally important, it is set to 1 in practice.

[0030] Step 10: Perform backpropagation of parameters on the convolutional neural network using gradient descent. During training, the entire dataset is fed into the network model for training, allowing the network to calculate iteratively updated gradient values ​​using all samples. By processing convolutional features through region-of-interest pooling layers and feeding the resulting features into two parallel computing tasks—classification and localization regression—the training of the target function is better achieved.

[0031] Step 11: Through multiple iterative calculations, the loss function value is made less than 0.1, thus obtaining the trained deep learning neural network.

[0032] Step 12: Use the enhanced sonar image database obtained in Steps 1-5 to repeatedly train the deep learning neural network obtained in Steps 6-10.

[0033] Step 13: Calculate the accuracy of the recognition results obtained after training. If the accuracy is low, repeat Step 1 to increase the diversity of sonar samples. Repeat Steps 1-10 for the sonar samples acquired by the forward-looking multibeam imaging sonar mounted on the unmanned surface vessel.

[0034] Step 14: Display the target identification type value on the front end of the unmanned surface vessel's display.

Claims

1. A method for intelligent underwater target identification of unmanned surface vessels based on imaging sonar, characterized in that: Step 1: Use the forward-looking imaging sonar on the unmanned surface vessel to acquire sonar images, and then transmit the acquired data to the information processing computer on the unmanned surface vessel to build an image and video database for training and verification. Step 2: Perform Curvelet-enhance decomposition on the acquired original sonar image to obtain the low-frequency subband coefficient F0 and the high-frequency subband coefficient F. s,n s represents the scale, and n represents the sub-band direction; Step 3: Apply the S-shape function algorithm to the normalized low-frequency subband coefficients: In the formula, F0 and F0' are the low-frequency subband coefficients before and after enhancement, respectively; M0 is the maximum value of the low-frequency coefficient; k1 is a constant, k1>1, where the S-shaped function algorithm is in the form of: y=vx / (x+exp(A-Bx)), where v is the maximum gray value; Step 4: Perform nonlinear enhancement processing on each high-frequency subband coefficient, and simultaneously perform thresholding processing according to a set threshold. When abs(F s,n When )≥T: When abs(F s,n ) < T: F' s，n ＝0 In the formula F s,n and F′ s,n These are the coefficients of each high-frequency subband before and after enhancement; M s,n It is the maximum value of the coefficients of this layer; k2 is a constant, k2 > 1; where the nonlinear gain function is: f(x)=A[omicron(C(x-B)-omicron(-C(x+B))]×e (|x|-1)×D ; In the formula: 0 < B < 1; C takes values ​​between 20 and 50, D takes values ​​between 1 and 0.05, B is the control enhancement range parameter, and C and D are the control gain strength parameters; Step 5: Perform Curvelet-enhance inverse transform on all subband coefficients to obtain the enhanced sonar image; Step 6: Build a deep learning convolutional neural network on the enhanced sonar images after Curvelet-enhance transformation in steps 1-5; mark the feature regions to be identified in the enhanced sonar images through manual screening. Step 7: Create a convolutional layer. When the image is passed to the convolutional layer, set an m×n kernel. Each weight in the convolution kernel W is multiplied by the corresponding pixel x in the enhanced sonar image X, and then summed. The formula is as follows: The output is generated by adding a scalar bias to the calculation result z; the convolutional layer extracts local features in the image through the filtering of the convolutional kernel; Step 8: Perform max pooling on the region of interest (ROI) to convert the image of the filtered region into a feature vector of fixed size W×H. The ROI is a rectangular window in the convolutional feature map. Select the segmentation region using a search algorithm. Each ROI is represented by a four-element vector (x,y,w,h), where (x,y) identifies the top-left corner coordinates, and (w,h) represents the height and width of the rectangular window. The ROI pooling layer divides the region of size (w,h) into W×H grid sub-windows, each window being (w / W)×(h / H). Then, the feature values ​​in each sub-window are max pooled into the corresponding output network. After performing this operation on each feature channel, h is the same as in standard max pooling. Step 9: Perform multi-task training, simultaneously using object classification and candidate region bounding box regression calculations as two parallel output layers. The first task outputs the probability distribution of each region of interest across K+1 classes, where K is the number of object classes plus the background class, and calculates the probability using the softmax function. The second task calculates the regression offset of the bounding box. The loss function for multi-task calculations is the joint training classification and candidate region bounding box regression calculation: L(p,u,t u ,v)=L cls (p,u)+λ[u≥1]+L loc (t u ,v) Wherein, parameter u marks the candidate region content as the true category of the target, u = 0 indicates that the region content is background; L cls (p,u)=-logP u Let L be the loss function corresponding to category u. loc (t u v) The loss function at the bounding box position is calculated using the smoothed L1 loss function, t u This is the bounding box predicted by category u; the square brackets [u≥1] indicate that if the condition inside the box is u≥1, it is 1, otherwise it is 0; the parameter λ controls the balance between the two loss functions; Step 10: Perform backpropagation of parameters on the convolutional neural network using the gradient descent method; during training, feed the entire dataset into the network model for training and learning, allowing the network to calculate the iteratively updated gradient values ​​using all samples; process the convolutional features by pooling layers in the region of interest, and feed the resulting features into two parallel computing tasks for training: classification and localization regression. Step 11: Through multiple iterative calculations, the loss function value is made less than 0.1, resulting in a trained deep learning neural network; Step 12: Use the enhanced sonar image database obtained in Steps 1-5 to repeatedly train the deep learning neural network obtained in Steps 6-10; Step 13: Calculate the accuracy of the recognition results obtained after training; if the accuracy is low, repeat step 1 to increase the diversity of sonar samples; repeat steps 1-10 for the sonar samples collected by the forward-looking multibeam imaging sonar carried by the unmanned surface vessel. Step 14: Display the target identification type value on the front end of the unmanned surface vessel's display.

Images