A synthetic aperture sonar image target classification method and system

Abstract

The application provides a synthetic aperture sonar image target classification method and system, the method comprises the following steps: (1) obtaining an image training set and a verification set; (2) improving a standard convolution module in a deformable convolution module into a depth separation convolution, obtaining a depth separation deformable convolution; (3) adding the aforementioned depth separation deformable convolution module to an optimized residual network, obtaining a target classification model; (4) training the model by using SAS images, obtaining a target classification result. The application overcomes the defects of manual feature extraction, solves the problem of low classification accuracy caused by irregular target morphology and scale difference, has small resource consumption and low calculation complexity, and can be quickly applied to actual devices, thereby providing reliable performance for underwater automatic target recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of underwater acoustic signal processing, and in particular to a synthetic aperture sonar image target classification method and system. BACKGROUND

[0002] In recent years, countries around the world have used underwater unmanned platforms to promote the smooth progress of marine research. Sonar images can provide more intuitive information about underwater scenes, so they have become an important way to achieve automatic target recognition (ATR). The rapid development of synthetic aperture imaging technology has made it possible to obtain high-resolution underwater images. Using sonar images to achieve target classification is an important approach to sonar intelligence research, which has attracted widespread attention from scholars at home and abroad.

[0003] In the past few decades, research on underwater target classification has often relied on manually designed and extracted features. However, manually designed features require a high level of expertise, and some critical information is inevitably lost during the feature extraction process. In addition, for synthetic aperture sonar, due to its range imaging principle, complex and uncertain marine environment, speed, and system parameter settings, it is prone to poor image quality, blurred target edges, and irregular geometric shapes. In extreme cases, the image may be distorted. In this case, the target features extracted by manual methods will no longer be stable, and such image recognition methods lack robustness and have limited generalization ability.

[0004] According to existing research, deep learning can learn effective features from training data for different tasks, and combine the feature extraction and classification steps of traditional methods into an end-to-end classification model, which has high efficiency and is widely used. However, the convolution kernels used in the above convolutional neural network are fixed in shape, and the sampling process can only be based on the fixed position of the feature map, which has inherent defects in modeling complex unknown shape transformations. Deformable convolution adds a parallel network to the regular convolution operation, and the convolution kernel can adaptively change according to the target shape, but it consumes too much computing resources.

[0005] In summary, in the task of underwater target classification, there is an urgent need for a target classification method that can adapt to various image qualities to improve the accuracy and efficiency of subsequent target classification. SUMMARY

[0006] The present application aims to overcome the defects of the existing method that the sampling process can only be based on the fixed position of the feature map, or the high consumption of computing resources, and proposes a residual network synthetic aperture sonar image target classification method and system.

[0007] In order to achieve the above object, the application provides a synthetic aperture sonar image target classification method, which comprises the following steps:

[0008] The collected underwater synthetic aperture sonar image is pretreated;

[0009] The pretreated underwater synthetic aperture sonar image is input into the established and trained target classification model to obtain the corresponding target classification result.

[0010] The target classification model is a residual network model added with a depth-separated deformable convolution module and is obtained by training with a supervised learning method.

[0011] As an improvement of the above method, the pretreatment of the collected underwater synthetic aperture sonar image is to divide the original image into an image set with equal size.

[0012] As an improvement of the above method, the input of the target classification model is an image set with equal size, and the output is a classification number of the image.

[0013] The specific structure of the target classification model is as follows:

[0014] The ResNet50 network is used as a basic model, and three 3x3 size convolution layers are used to replace the original 7x7 convolution of the ResNet50 network.

[0015] The depth-separated deformable convolution module is introduced into the feature extraction process of the residual network: the improved depth-separated deformable convolution is added at the position of the third convolution layer of the input layer and after the residual unit, and the target is represented by combining the traditional deep features and the deformable deep features.

[0016] The ReLU nonlinear activation function is used in each convolution layer and residual learning unit for processing, and a BN layer is added before the activation function, and the calculation process is as follows:

[0017] ReLU(r)=max(r,0)

[0018] Wherein, r is the function argument.

[0019] As an improvement of the above method, the improved depth-separated deformable convolution model is as follows:

[0020] The standard two-dimensional convolution kernel in the deformable convolution is used to sample the input image set in the regular grid R in the form of a sliding window to generate a feature map z; wherein, represents the sampling grid of m x m convolution with a step of 1, and m is an odd number greater than 1;

[0021] The standard two-dimensional convolution in deformable convolution is decomposed into a depth convolution and a 1x1 point-wise convolution, forming a depthwise separable deformable convolution;

[0022] In the depthwise separable deformable convolution, each point in the sampling grid R increases the offset amount {ΔP n |n=1,…,N},N=|R|, for each position P0 in the output feature map z, the feature value z(P0) is:

[0023]

[0024] where w(P n ) represents the convolution kernel weight of the sampling position; z(P0+P n +ΔP n ) represents the input feature value of the sampling offset position; ΔP n represents the offset amount of the sampling point position; P n represents all sampling positions in the receptive field region;

[0025] The pixel value of each floating-point coordinate is obtained by bilinear interpolation, that is:

[0026]

[0027] where p is any position on the feature map, z(q) is the pixel value at the four adjacent integer coordinates around p, and G(·,·) is the weight corresponding to the four integer coordinates, which is two-dimensional and can be divided into two one-dimensional kernels:

[0028] G(q,p)=g(q a ,p a )g(q b ,p b )

[0029] g(q a ,p a ) and g(q b ,p b ) are two one-dimensional kernels, and a and b represent two directions respectively.

[0030] As an improvement of the above method, the method further comprises a training step of the target classification model; specifically comprising:

[0031] Step 1) Obtain an image training set and a verification set, give a label vector Y corresponding to a target in a synthetic aperture sonar image X, pre-process the image to obtain an image set with the same size and the respective corresponding label vector;

[0032] Step 2) Train and test the target classification model using the image set to obtain the target classification result.

[0033] As an improvement of the above method, the step 1) specifically comprises:

[0034] Step 1-1) Given an original synthetic aperture sonar image X and a label vector Y corresponding to the target in the image;

[0035] Step 1-2) The original image is segmented to obtain a set of images x1, x2,..., xK of the same size containing K target categories n and the corresponding label vectors y1, y2,..., yK respectively; n n is the number of image sets;

[0036] Step 1-3) Randomly select N1 images of C1 target categories and their labels to form a training data set D train , and the remaining N2 images of C2 target categories and their labels form a training data set D test , wherein C1+C2=K,

[0037] As an improvement of the above method, the step 2) specifically comprises:

[0038] Step 2-1) When constructing the network, the parameters of each layer are randomly initialized with Gaussian distribution G(μ,σ 2 ), wherein the mean μ=0 and the standard deviation σ=1;

[0039] Step 2-2) Select Adam optimization algorithm to update network parameters, and set the learning rate to 0.0002;

[0040] Step 2-3) Use cross-entropy loss function to calculate the loss value, and the calculation formula is:

[0041]

[0042] Wherein, k represents the number of training set samples, s represents the number of categories, q(g i,j ) represents the category of the target, and p(g i,j ) represents the probability of the predicted category.

[0043] The application also provides a synthetic aperture sonar image target classification system, which classifies the collected underwater synthetic aperture sonar image based on the above method, and the system comprises:

[0044] A preprocessing module is configured to preprocess the collected underwater synthetic aperture sonar image;

[0045] A target classification module is configured to input the preprocessed underwater synthetic aperture sonar image into the established and trained target classification model to obtain the corresponding target classification result.

[0046] Compared with the prior art, the advantages of the present application are:

[0047] 1. In the imaging process of synthetic aperture sonar, due to environmental factors and slant range imaging principles, the image may have noise interference and edge blur, the shape of the target in the image is irregular, the scale and morphology are quite different, and in extreme cases, distortion may occur, which makes the classification accuracy low. The convolution kernel of deformable convolution can adaptively change according to the target shape, so deformable convolution is introduced into the underwater target classification of synthetic aperture sonar image, which ingeniously solves this problem.

[0048] 2. The deformable convolution module calculates the offset by adding a parallel network, which is completed using standard convolution calculation. Considering the resource consumption caused by the high calculation cost, the present application uses depthwise separable convolution instead of the original standard convolution, which reduces the network parameters of the module and the computational complexity, and is extremely friendly to hardware devices with small memory resources.

[0049] 3. The deep neural network has more non-linear mapping structures than the shallow neural network, and the non-linear expression ability is stronger with the increase of the network depth, which is more beneficial to the acquisition of abstract features. The present application uses ResNet as the basic network, and the residual module forms a jump connection between the convolution layers, realizes the identity mapping of the input and output, and can effectively avoid the gradient vanishing problem caused by the increase of the depth of the network during the back propagation process.

[0050] 4. By optimizing the input layer of the network, more effective features can be preserved while increasing the network depth. Improved depthwise separable deformable convolution is added after the input layer and the residual unit, which improves the classification ability of the model for the target and has good robustness, and can be more quickly applied to practical devices, providing technical support for underwater target classification. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 It is a residual network synthetic aperture sonar image target classification method overall flow chart;

[0052] Figure 2 It is a depthwise separable deformable convolution principle diagram. The input feature map is calculated for offset by a depthwise convolution (Depthwise Conv) and a pointwise convolution (Pointwise Conv), respectively, and irregular sampling is completed.

[0053] Figure 3 It is a residual network model diagram of depthwise separable deformable convolution, including convolution layer, depthwise separable deformable convolution, residual unit and other modules.

[0054] Figure 4The accuracy and cost function curve of model training. DETAILED DESCRIPTION

[0055] The technical solutions of the present application will be described in detail below in combination with the drawings and specific examples.

[0056] Embodiment 1 of the present application, as shown in the drawings, proposes a synthetic aperture sonar image target classification method, the implementation of which includes the following steps: Figure 1

[0057] Step 1: Obtain the image training set and the verification set, and the specific implementation is as follows:

[0058] Step 1-1) Given the original synthetic aperture sonar image X and the label vector Y corresponding to the target in the image;

[0059] Step 1-2) The original image is segmented and processed in size to obtain an image set x1, x2,..., xK containing K target categories with the same size, and their respective label vectors y1, y2,..., yK. n n The image size is unified to 128x128 pixels; n is the number of image sets.

[0060] Step 1-3) Randomly select N1 images of C1 target and their labels to form the training data set D train , and the remaining N2 images of C2 target and their labels form the training data set D test , where C1+C2=K,

[0061] Step 2: The standard convolution module in the deformable convolution module is improved to a depth separation convolution to obtain a depth separation deformable convolution, and the specific implementation is as follows:

[0062] Step 2-1) The standard two-dimensional convolution kernel is sampled in the regular grid R on the input image set x in the form of a sliding window to generate a feature map z, and the size of the feature map z is denoted as D F ×D F ×N (where D F ×D F is the size of the feature map, and N is the number of channels). Wherein, represents the sampling grid of m*m convolution with a step size of 1, m is an odd number greater than 1, generally taking 3, 5, 7,...; for example, R={(-1,-1),(-1,0),…,(0,1),(1,1)} represents the sampling grid of 3*3 convolution with a step size of 1, for each position P0 in the output feature map z, the feature value z(P0) is

[0063] ​​

[0064] where w(P n ) is the convolution kernel weight of the sampling position, z(P0+P n ) is the input feature value of the sampling position, P n is all the sampling positions in the receptive field area, and R is the receptive field area.

[0065] Step 2-2) decompose the traditional convolution in deformable convolution into a depthwise convolution and a 1x1 pointwise convolution, assuming that the input image size is D F x D F M1 channel image, using a convolution kernel with a size of D K x D K to obtain a new set of M2 channel output feature maps, then the ratio of the calculation amount of depthwise separable convolution to that of standard convolution is:

[0066]

[0067] Step 2-3) as shown in Figure 2 , Figure 3 , in the depthwise separable deformable convolution, each point in the sampling grid R is increased by the offset {ΔP n | n = 1, …, N}, N = |R|, and the above formula becomes

[0068]

[0069] where z(P0+P n + ΔP n ) is the input feature value of the sampling offset position, ΔP n is the offset of the sampling point position, and the output result after the depthwise separable convolution is D F x D F x 2N, that is, the size of the output feature map is unchanged, and 2N indicates that there are two directions of offset.

[0070] Step 2-4) obtain the pixel value of each floating point coordinate by bilinear interpolation, obtain the accurate pixel value and can perform gradient back propagation, that is,

[0071]

[0072] where p is an arbitrary position p0+p n + Δp n (the result may have a decimal), z(q) is the pixel value at the four adjacent integer coordinates around p, G(·,·) is the weight corresponding to the four integer coordinates, which is two-dimensional and can be divided into two one-dimensional kernels, such as:

[0073] G(q, p) = g(q a , pa )g(q b ,p b (8)

[0074] g(q a ,p a ) and g(q b ,p b ) represents two one-dimensional kernels, where a and b represent two directions respectively.

[0075] Step 3: Construct a synthetic aperture sonar image classification model based on depth-separated deformable convolution, as detailed below:

[0076] Step 3-1) Select the ResNet50 network as the base model. The residual module in the network enables skip connections between convolutional layers, achieving an identity mapping between input and output, effectively avoiding the gradient vanishing problem caused by backpropagation. In the basic residual learning unit, x′ is the input, F(x′) is the residual mapping, and the output of the residual unit is:

[0077] H(x′)=F(x′)+x′ (9)

[0078] When the residual F(x′) = 0, the residual learning unit achieves the identity mapping, and the output of the deep layer L is:

[0079]

[0080] x′ L H(x′) represents the input of the Lth layer. L ) represents the output of the Lth layer, x′ l Let L be the feature value of the l-th layer, and L represent the number of network layers.

[0081] Step 3-2) Downsample the input image and then input it into the residual unit. Replace the original 7×7 convolution of the ResNet50 network with three 3×3 convolutional layers. On the one hand, this ensures that the optimized network has the same receptive field as the original network layer, and on the other hand, it increases the network depth to further extract deeper semantic information.

[0082] Step 3-3) The depth separable deformable convolution module is introduced into the residual network during the feature extraction process, and improved depth separable deformable convolution is added after the third convolutional layer at the input layer and the residual unit. Based on the traditional deep features, the target is represented by combining the deformable deep features, which improves the robustness of the model. The size and shape of the convolution kernel can be adaptively adjusted according to the size and shape of the target in the image, the sampling of the offset is completed, and the sampling points are concentrated through the bilinear interpolation algorithm, so that the robust features of different scales and directions are efficiently extracted, the discrimination performance of the target is enhanced, and the model has better applicability and generalization ability in the underwater target classification task.

[0083] Step 3-4) In order to reduce the overfitting of the model and the training parameters, ReLU nonlinear activation function is used in each convolutional layer and residual learning unit to enhance the nonlinear expression ability. In order to speed up the convergence of the network, prevent gradient explosion, and increase the prediction accuracy of the model, a BN (Batch Normalization) layer is added before the activation function, and the calculation process is as follows:

[0084] ReLU(r)=max(r,0) (11)

[0085] Step 4: The target classification model is trained and tested using the image dataset to obtain the target classification result, and the specific implementation is as follows:

[0086] Step 4-1) When constructing the network, the parameters of each layer are randomly initialized with Gaussian distribution G(μ,σ 2 ), where the mean μ=0 and the standard deviation σ=1.

[0087] Step 4-2) The Adam optimization algorithm is selected to update the network parameters, so that the error between the output image and the label converges to the best, and the learning rate is set to 0.0002.

[0088] Step 4-3) The cross-entropy loss function (categorical cross entropy) is used to calculate the loss value, and the calculation formula is as follows:

[0089]

[0090] where k represents the number of training set samples, s represents the number of categories, q(g i,j ) represents the category of the target, and p(g i,j ) represents the probability of the predicted category.

[0091] Step 4-4) Comparative experiments are conducted using different models, and the classification effect is measured by various evaluation indicators.

[0092] As embodiment 2 of the present application, the present application also provides a synthetic aperture sonar image target classification system, which classifies the collected underwater synthetic aperture sonar image based on the above method, and the system comprises:

[0093] a preprocessing module, configured to preprocess the collected underwater synthetic aperture sonar image;

[0094] a target classification module: configured to input the preprocessed underwater synthetic aperture sonar image into the established and trained target classification model to obtain the corresponding target classification result.

[0095] The technical effects of the present application will be further described below in combination with a simulation experiment:

[0096] Conditions and contents of the simulation experiment:

[0097] The simulation experiment platform is: Windows system, Keras environment network built based on TensorFlow background, the experimental computer CPU is i7-10750H, and the GPU is RTX2070. The simulation experiment data set includes 924 spherical, cylindrical and circular cone target images.

[0098] In order to verify the classification performance of the method on underwater targets, 30% of the data in the data set are randomly extracted as the training set to train the model, and the remaining data are used as the verification set to test the performance of the model. The Adam method is used as the optimization algorithm, the learning rate is set to 0.0002, the batch size of the picture sent into the network is 16 each time during the training process, the iteration number is 100, the loss value is calculated using the categorical cross entropy loss function, and the training curve can be seen Figure 4 . Several representative network models for underwater target classification at the present stage are selected for comparison, including VGG, UNet and Light-ResNet, each method is subjected to 10 random experiments, and the average value of the experimental results is calculated. Table 1 shows the classification accuracy and model complexity of several network models on the data set in this paper:

[0099] Table 1 Classification performance of different models / %

[0100]

[0101] As can be seen from Table 1, the classification accuracy of the model proposed in the application is higher than that of other network models. The classification accuracy of the VGGNet model is as high as 94.5%, only lower than the model proposed in the application, but the floating point operation amount is too large, and the complexity of the model is high; the UNet network has a small parameter amount, but has a large calculation amount, low operation efficiency and accuracy, and the accuracy is only 86.0%; the parameter amount of the Light-ResNet network is reduced compared with the original network, but the classification accuracy is greatly reduced.

[0102] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the examples, those skilled in the art should understand that modifications or equivalent replacements of the technical solutions of the present application do not deviate from the spirit and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A synthetic aperture sonar image target classification method, the method comprising: preprocessing the collected underwater synthetic aperture sonar image; inputting the preprocessed underwater synthetic aperture sonar image into a target classification model established and trained to obtain a corresponding target classification result; the target classification model is a residual network model added with a depth-separated deformable convolution module and is trained by a supervised learning method; the input of the target classification model is an image set with equal size, and the output is the classification number of the image; the specific structure of the target classification model is: using a ResNet50 network as a basic model, using three 3x3 size convolution layers instead of the original 7x7 convolution of the ResNet50 network; introducing a depth-separated deformable convolution module into the feature extraction process of the residual network: adding an improved depth-separated deformable convolution at the position of the third convolution layer of the input layer and after the residual unit, combining the deformable depth features on the basis of the traditional depth features to represent the features of the target; using a ReLU nonlinear activation function for processing in each convolution layer and residual learning unit, and adding a BN layer before the activation function, and the calculation process is: ； wherein r is a function argument; the improved depth-separated deformable convolution model is as follows: sampling a regular grid R on the input image set with a standard two-dimensional convolution kernel in deformable convolution in the form of a sliding window ; wherein represents a stride of 1 m m sampling grid of the convolution, m is an odd number greater than 1;​ the standard two-dimensional convolution in the deformable convolution is decomposed into a depth convolution and a 1x1 point-by-point convolution to form a depth-separated deformable convolution.

2. The synthetic aperture sonar image target classification method according to claim 1, wherein, the preprocessing of the collected underwater synthetic aperture sonar image is to divide the original image into an image set with equal size.

3. The synthetic aperture sonar image target classification method of claim 1, wherein, the improved depth-separated deformable convolution model further comprises: In deep separable deformable convolution, each point in the sampling grid R increases the offset by For each position in the output feature map z whose eigenvalues are: ； wherein, denotes the convolution kernel weight for the sampling position; denotes the input feature value for the sampling offset position; denotes the offset for the sampling point position; denotes all sampling positions in the receptive field region; obtaining the pixel value of each floating point coordinate by bilinear interpolation, that is: ； wherein, is the pixel value at the integer coordinate, is are the pixel values at the four integer coordinates surrounding, are the weights corresponding to the four integer coordinates, which is two-dimensional and can be separated into two one-dimensional kernels: ； and are two one-dimensional kernels, a and b denote two directions, respectively.

4. The synthetic aperture sonar image target classification method of claim 1, wherein, the method further comprises a training step of the target classification model; specifically comprising: Step 1) obtaining an image training set and a verification set, giving a label vector Y corresponding to the target in the synthetic aperture sonar image X, preprocessing the image to obtain an image set with the same size and the corresponding label vector of each image; Step 2) training and testing the target classification model using the image set to obtain the target classification result.

5. The synthetic aperture sonar image target classification method of claim 4, wherein, Step 1) specifically comprises: Step 1-1) giving an original synthetic aperture sonar image X and a label vector Y corresponding to the target in the image; Step 1-2) The original image is segmented to obtain a set of images of the same size containing K target categories and respective corresponding label vectors ; n is the number of image sets Step 1-3) Randomly select Class-targeted The training data set is composed of the images and their labels , the remaining Class-targeted The training data set is composed of the images and their labels wherein , .

6. The synthetic aperture sonar image target classification method of claim 5, wherein, Step 2) specifically comprises: Step 2-1) Gaussian distribution of random initialization of parameters for each layer at the time of constructing the network, where mean , standard deviation ;​ Step 2-2) selecting the Adam optimization algorithm to update the network parameters, and setting the learning rate to 0.0002; Step 2-3) using a cross-entropy loss function to calculate the loss value, and the calculation formula is: ； wherein, k represents the number of training set samples, s represents the number of classes, denotes the class of the target, denotes the probability of the predicted class.

7. A synthetic aperture sonar image target classification system, based on the method of one of claims 1 to 6, for classifying acquired underwater synthetic aperture sonar images, characterized in that the system comprises: a preprocessing module for preprocessing the collected underwater synthetic aperture sonar image; a target classification module: for inputting the preprocessed underwater synthetic aperture sonar image into a target classification model established and trained to obtain a corresponding target classification result.

Images