Synthetic aperture sonar image target detection method based on pulse neural network

Abstract

The application provides a synthetic aperture sonar image target detection method based on a pulse neural network. The method comprises the following steps: step 1: constructing a deep convolutional neural network, training the deep convolutional neural network by using a synthetic aperture radar (SAR) labeled image to obtain a target detection model DCN SAR for the SAR image; step 2: converting the deep convolutional neural network into a pulse neural network by using a pulse neuron, taking the weight of the target detection model DCN SAR as the initial weight of the pulse neural network, training the pulse neural network by using a synthetic aperture sonar (SAS) labeled image to obtain a target detection model DSN SAS for the SAS image; and step 3: acquiring a SAS image to be detected, performing sliding detection on the SAS image to be detected by using the target detection model DSN SAS , and positioning and labeling targets in the SAS image to be detected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater target detection of sonar images, and particularly relates to a synthetic aperture sonar image target detection method based on a spiking neural network. BACKGROUND

[0002] SAS can realize high-resolution imaging in a wide swath, and the target recognition rate of SAS images can be greatly improved compared with traditional sonars, which has irreplaceable advantages for important tasks such as underwater search and rescue, buried object detection, military target detection, target classification and identification.

[0003] In recent years, convolutional neural networks have been widely used in large-scale natural image datasets (such as ImageNet, COCO, etc.), but there is a lack of research on the application of target detection and recognition in the field of underwater acoustic images. Due to the lack of target detection and classification datasets for sonar images, and the problems of sample scarcity and class imbalance of underwater target samples, there is an urgent need for a target detection method suitable for synthetic aperture sonar images to address the problems of small amount of SAS image data, low detection accuracy of existing target detection models for underwater targets, and system energy consumption in mobile environments. SUMMARY

[0004] In view of the low detection accuracy of existing target detection models for underwater targets and the system energy consumption in mobile environments, the present application provides a synthetic aperture sonar image target detection method based on a spiking neural network.

[0005] The synthetic aperture sonar image target detection method based on a spiking neural network provided by the present application comprises:

[0006] Step 1: Construct a deep convolutional neural network, train the deep convolutional neural network using synthetic aperture radar (SAR) labeled images to obtain a SAR image-oriented target detection model DCN SAR ;

[0007] Step 2: Convert the deep convolutional neural network to a spiking neural network using spiking neurons, use the weights of the target detection model DCN SAR as the initial weights of the spiking neural network, and train the spiking neural network using synthetic aperture sonar (SAS) labeled images to obtain a SAS image-oriented target detection model DSN SAS ;

[0008] Step 3: Obtain a SAS image to be detected, and use the target detection model DSN SAS to perform sliding detection on the SAS image to be detected, and locate and label the targets therein.

[0009] Further, step 3 specifically comprises:

[0010] Step 3.1: obtaining a to-be-detected SAS image, and performing visual saliency calculation on the to-be-detected SAS image to obtain a corresponding saliency map;

[0011] Step 3.2: setting a saliency value threshold, intercepting a region greater than the saliency value threshold in the saliency map, and taking the region as a to-be-detected target region;

[0012] Step 3.3: using the target detection model DSN SAS performing sliding detection on the to-be-detected target region.

[0013] Further, in step 1, the deep convolutional neural network comprises a backbone feature extraction network, a strengthened feature extraction network, a classifier and a regressor.

[0014] Further, in step 1, before training the deep convolutional neural network, further comprising: modifying a max-pooling layer with a step length of 2 into a convolutional layer with the same length, deleting a max-pooling layer with a step length of 1, modifying an up-sampling layer into a deconvolutional layer, modifying a Leaky_ReLU activation function into a ReLU activation function, and modifying a batch normalization layer into a fusion layer composed of a convolutional layer and a batch normalization layer.

[0015] Further, in step 2, the deep convolutional neural network is converted into a spiking neural network by using a spiking neuron, specifically comprising: replacing a ReLU activation function in the deep convolutional neural network with a spiking neuron, and setting an average pooling layer in the deep convolutional neural network into a spatial down-sampling layer.

[0016] Further, in step 3.1, the visual saliency calculation is performed by using an LC algorithm, an HC algorithm, an RC algorithm or a GR algorithm.

[0017] The present application has the following beneficial effects:

[0018] The pulse neural network-based synthetic aperture sonar image target detection method of the present application utilizes the characteristic that SAS is very similar to SAR in imaging principle, first trains a deep convolutional neural network using a SAR image, then migrates the weight of the learned deep convolutional neural network model to the pulse neural network converted by the deep convolutional neural network model using the transfer learning technology, thereby realizing the pre-training of the pulse neural network; on this basis, the pulse neural network is further trained using a SAS image, so that even in the case of a small amount of SAS image samples, the accuracy of SAS image target detection can be improved; at the same time, the problems of low accuracy, high energy consumption and long detection time of large-format SAS images existing in the existing deep convolutional neural network in SAS image target detection can also be solved. The present application improves the accuracy of seabed sonar image target detection tasks and solves the problem of low energy consumption demand in practical applications, and can be well applied to unmanned autonomous mobile devices. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is one of the flowcharts of the pulse neural network-based synthetic aperture sonar image target detection method provided by the embodiments of the present application;

[0020] Figure 2 is the second flowchart of the pulse neural network-based synthetic aperture sonar image target detection method provided by the embodiments of the present application;

[0021] Figure 3 is the third flowchart of the pulse neural network-based synthetic aperture sonar image target detection method provided by the embodiments of the present application;

[0022] Figure 4 is an example diagram of underwater target detection results of SAS images using the method of the present application provided by the embodiments of the present application. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0024] Embodiment 1

[0025] As shown in Figure 1 , the embodiments of the present application provide a pulse neural network-based synthetic aperture sonar image target detection method, which comprises the following steps:

[0026] S101: Construct a deep convolutional neural network, train the deep convolutional neural network using synthetic aperture radar (SAR) labeled images to obtain a target detection model DCN for SAR images SAR ;

[0027] S102: Convert the deep convolutional neural network into a spiking neural network using spiking neurons, use the weights of the target detection model DCN SAR as the initial weights of the spiking neural network, train the spiking neural network using synthetic aperture sonar (SAS) labeled images to obtain a target detection model DSN for SAS images SAS ; as shown in Figure 2 .

[0028] S103: Obtain a SAS image to be detected, and use the target detection model DSN SAS to perform sliding detection on the SAS image to be detected, and locate and label targets therein.

[0029] The SAS image target detection method based on the spiking neural network provided by the embodiment of the present application utilizes the characteristic that SAS is very similar to SAR in imaging principle, first trains a deep convolutional neural network using SAR images, then migrates the weights of the learned deep convolutional neural network model to a spiking neural network converted from the deep convolutional neural network model using a transfer learning technology, thereby realizing pre-training of the spiking neural network; on this basis, the SAS image is further used to train the spiking neural network, so that the accuracy of SAS image target detection can be improved even in the case of a small amount of SAS image samples; at the same time, the problems of low accuracy, high energy consumption, and long detection time of large-format SAS images of the existing deep convolutional neural network in SAS image target detection can also be solved.

[0030] Embodiment 2

[0031] On the basis of the above-mentioned embodiment, in order to further reduce the problem of long detection time of large-format SAS images, the embodiment of the present application further provides a SAS image target detection method based on a spiking neural network, comprising the following steps:

[0032] S201: Construct a deep convolutional neural network, train the deep convolutional neural network using synthetic aperture radar (SAR) labeled images to obtain a target detection model DCN for SAR images SAR ;

[0033] S202: Convert the deep convolutional neural network into a spiking neural network using spiking neurons, use the weights of the target detection model DCN SARweight of the pulse neural network as initial weights of the pulse neural network, and training the pulse neural network by using SAS labeled images to obtain a target detection model DSN for SAS images SAS ;

[0034] S203: as shown in Figure 3 , obtain a SAS image to be detected, and perform visual saliency calculation on the SAS image to be detected to obtain a corresponding saliency map SaliencyMap;

[0035] Specifically, the LC algorithm, the HC algorithm, the RC algorithm or the GR algorithm can be used for visual saliency calculation.

[0036] S204: set a saliency value threshold, intercept a region in the saliency map greater than the saliency value threshold, and take the region as a target region to be detected, as shown in Figure 3 ;

[0037] S205: use the target detection model DSN SAS to perform sliding detection on the target region to be detected, position and label a position on the SAS image greater than a detection threshold, and realize detection of an underwater target, as shown in Figure 3 .

[0038] The SAS image target detection method based on the pulse neural network provided by the embodiment of the application uses the characteristic that the saliency value of the region where the target is located is high, performs saliency calculation on the SAS image to be detected, intercepts a region with a saliency value greater than a set threshold as a target region to be detected, and thus a large amount of background information region can be filtered in advance, and only the target region to be detected is subjected to sliding detection, so that the detection time of a large-area SAS image can be significantly reduced.

[0039] As an implementable manner, in each of the above embodiments, the deep convolutional neural network comprises a backbone feature extraction network, a strengthened feature extraction network, a classifier and a regressor.

[0040] Specifically, the classifier and the regressor are used for boundary box prediction, the boundary box corresponds to four parameters (x, y, w, h), (x, y) represents the center coordinates of the predicted boundary box, and w and h represent the width and height of the predicted boundary box, respectively. According to the intersection over union IoU of the predicted boundary box and the real boundary box, the maximum value is taken as the detection result of the current underwater target.

[0041] Preferably, in view of the subsequent need to convert the deep convolutional neural network into a spiking neural network, it is necessary to appropriately trim or modify the deep convolutional neural network before training, specifically: modifying the max pooling layer with a step size of 2 into a convolution layer with the same step size, deleting the max pooling layer with a step size of 1, modifying the up-sampling layer into a de-convolution layer, modifying the Leaky_ReLU activation function into a ReLU activation function, and modifying the batch normalization layer into a fusion layer composed of a convolution layer and a batch normalization layer.

[0042] As an implementable manner, the converting the deep convolutional neural network into a spiking neural network using spiking neurons specifically includes: replacing the ReLU activation function with a spiking neuron, and setting the average pooling layer as a spatial down-sampling layer.

[0043] Specifically, for the converted spiking neural network, the convolution layer feature of the multi-layer spiking neural network is contained in each time step, each neuron performs convolution operation on the input pulse and the convolution kernel to calculate its input current, and integrates it into the membrane potential, if the membrane potential exceeds the convolution threshold, the neuron will generate a pulse and reset the membrane potential, otherwise the membrane potential will decay. The calculation formula of its input current can be expressed as:

[0044]

[0045] where t represents the total time step, represents the total current flow into the membrane potential of the l+1 layer accumulated at time t, n l represents the total number of pre-neurons in the lth layer, x i (t) represents the sum of pulses of the ith pre-neuron at time t.

[0046] The neuron in the multi-layer spiking neural network can be described by three processes of charging, discharging and resetting, specifically as follows:

[0047] H(t) = f(V(t-1), X(t)) (2)

[0048] S(t) = θ(H(t) - V threshold ) (3)

[0049] V(t) = H(t) · (1 - S(t)) + V reset · S(t) (4)

[0050] where H(t), S(t), V(t) represent the charging, discharging and resetting equations of the spiking neuron respectively; H(t) is the instantaneous state of the neuron, which is the instantaneous voltage before it fires a pulse, V(t) is the membrane voltage of the neuron, and before firing a pulse, V(t) = H(t).threshold is the threshold value of the neuron, when the membrane voltage exceeds this threshold value, the pulse neuron will fire a pulse; X(t) is the input voltage, for example, the increment of the voltage; f(V(t-1), X(t)) represents the update of the state of the neuron, and the update equation is different for different neurons; θ(x) is a step function, that is, the value of S(t) is 0 or 1, V reset is the reset voltage, when the neuron releases a pulse S(t) = 1, the membrane voltage will be reset to V reset , that is, V(t) = V reset .

[0051] For Leaky integrate and fire (LIF) neurons, the dynamic differential equation when the threshold is not exceeded is described, and the corresponding difference equation is:

[0052]

[0053] τ m (V(t)-V(t-1))=-(V(t-1)-V reset )+X(t)(6)

[0054] The corresponding charging equation is

[0055]

[0056] τ m represents the time constant.

[0057] In addition, it should be noted that in each of the above embodiments, before training the respective target detection model using the SAR labeled image or the SAS labeled image, the size of different SAR labeled images or SAS labeled images should be adjusted in proportion, and then the background filling operation is performed on the adjusted data image.

[0058] In order to verify the effectiveness of the method of the present application, the present application also makes related experiments. In view of the problem of lack of SAS image detection data set, this experiment constructs a SAS data set which can be used for sonar image target detection. The data set includes 322 high-resolution images (including 323 targets, of which 266 are shipwreck targets and 57 are aircraft targets). The experimental results are shown in Table 1.

[0059] Table 1: Target detection accuracy of synthetic aperture sonar data set

[0060] Method mAP@0.5 mAP@0.5:0.95 Direct training 0.73 0.399 Pre-training 0.912 0.574

[0061] The mAP is used to determine the overall detection accuracy. Commonly set IoU thresholds are 0.5 and 0.5:0.95. When the threshold is set to 0.5, mAP@0.5 represents the average mAP calculated for all images of each class when the IoU of the detection results for all test images is greater than 0.5. When the threshold is set to 0.5:0.95, mAP@0.5:0.95 represents the average mAP across different IoUs (thresholds from 0.5 to 0.95, with a step size of 0.05), specifically (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). The AP is calculated as follows:

[0062]

[0063] in

[0064]

[0065] in In Recall The corresponding Precision at that time.

[0066] As shown in Table 1, underwater target detection on the SAS dataset demonstrates that the method of the present invention, by employing pre-training, can achieve high detection accuracy on synthetic aperture sonar image datasets. Figure 4 This is an example diagram showing the detection results of underwater targets using the method of the present invention.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting a target in a synthetic aperture sonar image based on a pulse neural network, characterized in that, The method comprises the following steps: Step 1: constructing a deep convolutional neural network, training the deep convolutional neural network by using a synthetic aperture radar (SAR) labeled image to obtain a target detection model DCN for SAR images SAR The deep convolutional neural network comprises a backbone feature extraction network, a strengthened feature extraction network, a classifier and a regressor; a maximum pooling layer with a step length of 2 is modified into a convolutional layer with the same step length, a maximum pooling layer with a step length of 1 is deleted, an up-sampling layer is modified into a deconvolutional layer, a Leaky_ReLU activation function is modified into a ReLU activation function, and a batch normalization layer is modified into a fusion layer composed of a convolutional layer and a batch normalization layer; Step 2: converting the deep convolutional neural network into a spiking neural network using spiking neurons, taking the weights of the target detection model DCN SAR as the initial weights of the spiking neural network, training the spiking neural network using synthetic aperture sonar (SAS) labeled images to obtain a SAS image-oriented target detection model DSN SAS ; wherein the converting the deep convolutional neural network into a spiking neural network using spiking neurons specifically comprises: replacing the ReLU activation function therein with a spiking neuron, and setting the average pooling layer therein as a spatial down-sampling layer. Step 3: Obtain the SAS image to be detected, and adopt the target detection model DSN SAS Perform sliding detection on the SAS image to be detected, and locate and label the target, specifically including: Step 3.1: obtaining a to-be-detected SAS image, and performing visual saliency calculation on the to-be-detected SAS image to obtain a corresponding saliency map; Step 3.2: setting a saliency value threshold, intercepting a region greater than the saliency value threshold in the saliency map, and taking the region as a to-be-detected target region; Step 3.3: using the target detection model DSN SAS The target region to be detected is detected by sliding.

2. The method of claim 1, wherein the method further comprises: In step 3.1, the LC algorithm, the HC algorithm, the RC algorithm or the GR algorithm is used for visual saliency calculation.

Images