A SAR image ship detection method based on feature pyramid and information compensation

Abstract

The application discloses a SAR image ship detection method based on a feature pyramid and information compensation, and comprises the following steps: image data preprocessing, marking a ship target to be detected in an image by using an image marking software to obtain a center point coordinate of the ship target in the image; using a feature pyramid network and an information compensation module to extract features of an input image; using a preprocessed training set to train; and using a network obtained by training to test a test set. The application has the beneficial effect that multi-scale features are extracted by using a feature extraction network and an information compensation module, and the detection performance on multi-scale ships is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning in the computer field, and particularly relates to a SAR image ship detection method based on a feature pyramid and information compensation. BACKGROUND

[0002] Synthetic aperture radar (SAR) is a kind of high-resolution imaging radar. As a kind of active microwave imaging sensor, the SAR imaging process is less affected by the environment. It can detect hidden objects and work in all-weather conditions. With the rapid development of spaceborne and airborne SAR, SAR has been widely used in civil and military fields. The research on ship detection is very important in many aspects, such as ocean monitoring, maritime management and military intelligence acquisition. In recent years, ship detection in SAR images has become a research hotspot in the world. However, there are few methods suitable for multi-scale ship detection at present, and ship detection at sea is still a difficult task. SUMMARY

[0003] In view of the defects of the multi-scale problem of the existing SAR image ship target, the present application aims to provide a SAR image ship detection method based on a feature pyramid and information compensation, which is suitable for autonomous detection of multi-scale ship targets in SAR images.

[0004] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0005] A SAR image ship detection method based on a feature pyramid and information compensation, the method comprising the following steps:

[0006] S1, image data preprocessing, using image labeling software to label the ship targets to be detected in the image, obtaining the center point coordinates (x, y) of the ship targets in the image, the width w and the height h of the target;

[0007] S2, using a feature pyramid network and an information compensation module to extract features from the input image;

[0008] S3, using the preprocessed training set for training;

[0009] S4, using the trained network to test the test set.

[0010] It should be noted that the step S1 is the total number of images, which is 5604, and 3642 images are randomly selected from the images as a training set, and the remaining 1962 images are a test set; and the image size of the training set is uniformly scaled to 800pixel x 800pixel.

[0011] It should be noted that in step S2, a feature pyramid network is built based on the deep residual network ResNet-50; the weights of ResNet-50 are derived from weight files pre-trained on the ImageNet dataset; given that the final output of each convolutional stage of ResNet-50 is {C1, C2, C3, C4, C5}, FPN outputs feature layers {P3, P4} of different scales through a top-down path and lateral connections. P5 is generated using an information compensation module.

[0012] It should be noted that step S2 further includes:

[0013] S2.1 uses scale-invariant adaptive average pooling to generate multi-scale feature maps with different spatial context information:

[0014] F i =AvgPool(C5,S×α) i ), i = 1, 2, 3

[0015] Where S = w × h represents the scale of feature map C5, and S × α i Representative feature map F i The scale, AvgPool represents the average pooling operation;

[0016] S2.2 uses a 1×1 convolutional layer to reduce the number of channels in each feature map to 256:

[0017] F i ′=Conv 1×1 (F i ,256),i=1,2,3

[0018] Among them, Conv 1×1 This represents a 1×1 convolution operation.

[0019] S2.3 Uses bilinear interpolation to upsample the above feature maps for subsequent fusion.

[0020] F i " = Upsample(F i ′,S),i=1,2,3

[0021] Upsample represents the bilinear interpolation operation;

[0022] S2.4 generates a corresponding spatial weight map for each input feature:

[0023] F″=Concat(F1″,F2″,F3″)

[0024] W = Sigmoid{Conv 3×3 [Conv 1×1(F″)]}

[0025] where Concat represents the fusion of feature maps along the channel, Conv 3×3 represents a 3x3 convolution operation, and Sigmoid represents a Sigmoid function operation.

[0026] S2.5 uses a spatial weight map W to extract multi-scale context information of the feature map F", and the output feature map F ASF is defined as follows:

[0027]

[0028] S2.6 adds the feature map F ASF and the feature map C5 and uses a 3x3 convolution layer to generate a feature map P5 of the highest pyramid level:

[0029] P5 = Conv 3×3 [ F ASF + Conv 1×1 (C5)]

[0030] P6 is obtained by convolving C5 through a 3x3 convolution layer with a step size of 2; P7 is calculated from P6 through a Rectified Linear Unit (ReLU) and a 3x3 convolution with a step size of 2. The output of the final feature pyramid network is {P3, P4, P5, P6, P7}.

[0031] It should be noted that in step S3, the areas of the anchors on the pyramid levels {P3, P4, P5, P6, P7} are set to {32 2 , 64 2 , 128 2 , 256 2 , 512 2}, respectively; the sizes and aspect ratios of the anchors on each pyramid level are set to {2 0 , 2 1 / 3 , 2 2 / 3} and {1:2, 1:1, 2:1}, respectively; a smooth L1 loss and a Focal loss are used as the loss function, and the network is trained using Stochastic Gradient Descent (SGD) until the loss function converges and no longer decreases, obtaining a weight file.

[0032] It should be noted that a total of 24 epochs are trained, and the weight decay and momentum are 0.0001 and 0.9, respectively. The initial learning rate is 0.01, and the learning rate is attenuated to 1 / 10 of the original at the 16th and 22nd epochs.

[0033] It should be noted that the weight file obtained in step S3 is used to test the test set image in step S4; wherein, the test set image is uniformly scaled to 800pixel*800pixel, same as the pre-processing of the training set image.

[0034] The application has the beneficial effect that the multi-scale feature is extracted by using the feature extraction network and the information compensation module, and the detection performance of the multi-scale ship is improved. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 It is a flowchart of the SAR image ship detection method based on the feature pyramid and information compensation of the application. DETAILED DESCRIPTION

[0036] The application will be further described below in combination with the drawings, and it should be noted that the embodiment is based on the technical solution, and detailed implementation and specific operation process are given, but the protection scope of the application is not limited to the embodiment.

[0037] EMBODIMENT

[0038] As shown in the figure, the application is a SAR image ship detection method based on the feature pyramid and information compensation, and the method comprises the following steps: Figure 1

[0039] S1, image data pre-processing, using image labeling software to label the ship target to be detected in the image, obtaining the center point coordinates (x, y) of the ship target in the image, the width w and the height h of the target;

[0040] S2, using the feature pyramid network and the information compensation module to extract features from the input image;

[0041] S3, using the pre-processed training set for training;

[0042] S4, using the trained network to test the test set.

[0043] Further, the total number of images in step S1 is 5604, and 3642 images are randomly selected as the training set, and the remaining 1962 images are the test set; and the image size of the training set is uniformly scaled to 800pixel*800pixel.

[0044] ​Further, the step S2 in the application is based on a deep residual network ResNet-50 to build a feature pyramid network; the weight of ResNet-50 is derived from the weight file pre-trained on an ImageNet dataset; the last output of each convolution stage of ResNet-50 is given as {C1, C2, C3, C4, C5}, and FPN outputs feature layers of different scales {P3, P4} through a top-down path and a horizontal connection. P5 is generated by using an information compensation module.

[0045] Further, the step S2 in the application further comprises:

[0046] S2.1 generating multi-scale feature maps with different spatial context information by using a scale-invariant adaptive average pooling:

[0047] F i i=1,2,3 i

[0048] wherein S=w×h represents the scale size of the feature map C5, S×α i represents the scale of the feature map F i , and AvgPool represents an average pooling operation;

[0049] S2.2 reducing the number of channels of each feature map to 256 by using a 1×1 convolution layer:

[0050] F i ′=Conv 1×1 (F i ,256),i=1,2,3

[0051] wherein Conv 1×1 represents a 1×1 convolution operation.

[0052] S2.3 upsampling the above feature maps by using a bilinear interpolation for subsequent fusion.

[0053] F i ″=Upsample(F i ′,S),i=1,2,3

[0054] wherein Upsample represents a bilinear interpolation operation;

[0055] S2.4 generating a corresponding spatial weight map for each input feature:

[0056] F″=Concat(F1″,F2″,F3″)

[0057] W=Sigmoid{Conv 3×3 [Conv​1×1 (F″)]

[0058] where Concat denotes the fusion of feature maps along the channel, Conv 3×3 denotes a 3x3 convolution operation, and Sigmoid denotes a Sigmoid function operation.

[0059] S2.5 uses a spatial weight map W to extract multi-scale context information of the feature map F'', and outputs a feature map F ASF is defined as follows:

[0060]

[0061] S2.6 adds the feature map F ASF and the feature map C5 and uses a 3x3 convolution layer to generate a feature map P5 of the highest pyramid level:

[0062] P5 = Conv 3×3 [ F ASF + Conv 1×1 (C5)]

[0063] P6 is obtained by convolving C5 through a 3x3 convolution layer with a step size of 2; P7 is calculated from P6 through a rectified linear unit (ReLU) and a 3x3 convolution with a step size of 2. The output of the final feature pyramid network is {P3, P4, P5, P6, P7}.

[0064] Further, in step S3, the areas of the anchors on the pyramid levels {P3, P4, P5, P6, P7} are set to {32 2 , 64 2 , 128 2 , 256 2 , 512 2}, respectively; the sizes and aspect ratios of the anchors on each pyramid level are set to {2 0 , 2 1 / 3 , 2 2 / 3} and {1:2, 1:1, 2:1}, respectively; a smooth L1 loss and a focal loss are used as loss functions, and the network is trained using stochastic gradient descent (SGD) until the loss function converges and no longer decreases, to obtain a weight file.

[0065] Further, the application trains 24 epochs in total in step S3, and the weight decay and momentum are 0.0001 and 0.9 respectively. The initial learning rate is 0.01, and the learning rate is attenuated to 1 / 10 of the original at the 16th epoch and the 22nd epoch.

[0066] Further, the application tests the test set images by using the weight file obtained in step S3 in step S4; wherein, the test set images are uniformly scaled to 800pixelx800pixel, same as the pre-processing of the training set images.

[0067] For those skilled in the art, other various corresponding changes and modifications can be made according to the technical solutions and concepts described above, and all these changes and modifications shall belong to the protection scope of the claims of the present application.

Claims

1. A SAR image ship detection method based on feature pyramid and information compensation, characterized in that, The method comprises the following steps: S1, image data preprocessing, using image labeling software to label the ship target to be detected in the image, obtaining the center point coordinates (x, y) of the ship target in the image, the width w and the height h of the target; S2, using a feature pyramid network and an information compensation module to extract features from the input image; S3, using the preprocessed training set for training; S4, using the trained network to test the test set; In step S2, a feature pyramid network is built based on a deep residual network ResNet-50; the weights of ResNet-50 are derived from the weight file pre-trained on the ImageNet dataset; the last output of each convolution stage of ResNet-50 is given as {C1, C2, C3, C4, C5}, and FPN outputs feature layers of different scales {P3, P4} through a top-down path and a horizontal connection. P5 is generated by using an information compensation module; The step S2 further comprises: S2.1, using scale-invariant adaptive average pooling to generate multi-scale feature maps with different spatial context information: F i = AvgPool(C5, S x a i ), i = 1, 2, 3 where S = w x h represents the scale size of the feature map C5, S x a i represents the scale of the feature map F i AvgPool represents the average pooling operation; S2.2, using a 1x1 convolution layer to reduce the number of channels of each feature map to 256: F i ′= Conv 1×1 (F i , 256), i = 1, 2, 3 where Conv 1×1 represents a 1 x 1 convolution operation. S2.3, using bilinear interpolation to upsample the above feature maps for subsequent fusion. F i "= Upsample(F i ', S), i = 1, 2, 3 Where Upsample represents a bilinear interpolation operation; S2.4, generating a corresponding spatial weight map for each input feature: F'' = Concat (F1'', F2'', F3'') W = Sigmoid{Conv 3×3 [Conv 1×1 (F")]} wherein Concat represents fusing feature maps along channels, Conv 3×3 represents a 3x3 convolution operation, and Sigmoid represents a Sigmoid function operation. S2.5 extracts multi-scale contextual information of the feature map F" using the spatial weight map W, and outputs a feature map F ASF is defined as follows: S2.6 The feature maps F ASF After adding and summing the feature maps C5, a 3x3 convolutional layer is used to generate the feature maps P5 of the highest pyramid level: P5 = Conv 3×3 [ F ASF + Conv 1×1 (C5)] P6 is obtained by convolving C5 through a 3x3 convolution layer with a step size of 2; P7 is calculated from P6 through a rectified linear unit (ReLU) and a 3x3 convolution with a step size of 2. The output of the final feature pyramid network is {P3, P4, P5, P6, P7}.

2. The SAR image ship detection method based on feature pyramid and information compensation according to claim 1, characterized in that, In step S1, the total number of images is 5604, and 3642 images are randomly selected as the training set, and the remaining 1962 images are the test set; the image size of the training set is uniformly scaled to 800pixelx800pixel. 3.The SAR ship detection method based on feature pyramid and information compensation according to claim 1, characterized in that, The step S3, in the pyramid hierarchy {P3, P4, P5, P6, P7}, the area of anchor is set as {32 2 ,64 2 ,128 2 ,256 2 ,512 2} respectively; in each pyramid level, the size and aspect ratio of anchor are set as {2 0 ,2 1 / 3 ,2 2 / 3} and {1:2, 1:1, 2:1} respectively; using smooth L1 loss and focal loss as loss function, using stochastic gradient descent (SGD) for network training, training until the loss function converges and no longer decreases, obtaining the weight file.

4. The SAR image ship detection method based on feature pyramid and information compensation according to claim 3, characterized in that, A total of 24 epochs are trained, and the weight decay and momentum are 0.0001 and 0.9, respectively. The initial learning rate is 0.01, and the learning rate is attenuated to 1 / 10 at the 16th and 22nd epochs.

5. The SAR image ship detection method based on feature pyramid and information compensation according to claim 1, characterized in that, In step S4, the weight file obtained in step S3 is used to test the test set images; wherein the test set images are uniformly scaled to 800pixelx800pixel, the same as the training set image preprocessing.

Images