SAR image ship detection method and system based on neighborhood information feature enhancement

Abstract

The invention provides an SAR image ship detection method and system based on neighborhood information feature enhancement. The method comprises the steps that a to-be-recognized target sample SAR image is acquired; inputting the target sample SAR image into a pre-trained target detection model to obtain a ship target; wherein the pre-trained target detection model is obtained by training a context attention module and a feature extractor based on a sample with a ship target; the context attention module is constructed in combination with multi-scale feature extraction and an attention mechanism. According to the method, the context information in the feature map is acquired through the context attention module, and the feature information is enhanced and fused by combining the neural networks from bottom to top and from top to bottom, so that the problems of small target detection and complex background interference in the SAR image are solved.

Description

Technical Field

[0001] The present invention relates to the field of ship image detection, and in particular to a SAR image ship detection method and system based on neighborhood information feature enhancement. Background Art

[0002] SAR (Synthetic Aperture Radar) image is an image formed by an all-weather high-resolution microwave imaging radar. SAR image ship detection is the automatic identification of ship targets in SAR images through feature analysis and other technologies. It plays a very important role in the fields of ocean management and monitoring.

[0003] Traditional SAR image ship detection requires artificial prior knowledge to distinguish the difference between the background and the target. For example, the method based on CFAR (constant false-alarm rate) keeps the false alarm rate unchanged, calculates the statistics between the pixel points in the image and the surroundings, adaptively adjusts the detection threshold, and combines the sliding window traversal method to achieve ship detection. When using the CFAR method for ship detection, it is necessary to model the complex background in the image and design appropriate statistical variables for threshold calculation. This method that requires artificial prior knowledge limits its versatility to a certain extent. In addition, the sliding window detection method has a large amount of calculation and it is difficult to meet real-time application requirements.

[0004] At present, the ship detection method based on deep learning has been successfully applied to SAR images. Due to the particularity of the imaging method, there is a lot of background noise in SAR images. In addition, the difference in the scale of the target itself and the transformation of the physical resolution used in imaging make the target scale in SAR images very different. Therefore, when using the deep learning-based method for ship detection, it is necessary to fully consider the small ship targets and complex background clutter in the SAR image. Summary of the invention

[0005] In order to solve the problems of small targets and complex background clutter in the current SAR image ship recognition in the existing technology, a SAR image ship detection method based on neighborhood information feature enhancement is proposed, including:

[0006] Acquire a sample SAR image of a target to be identified;

[0007] Inputting the target sample SAR image into a pre-trained target detection model to obtain a ship target;

[0008] The pre-trained target detection model is obtained by training a context attention module and a feature extractor based on samples with ship targets;

[0009] The contextual attention module is constructed by combining multi-scale feature extraction and attention mechanism.

[0010] Optionally, the training of the target detection model includes:

[0011] Based on the sample SAR image with the ship target, the feature information of multiple stages is obtained through the feature extractor;

[0012] Using a context attention module and a feature extractor to enhance the feature information of the multiple stages and construct a feature map rich in semantic information and position information;

[0013] Based on the feature map rich in semantic information and position information, a convolution operation is used to map it to the classification and regression space to obtain a bounding box and category confidence of the predicted target;

[0014] Based on the bounding box of the predicted target and the category confidence combined with the bounding box of the real target, the loss function of the detection model network is constructed, and the target detection model is trained to obtain a trained target detection model.

[0015] Optionally, the using of a context attention module and a feature extractor to enhance the feature information of the multiple stages to construct a feature map rich in semantic information and position information includes:

[0016] Step 1: Use upsampling, context attention module and feature extractor to construct a feature map rich in semantic information from top to bottom;

[0017] Step 2: Use downsampling, context attention module and feature extractor to strengthen the position information in the feature map rich in semantic information from bottom to top.

[0018] Optionally, the top-down construction of a feature map rich in semantic information using upsampling and a context attention module and a feature extractor includes:

[0019] After upsampling the previous stage feature information extracted by the neural network and the next level feature information obtained through enhanced processing by the context attention module, the intermediate feature information is obtained by splicing the feature map, and then the feature map rich in semantic information is constructed by the feature extractor and the context attention module in turn.

[0020] Optionally, the calculation formula for constructing a feature map rich in semantic information is as follows:

[0021]

[0022] Among them, C i represents the feature information of the i-th stage extracted by the neural network; US(·) represents the upsampling operation; P i+1represents the feature information after being processed by the next level attention module; Concat(·) represents the stacking operation along the channel dimension; CSP2(·) represents the feature extraction structure; I i represents the intermediate feature information after splicing; P i It represents the feature information processed by the attention module at this level; CTAM(·) represents the contextual attention module.

[0023] Optionally, the bottom-up strengthening of the position information in the feature map rich in semantic information by using downsampling and a context attention module and a feature extractor includes:

[0024] After downsampling based on the feature map rich in semantic information and the next level feature information obtained through enhanced processing by the context attention module, the position information in the feature map rich in semantic information is enhanced through feature map splicing, feature extractor and context attention module in sequence.

[0025] Optionally, the calculation formula for strengthening the position information in the feature map rich in semantic information is as follows:

[0026]

[0027] Among them, P i ′ represents enhanced feature information; P i represents the feature information processed by the attention module at this level; P i-1 ′ represents the feature information enhanced by the previous level; I i ′ represents the intermediate feature information enhanced again by the contextual attention module.

[0028] Optionally, the construction process of the context attention module includes:

[0029] Based on the original feature information input into the context attention module, the feature information extracted by using convolution kernels of different sizes is stacked along the channel dimension and then convolved and fused;

[0030] The attention mechanism is used to highlight the important part of the fused feature information, which is then stacked and convolved with the original feature information along the channel dimension to obtain a contextual attention module, and the feature information processed by the contextual attention module is output.

[0031] Optionally, the feature information processed by the context attention module is calculated as follows:

[0032]

[0033] Among them, x in Represents the original feature information of the input; Convk×k (·) indicates the use of a convolutional layer with a convolution kernel size of k×k, where k is an odd number; Concat(·) indicates stacking operations along the channel dimension; represents the features after stacking; x represents the feature information after convolution; Attn(·) represents the attention mechanism; represents the feature information after being processed by the attention mechanism and stacked with the original feature information; x out Represents the output feature information.

[0034] Optionally, mapping the feature map rich in semantic information and position information to the classification and regression space using a convolution operation includes:

[0035] p i =Conv 1×1 (P i ′)

[0036] Among them, p i Represents the feature information after mapping.

[0037] Optionally, the loss function of the detection model network is constructed based on the bounding box of the predicted target and the category confidence combined with the bounding box of the real target, and the training of the target detection model includes:

[0038] Based on the difference between the bounding box of the predicted target and the bounding box of the real target, the binary cross entropy loss function is used to calculate the positioning loss and classification loss, and the total loss function of the target detection model network is obtained;

[0039] The target detection model is iteratively trained based on the total loss function until a preset number of iterations is reached to obtain a trained target detection model.

[0040] Optionally, the acquiring of multiple-stage feature information by a feature extractor based on the sample SAR image with the ship target includes:

[0041] Based on the sample SAR image, low-frequency information, horizontal high-frequency information, vertical high-frequency information and diagonal high-frequency information are obtained through wavelet decomposition transformation, and high-frequency feature information is extracted;

[0042] Extracting multi-scale feature information based on the sample SAR image through convolution kernels of different sizes in a neural network;

[0043] The high-frequency feature information is introduced as an auxiliary feature into the feature information of each scale for splicing, and a feature extractor is used to obtain feature information of multiple stages.

[0044] Based on the same inventive concept, the present invention also provides a SAR image ship detection system based on neighborhood information feature enhancement, comprising:

[0045] An acquisition module is used to obtain a sample SAR image of a target to be identified;

[0046] A detection module is used to input the target sample SAR image into a pre-trained target detection model to obtain a ship target;

[0047] The pre-trained target detection model is obtained by training a context attention module and a feature extractor based on samples with ship targets;

[0048] The contextual attention module is constructed by combining multi-scale feature extraction and attention mechanism.

[0049] Optionally, the system further includes:

[0050] The target detection model building module is used to train the context attention module and the feature extractor based on the samples with the ship target to obtain the target detection model;

[0051] The target detection model building module includes:

[0052] A target feature extraction unit, used for obtaining multiple-stage feature information through a feature extractor based on a sample SAR image with a ship target;

[0053] A target feature enhancement unit, used to enhance the feature information of the multiple stages by using a context attention module and a feature extractor, and construct a feature map rich in semantic information and position information;

[0054] A convolution mapping unit, used to map the feature map rich in semantic information and position information to the classification and regression space using a convolution operation to obtain a bounding box and a category confidence of the predicted target;

[0055] The target detection model building unit is used to build a loss function of the detection model network based on the bounding box of the predicted target and the category confidence combined with the bounding box of the real target, train the target detection model, and obtain a trained target detection model.

[0056] Optionally, the target feature enhancement unit is specifically used for:

[0057] Step 1: Use upsampling, context attention module and feature extractor to construct a feature map rich in semantic information from top to bottom;

[0058] Step 2: Use downsampling, context attention module and feature extractor to strengthen the position information in the feature map rich in semantic information from bottom to top.

[0059] Optionally, the target feature enhancement unit uses upsampling and a context attention module and a feature extractor to construct a feature map rich in semantic information from top to bottom, including:

[0060] After upsampling the previous stage feature information extracted by the neural network and the next level feature information obtained through enhanced processing by the context attention module, the intermediate feature information is obtained by splicing the feature map, and then the feature map rich in semantic information is constructed by the feature extractor and the context attention module in turn.

[0061] Optionally, the calculation formula for constructing a feature map rich in semantic information is as follows:

[0062]

[0063] Among them, C i represents the feature information of the i-th stage extracted by the neural network; US(·) represents the upsampling operation; P i+1 represents the feature information after being processed by the next level attention module; Concat(·) represents the stacking operation along the channel dimension; CSP2(·) represents the feature extraction structure; I i represents the intermediate feature information after splicing; P i It represents the feature information processed by the attention module at this level; CTAM(·) represents the contextual attention module.

[0064] Optionally, the target feature enhancement unit uses downsampling and a context attention module and a feature extractor to enhance the position information in the feature map rich in semantic information from bottom to top, including:

[0065] After downsampling based on the feature map rich in semantic information and the next level feature information obtained through enhanced processing by the context attention module, the position information in the feature map rich in semantic information is enhanced through feature map splicing, feature extractor and context attention module in sequence.

[0066] Optionally, the calculation formula for strengthening the position information in the feature map rich in semantic information is as follows:

[0067]

[0068] Among them, P i ′ represents enhanced feature information; P i represents the feature information processed by the attention module at this level; P i-1 ′ represents the feature information enhanced by the previous level; I i ′ represents the intermediate feature information enhanced again by the contextual attention module.

[0069] Optionally, the system also includes a context attention module construction unit, which is used to construct the context attention module.

[0070] The context attention module construction unit is specifically used for:

[0071] Based on the original feature information input into the context attention module, the feature information extracted by using convolution kernels of different sizes is stacked along the channel dimension and then convolved and fused;

[0072] The attention mechanism is used to highlight the important part of the fused feature information, which is then stacked and convolved with the original feature information along the channel dimension to obtain a contextual attention module, and the feature information processed by the contextual attention module is output.

[0073] Optionally, the feature information processed by the context attention module is calculated as follows:

[0074]

[0075] Among them, x in Represents the original feature information of the input; Conv k×k (·) indicates the use of a convolutional layer with a convolution kernel size of k×k, where k is an odd number; Concat(·) indicates stacking operations along the channel dimension; represents the features after stacking; x represents the feature information after convolution; Attn(·) represents the attention mechanism; represents the feature information after being processed by the attention mechanism and stacked with the original feature information; x out Represents the output feature information.

[0076] Optionally, the convolution mapping unit maps the feature map rich in semantic information and position information to the classification and regression space using a convolution operation through the following formula:

[0077] p i =Conv 1×1 (P i ′)

[0078] Among them, p i Represents the feature information after mapping.

[0079] Optionally, the target detection model building unit is specifically used to:

[0080] Based on the difference between the bounding box of the predicted target and the bounding box of the real target, the binary cross entropy loss function is used to calculate the positioning loss and classification loss, and the total loss function of the target detection model network is obtained;

[0081] The target detection model is iteratively trained based on the total loss function until a preset number of iterations is reached to obtain a trained target detection model.

[0082] Optionally, the target feature extraction unit is specifically used for:

[0083] Based on the sample SAR image, low-frequency information, horizontal high-frequency information, vertical high-frequency information and diagonal high-frequency information are obtained through wavelet decomposition transformation, and high-frequency feature information is extracted;

[0084] Extracting multi-scale feature information based on the sample SAR image through convolution kernels of different sizes in a neural network;

[0085] The high-frequency feature information is introduced as an auxiliary feature into the feature information of each scale for splicing, and a feature extractor is used to obtain feature information of multiple stages.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] The present invention provides a method and system for SAR image ship detection based on neighborhood information feature enhancement, the method comprising: obtaining a target sample SAR image to be identified; inputting the target sample SAR image into a pre-trained target detection model to obtain a ship target; wherein the pre-trained target detection model is obtained by training a context attention module and a feature extractor based on samples with ship targets; the context attention module is constructed by combining multi-scale feature extraction and attention mechanism. The present invention obtains context information in a feature map through a context attention module, and enhances and fuses the feature information in combination with bottom-up and top-down neural networks, thereby solving the problems of small target detection and complex background interference in SAR images. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A schematic flow chart of a method for detecting ship targets in SAR images based on neighborhood information enhancement features in an embodiment of the present invention;

[0089] Figure 2 A flowchart of an enhanced processing method for detecting ship targets in SAR images based on neighborhood information enhancement features in an embodiment of the present invention;

[0090] Figure 3 A process diagram of constructing a context attention module of a SAR image ship target detection method based on neighborhood information enhancement features in an embodiment provided by the present invention;

[0091] Figure 4 A schematic diagram of the attention mechanism process of the SAR image ship target detection method based on neighborhood information enhancement features in an embodiment provided by the present invention;

[0092] Figure 5 A schematic diagram of a feature extractor process of a SAR image ship target detection method based on neighborhood information enhancement features in an embodiment of the present invention;

[0093] Figure 6 A schematic diagram of a feature extractor process of a SAR image ship target detection method based on neighborhood information enhancement features in an embodiment of the present invention;

[0094] Figure 7 A schematic diagram of detection results of a SAR image ship target detection method based on neighborhood information enhancement features in an embodiment provided by the present invention;

[0095] Figure 8 In the embodiment provided by the present invention, a SAR image ship target detection method based on neighborhood information enhancement features detects the intention of electronic equipment. DETAILED DESCRIPTION

[0096] The specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0097] Embodiment 1:

[0098] The present invention provides a SAR image ship detection method based on neighborhood information feature enhancement. SAR image ship detection refers to automatically identifying the position of ship targets in SAR images, such as Figure 1 As shown, the method includes:

[0099] Step 1: Obtain a sample SAR image of the target to be identified;

[0100] Step 2: Input the target sample SAR image into a pre-trained target detection model to obtain a ship target;

[0101] The pre-trained target detection model is obtained by training a context attention module and a feature extractor based on samples with ship targets;

[0102] The contextual attention module is constructed by combining multi-scale feature extraction and attention mechanism.

[0103] Step 1: Obtain sample SAR images of the target to be identified, including SAR images of ship targets of different sizes or SAR images of small ship targets.

[0104] Before introducing step 2, the training of the target detection model involved in this application is first introduced. The training of the target detection model includes:

[0105] S1: Based on the sample SAR image with the ship target, the feature extractor is used to obtain feature information at multiple stages, including:

[0106] Based on the sample SAR image, low-frequency information, horizontal high-frequency information, vertical high-frequency information and diagonal high-frequency information are obtained through wavelet decomposition transformation, and high-frequency feature information is extracted;

[0107] Extracting multi-scale feature information based on the sample SAR image through convolution kernels of different sizes in a neural network;

[0108] The high-frequency feature information is introduced as an auxiliary feature into the feature information of each scale for splicing, and a feature extractor is used to obtain feature information of multiple stages.

[0109] like Figure 2 As shown in the figure, for the sample SAR image input by the input, a two-dimensional Haar Wavelet transform will decompose the image into four parts: low-frequency information L, horizontal high-frequency information H, vertical high-frequency information V and diagonal high-frequency information D. After a wavelet decomposition transform, the image will be compressed to one-fourth of the original. The calculation process of the two-dimensional Haar Wavelet decomposition transform can be expressed as:

[0110]

[0111] Among them, x i，j Represents the pixel value at the (i, j)th position of the image; L i,j Represents the low-frequency information at the (i, j)th position of the image; V i,j Represents the vertical high-frequency information at the (i, j)th position of the image; H i,j Represents the horizontal high-frequency information at the (i, j)th position of the image; D i,j Represents the diagonal high-frequency information at the (i, j)th position of the image.

[0112] After a wavelet decomposition transformation, the low-frequency information can be decomposed again to further obtain feature information of a larger scale. The present invention uses 4-times two-dimensional Haar Wavelet Haar wavelet decomposition transformation to extract high-frequency feature information at different scales, corresponding to the features at different scales extracted by the neural network, and uses a splicing feature extractor to extract high-frequency information.

[0113] The neural network CBL learning is used to extract feature information of sample SAR images at multiple scales, namely, the feature information of the image is extracted in the four stages C2-C5. Then the high-frequency information obtained by the Haar Wavelet decomposition transform is introduced into each stage as an auxiliary feature, so that the model can pay more attention to the high-frequency information characteristics unique to the target object.

[0114] S2: Using the context attention module and feature extractor to enhance the feature information of the multiple stages and construct a feature map rich in semantic information and position information, specifically including:

[0115] like Figure 3 As shown in Figure 2, the process of constructing the contextual attention module, CTAM(·), is as follows:

[0116] Based on the original feature information input into the context attention module, the feature information under different receptive fields is extracted by using convolution layers with convolution kernels of different sizes, which are stacked along the channel dimension, and a new convolution layer is constructed using the convolution kernel of the minimum size used in the previous step. The features after the feature map splicing are further fused. The process can be expressed as:

[0117]

[0118] Among them, x in Represents the original feature information of the input; Conv k×k (·) indicates the use of a convolutional layer with a convolution kernel size of k×k, where k is an odd number; Concat(·) indicates stacking operations along the channel dimension; represents the features after stacking; x represents the feature information after convolution;

[0119] After feature fusion under different receptive fields, the attention mechanism is used to highlight the important part of the fused feature information, and the original feature information is stacked along the channel dimension, and the convolution operation is used to fuse the feature information of the two to achieve feature enhancement, and finally the context attention module is obtained, and the feature information processed by the context attention module is output; the process is:

[0120]

[0121] Among them, Attn(·) represents the attention mechanism; represents the feature information after being processed by the attention mechanism and stacked with the original feature information; x out Represents the output feature information.

[0122] like Figure 4 As shown, the Attn(·) attention mechanism includes the Channel Attention channel attention mechanism and the Spatial Attention spatial attention mechanism, and the specific operations are as follows:

[0123]

[0124] Among them, x represents the original input features; CA(·) represents the features after channel attention processing; SA(·) represents the features after spatial attention processing; σ represents the sigmoid function; GAvgPool(·) and GMaxPool(·) represent global average pooling and global maximum pooling respectively; Avg(·) and Max(·) represent the average value calculation and maximum value calculation operations respectively.

[0125] like Figure 2 As shown, step S2 specifically includes:

[0126] S201: Use upsampling and context attention modules and feature extractors to construct feature maps rich in semantic information from top to bottom;

[0127] Specifically, after UpSampling based on the previous stage feature information extracted by the neural network and the next level feature information obtained through enhanced processing by the context attention module, the intermediate feature information is obtained by splicing the feature map, and then the feature map rich in semantic information is constructed through the feature extractor and the context attention module in sequence.

[0128] The feature information C obtained by the feature extractor i And the corresponding feature information P i+1 , calculate the feature information P i , the specific process is as follows:

[0129]

[0130] Among them, C i represents the feature information of the i-th stage extracted by the neural network; US(·) represents the upsampling operation; P i+1 represents the feature information processed by the next level attention module; Concat(·) represents the stacking operation along the channel dimension; I i represents the intermediate feature information after splicing; P i represents the feature information after being processed by the attention module at this level; CSP2(·) represents Figure 6 The feature extraction structure shown in Figure 2 is as follows: Figure 5 In the feature extraction structure shown, ×N means repeating the structure N times, and N=3 in the present invention.

[0131] After extracting the feature information of the last stage based on the neural network, SPPF spatial pyramid pooling processing is also included.

[0132] S202: using downsampling, a context attention module and a feature extractor to enhance the position information in the feature map rich in semantic information from bottom to top;

[0133] Specifically, based on the feature map rich in semantic information, that is, the feature information P i After downsampling the next level feature information obtained by the context attention module, the position information P in the feature map rich in semantic information is strengthened by feature map concatenation, feature extractor and context attention module in turn. i ′, the specific process is as follows:

[0134]

[0135] Among them, P i ′ represents enhanced feature information; P i represents the feature information processed by the attention module at this level; P i-1 ′ represents the feature information enhanced by the previous level; I i ′ represents the intermediate feature information enhanced again by the contextual attention module.

[0136] S3: Based on the feature map rich in semantic information and position information, a convolution operation is used to map it to the classification and regression space to obtain a bounding box and category confidence of the predicted target;

[0137] Specifically, the feature information of multiple stages after enhancement, that is, the feature map rich in semantic information and position information, is mapped to the classification and regression space using a convolution operation, and the mapping rules are as follows:

[0138] p i =Conv 1×1 (P i ′)

[0139] Among them, p i The predicted bounding box and class confidence are included.

[0140] S4: Based on the bounding box of the predicted target and the category confidence combined with the bounding box of the real target, the loss function of the detection model network is constructed, and the target detection model is trained to obtain a trained target detection model.

[0141] Specifically include:

[0142] Based on the difference between the bounding box of the predicted target and the bounding box of the real target, the binary cross entropy loss function is used to calculate the positioning loss and classification loss, and the total loss function of the target detection model network is obtained;

[0143] The predicted box and the real box are used to obtain the intersection over union (IoU) between the predicted box and the real box, that is:

[0144]

[0145] Among them, b prepresents the prediction box; b gt Represents the true box.

[0146] The difference CIoU between the predicted box and the real box is calculated according to the intersection over union (IoU) between the predicted box and the real box. The specific calculation method is as follows:

[0147]

[0148] Among them, α is a weight value; υ represents the similarity of aspect ratio; ρ represents the Euclidean distance between the center point of the predicted box and the real box; c represents the diagonal distance between the predicted box and the minimum closure area of ​​the real box; w and h represent the width and height of the box respectively; gt represents the true value.

[0149] According to the difference CIoU between the predicted box and the real box, the binary cross entropy loss function BEC(·) is used to calculate the positioning loss Loss box And classification loss Loss boj , the specific calculation formula is as follows:

[0150]

[0151] Among them, μ i is a hyperparameter; n represents the number of input features; Clamp(·) means truncating the input to fall into the interval [0, 1]; CIoU i Represents the difference between the i-th predicted box and the true box.

[0152] Finally, the loss function Loss of the detection model network is obtained, namely:

[0153] Loss = λ1Loss box +λ2Loss obj

[0154] Among them, λ1 and λ2 represent the hyperparameters of localization loss and classification loss, respectively.

[0155] The target detection model is iteratively trained based on the total loss function until a preset number of iterations is reached to obtain a trained target detection model.

[0156] Step 2: Input the target sample SAR image into a pre-trained target detection model to obtain a ship target, specifically including:

[0157] The above-trained target detection model is used to identify the target sample SAR image and obtain the ship target; Figure 7 As shown, the detection accuracy of small ship targets in SAR images is improved.

[0158] Embodiment 2:

[0159] The present invention based on the same inventive concept also provides a SAR image ship detection system based on neighborhood information feature enhancement, comprising:

[0160] An acquisition module is used to obtain a sample SAR image of a target to be identified;

[0161] A detection module is used to input the target sample SAR image into a pre-trained target detection model to obtain a ship target;

[0162] The pre-trained target detection model is obtained by training a context attention module and a feature extractor based on samples with ship targets;

[0163] The contextual attention module is constructed by combining multi-scale feature extraction and attention mechanism.

[0164] The system further comprises:

[0165] The target detection model building module is used to train the context attention module and the feature extractor based on the samples with the ship target to obtain the target detection model;

[0166] The target detection model building module includes:

[0167] A target feature extraction unit, used for obtaining multiple-stage feature information through a feature extractor based on a sample SAR image with a ship target;

[0168] A target feature enhancement unit, used to enhance the feature information of the multiple stages by using a context attention module and a feature extractor, and construct a feature map rich in semantic information and position information;

[0169] A convolution mapping unit, used to map the feature map rich in semantic information and position information to the classification and regression space using a convolution operation to obtain a bounding box and a category confidence of the predicted target;

[0170] The target detection model building unit is used to build a loss function of the detection model network based on the bounding box of the predicted target and the category confidence combined with the bounding box of the real target, train the target detection model, and obtain a trained target detection model.

[0171] The target feature enhancement unit is specifically used for:

[0172] Step 1: Use upsampling, context attention module and feature extractor to construct a feature map rich in semantic information from top to bottom;

[0173] Step 2: Use downsampling, context attention module and feature extractor to strengthen the position information in the feature map rich in semantic information from bottom to top.

[0174] The target feature enhancement unit uses upsampling and context attention modules and feature extractors to construct a feature map rich in semantic information from top to bottom, including:

[0175] After upsampling the previous stage feature information extracted by the neural network and the next level feature information obtained through enhanced processing by the context attention module, the intermediate feature information is obtained by splicing the feature map, and then the feature map rich in semantic information is constructed by the feature extractor and the context attention module in turn.

[0176] The calculation formula for constructing a feature map rich in semantic information is as follows:

[0177]

[0178] Among them, C i represents the feature information of the i-th stage extracted by the neural network; US(·) represents the upsampling operation; P i+1 represents the feature information after being processed by the next level attention module; Concat(·) represents the stacking operation along the channel dimension; CSP2(·) represents the feature extraction structure; I i represents the intermediate feature information after splicing; P i It represents the feature information processed by the attention module at this level; CTAM(·) represents the contextual attention module.

[0179] The target feature enhancement unit uses downsampling and a context attention module and a feature extractor to enhance the position information in the feature map rich in semantic information from bottom to top, including:

[0180] After downsampling based on the feature map rich in semantic information and the next level feature information obtained through enhanced processing by the context attention module, the position information in the feature map rich in semantic information is enhanced through feature map splicing, feature extractor and context attention module in sequence.

[0181] The calculation formula for strengthening the position information in the feature map rich in semantic information is as follows:

[0182]

[0183] Among them, P i ′ represents enhanced feature information; P i represents the feature information processed by the attention module at this level; P i-1 ′ represents the feature information enhanced by the previous level; I i ′ represents the intermediate feature information enhanced again by the contextual attention module; CTAM(·) represents the contextual attention module.

[0184] The system also includes a contextual attention module construction unit, which is used to construct a contextual attention module.

[0185] The context attention module construction unit is specifically used for:

[0186] Based on the original feature information input into the context attention module, the feature information extracted by using convolution kernels of different sizes is stacked along the channel dimension and then convolved and fused;

[0187] The attention mechanism is used to highlight the important part of the fused feature information, which is then stacked and convolved with the original feature information along the channel dimension to obtain a contextual attention module, and the feature information processed by the contextual attention module is output.

[0188] The feature information processed by the context attention module is calculated as follows:

[0189]

[0190] Among them, x in Represents the original feature information of the input; Conv k×k (·) indicates the use of a convolutional layer with a convolution kernel size of k×k, where k is an odd number; Concat(·) indicates stacking operations along the channel dimension; represents the features after stacking; x represents the feature information after convolution; Attn(·) represents the attention mechanism; represents the feature information after being processed by the attention mechanism and stacked with the original feature information; x out Represents the output feature information.

[0191] The convolution mapping unit maps the feature map rich in semantic information and position information to the classification and regression space using a convolution operation through the following formula:

[0192] p i =Conv 1×1 (P i ′)

[0193] Among them, p i Represents the feature information after mapping.

[0194] The target detection model building unit is specifically used for:

[0195] Based on the difference between the bounding box of the predicted target and the bounding box of the real target, the binary cross entropy loss function is used to calculate the positioning loss and classification loss, and the total loss function of the target detection model network is obtained;

[0196] The target detection model is iteratively trained based on the total loss function until a preset number of iterations is reached to obtain a trained target detection model.

[0197] The target feature extraction unit is specifically used for:

[0198] Based on the sample SAR image, low-frequency information, horizontal high-frequency information, vertical high-frequency information and diagonal high-frequency information are obtained through wavelet decomposition transformation, and high-frequency feature information is extracted;

[0199] Extracting multi-scale feature information based on the sample SAR image through convolution kernels of different sizes in a neural network;

[0200] The high-frequency feature information is introduced as an auxiliary feature into the feature information of each scale for splicing, and a feature extractor is used to obtain feature information of multiple stages.

[0201] Embodiment 3:

[0202] like Figure 8 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0203] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of a SAR image ship detection method based on neighborhood information feature enhancement in the above embodiment.

[0204] Embodiment 4:

[0205] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in the electronic device for storing programs and data. It can be understood that the storage medium here can include both the built-in storage medium in the electronic device and the extended storage medium supported by the electronic device. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of a SAR image ship detection method based on neighborhood information feature enhancement in the above embodiment.

[0206] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0207] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0208] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0209] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0210] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A SAR image ship detection method based on neighborhood information feature enhancement, characterized in that: The method comprises: Acquire a sample SAR image of a target to be identified; Inputting the target sample SAR image into a pre-trained target detection model to obtain a ship target; The pre-trained target detection model is obtained by training a context attention module and a feature extractor based on samples with ship targets; The contextual attention module is constructed by combining multi-scale feature extraction and attention mechanism.

2. The detection method according to claim 1, characterized in that: The training of the target detection model includes: Based on the sample SAR image with the ship target, the feature information of multiple stages is obtained through the feature extractor; Using a context attention module and a feature extractor to enhance the feature information of the multiple stages and construct a feature map rich in semantic information and position information; Based on the feature map rich in semantic information and position information, a convolution operation is used to map it to the classification and regression space to obtain a bounding box and category confidence of the predicted target; Based on the bounding box of the predicted target and the category confidence combined with the bounding box of the real target, the loss function of the detection model network is constructed, and the target detection model is trained to obtain a trained target detection model; Preferably, the process of using a context attention module and a feature extractor to enhance the feature information of the multiple stages and construct a feature map rich in semantic information and position information comprises: Step 1: Use upsampling, context attention module and feature extractor to construct a feature map rich in semantic information from top to bottom; Step 2: Use downsampling, context attention module and feature extractor to strengthen the position information in the feature map rich in semantic information from bottom to top.

3. The detection method according to claim 2, characterized in that: The method of constructing a feature map rich in semantic information from top to bottom by using upsampling, context attention module and feature extractor includes: After upsampling the feature information of the previous stage extracted by the neural network and the next level feature information obtained by enhanced processing by the context attention module, the intermediate feature information is obtained by feature map splicing, and then the feature map rich in semantic information is constructed by the feature extractor and the context attention module in sequence; Preferably, the calculation formula for constructing a feature map rich in semantic information is as follows: Among them, C i represents the feature information of the i-th stage extracted by the neural network; US(·) represents the upsampling operation; Pi +1 represents the feature information processed by the next level of attention module; Concat(·) represents the stacking operation along the channel dimension; CSP2(·) represents the feature extraction structure; Ii represents the concatenated intermediate feature information; Pi represents the feature information processed by the current level of attention module; CTAM(·) represents the contextual attention module; Preferably, the method of strengthening the position information in the feature map rich in semantic information from bottom to top by using downsampling and a context attention module and a feature extractor comprises: After downsampling based on the feature map rich in semantic information and the next level feature information obtained by strengthening the processing of the context attention module, the position information in the feature map rich in semantic information is strengthened by feature map concatenation, feature extractor and context attention module in sequence; Preferably, the calculation formula for strengthening the position information in the feature map rich in semantic information is as follows: Among them, P i ′ represents enhanced feature information; P i represents the feature information processed by the attention module at this level; P i-1 ′ represents the feature information enhanced by the previous level; I i ′ represents the intermediate feature information enhanced again by the contextual attention module.

4. The detection method according to claim 1, characterized in that: The construction process of the context attention module includes: Based on the original feature information input into the context attention module, the feature information extracted by using convolution kernels of different sizes is stacked along the channel dimension and then convolved and fused; The attention mechanism is used to highlight the important part of the fused feature information, and the original feature information is stacked along the channel dimension and then convolved to obtain a contextual attention module, and the feature information processed by the contextual attention module is output; Preferably, the feature information processed by the context attention module is calculated as follows: Among them, x in Represents the original feature information of the input; Convk × k(·) means using a convolutional layer with a kernel size of k×k, where k is an odd number; Concat(·) means stacking operations along the channel dimension; Represents the features after stacking; represents the feature information after convolution; Attn(·) represents the attention mechanism; represents the feature information after being processed by the attention mechanism and stacked with the original feature information; x out Represents the output feature information.

5. The detection method according to claim 3, characterized in that: The feature map rich in semantic information and position information is mapped to the classification and regression space using a convolution operation by the following formula: p i =Conv 1×1 (P i ′) Among them, p i Represents the feature information after mapping.

6. The detection method according to claim 2, characterized in that: The loss function of the detection model network is constructed based on the bounding box of the predicted target and the category confidence combined with the bounding box of the real target, and the target detection model is trained including: Based on the difference between the bounding box of the predicted target and the bounding box of the real target, the binary cross entropy loss function is used to calculate the positioning loss and classification loss, and the total loss function of the target detection model network is obtained; The target detection model is iteratively trained based on the total loss function until a preset number of iterations is reached to obtain a trained target detection model.

7. The detection method according to claim 2, characterized in that: The method of obtaining multiple stages of feature information through a feature extractor based on a sample SAR image with a ship target includes: Based on the sample SAR image, low-frequency information, horizontal high-frequency information, vertical high-frequency information and diagonal high-frequency information are obtained through wavelet decomposition transformation, and high-frequency feature information is extracted; Extracting multi-scale feature information based on the sample SAR image through convolution kernels of different sizes in a neural network; The high-frequency feature information is introduced as an auxiliary feature into the feature information of each scale for splicing, and a feature extractor is used to obtain feature information of multiple stages.

8. A SAR image ship detection system based on neighborhood information feature enhancement, characterized in that: The system comprises: An acquisition module is used to obtain a sample SAR image of a target to be identified; A detection module is used to input the target sample SAR image into a pre-trained target detection model to obtain a ship target; The pre-trained target detection model is obtained by training a context attention module and a feature extractor based on samples with ship targets; The contextual attention module is constructed by combining multi-scale feature extraction and attention mechanism.

9. An electronic device, characterized in that: include: at least one processor and memory; The memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the detection method according to any one of claims 1 to 7 is implemented.

10. A readable storage medium, characterized in that: An execution program is stored thereon, and when the execution program is executed, the detection method according to any one of claims 1 to 7 is implemented.

Images