Optical remote sensing image arbitrary direction ship target detection method based on contour modeling

Abstract

The application discloses a kind of optical remote sensing image arbitrary direction ship target detection method based on profile modeling.The application utilizes the Fourier profile modeling strategy newly proposed, by regression Fourier profile description coefficient, avoid from decoupling several independent characteristic parameters in rotating detection frame to carry out characteristic regression, to effectively solve the angle periodicity and the regression negative optimization problem caused by long-short side exchange;Compared with prior art, the application can well detect arbitrary direction ship target in high-resolution optical remote sensing image, and also achieves good results when facing complex conditions such as rotation ship target direction arbitrary, scale change, dense distribution, solves the regression problem caused by angle periodicity and long-short side exchange, and can reduce the false alarm rate caused by angle regression inaccuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical remote sensing image processing and target detection, and particularly relates to an optical remote sensing image arbitrary direction ship target detection method based on contour modeling. BACKGROUND

[0002] In recent years, with the rapid development of remote sensing technology, the observation technology of various satellite sensors on the ground is constantly improved, and the remote sensing data has the characteristics of high efficiency, multi-scale, large capacity and rich details. Optical remote sensing image target detection has become a research hotspot. Early target detection is based on traditional image processing methods, and the target is described by manually designed features, such as Haar-like, LBP and SIFT. Because the convolutional neural network has strong automatic feature extraction and representation ability, it has been widely used in target detection tasks, and has significantly improved the target detection ability and stability.

[0003] In the field of arbitrary direction ship target detection, the rotation target detection method based on convolutional neural network can be divided into three categories: angle-based, vertex coordinate-based and vector-based description methods. Among them, the angle-based arbitrary direction target description method uses five independent parameters of center point coordinates, length, width and rotation angle to describe the rotated target; the vertex coordinate-based arbitrary direction target description method usually uses four vertex coordinates, that is, the eight-parameter method to complete the description of the target; the vector-based arbitrary direction target description method uses mutually perpendicular vectors in the Cartesian coordinate system to describe the rotated target. However, due to the inherent periodicity of the angle and the exchange of the long and short sides, it is difficult to directly regress the angle. The vertex coordinate-based and vector-based description methods need to regress too many independent parameters, which slows down the convergence speed and reduces the detection rate. In summary, the existing arbitrary direction target description method still has the problem of inaccurate angle information regression, which affects the performance of optical remote sensing arbitrary direction ship target detection. Therefore, it is urgent to develop a high-precision arbitrary direction target representation method. SUMMARY

[0004] In view of this, this invention provides a method for detecting ship targets in arbitrary directions in high-resolution optical remote sensing images based on contour modeling. A Fourier contour modeling method with equal-angle sampling is designed for detecting ship targets in arbitrary directions without anchor frames. This method first establishes a unified description method in polar coordinates using the inherent angular periodicity of the contour and completes equal-angle interval sampling based on the contour. Then, a Fourier transform is performed on the two-dimensional periodic distribution function of the sampling points to complete the contour description of the ship target in arbitrary directions, which is then used for model parameter regression. Furthermore, a geometric loss function is established to facilitate model training, enabling the model to effectively capture the ship target's directional information. This invention can achieve efficient and high-precision detection of ship targets in arbitrary directions in high-resolution optical remote sensing images, and can improve the average accuracy of ship target detection under low-resource constraints.

[0005] The present invention provides a method for detecting ship targets in arbitrary directions from optical remote sensing images based on contour modeling, comprising:

[0006] Step 1: Extract multi-scale features from the optical remote sensing image to obtain a multi-scale feature map;

[0007] Step two involves classifying and performing regression analysis on the multi-scale feature maps. The regression analysis is performed by predicting Fourier contour information parameters, which include the target center point location information, a scale factor, and a shape factor. The scale factor is determined by sampling at equal angles with the target center point as the pole, where the sampling points are the intersections of rays at equal angles with the ship's contour. The maximum distance between the sampling point and the pole is the scale factor. The shape factor is determined by normalizing the distance between the sampling point and the pole using the scale factor, obtaining normalized sampling points, establishing a distribution function for the normalized sampling points, and performing a Fourier transform on the distribution function. The resulting high-frequency, low-frequency, and zero-frequency features are the shape factors.

[0008] Step 3: Decode the Fourier contour information parameters predicted in Step 2 to obtain the ship target detection results.

[0009] Preferably, step one further includes: performing feature fusion on the multi-scale feature map to obtain a fused multi-scale feature map; and using the fused multi-scale feature map to perform subsequent steps.

[0010] Preferred feature fusion methods include FPN, PANet, NAS-FPN, or CFPN.

[0011] Ideally, by analyzing feature maps C3~C at various scales... n Perform 1 The convolution and jumper parts of 1 generate the corresponding feature pyramids P3~P n , for P nP is obtained by performing a convolution operation with a step of 2 n+1 ; P3~ P n+1 is a fused multi-scale feature map.

[0012] Preferably, in step one, a ResNet-18 backbone network is used for multi-scale feature extraction.

[0013] Preferably, in step two, an intrinsic geometric information loss function is used for supervised learning; the intrinsic geometric information loss function is a weighted sum of a classification loss function, a positioning loss function, and a contour loss function.

[0014] Preferably, the intrinsic geometric information loss function L is:

[0015]

[0016]

[0017]

[0018] wherein, N pos , N neg are the number of contour points predicted to be positive and the number of contour points predicted to be negative, respectively; is a classification loss function, are the probability of being predicted to be in the ship target area and its labeled true value, respectively; is a positioning loss function, are the i-th position factor predicted and its labeled true value, respectively; is a contour loss function, are the connection values of the i-th shape factor and scale factor and their labeled true values, respectively.

[0019] Preferably, step three is specifically:

[0020] S1, performing inverse Fourier transform on the shape factor predicted in step two to obtain the normalized one-dimensional distance distribution of the predicted ship target;

[0021] S2, based on the scale factor, denormalizing the distance distribution of S1;

[0022] S3, converting the denormalized distance distribution to the Cartesian coordinate system to obtain the coordinate points of the ship target.

[0023] Beneficial effects:

[0024] (1) This invention utilizes a newly proposed Fourier contour modeling strategy. By regressing the Fourier contour description coefficients, it avoids decoupling several independent feature parameters from the rotating detection box for feature regression, thereby effectively solving the problem of negative optimization caused by angle periodicity and long-short side exchange. Compared with existing technologies, this invention can effectively detect ship targets in any direction in high-resolution optical remote sensing images. It also achieves good results when facing complex conditions such as arbitrary direction of rotating ship targets, scale changes, and dense distribution. It solves the regression problem caused by angle periodicity and long-short side exchange, and can reduce the false alarm rate caused by inaccurate angle regression.

[0025] (2) This invention uses a relatively lightweight backbone network to achieve high-performance detection of rotating ship targets with a balance between speed and accuracy. It can also achieve competitive detection results at different IoU, such as 0.5IoU, 0.7IoU and 0.9IoU, and has good practical application value.

[0026] (3) The present invention adopts a top-down feature fusion strategy to fuse shallow and deep features of multi-scale feature maps, so that the network learns effective multi-scale information of rotating ship targets, eliminates the influence of large scale span of ship targets, and further improves detection efficiency.

[0027] (4) This invention can significantly improve the detection effect of ship targets in any direction in optical remote sensing images. In particular, it can achieve good detection results for ship targets with diverse directions and huge aspect ratios, and can also achieve good detection results in complex environments and densely distributed ship targets. On the basis of improving the detection rate, it improves the accuracy of ship description. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the Fourier contour information parameters of the present invention.

[0029] Figure 2 This is a flowchart of the detection method of the present invention.

[0030] Figure 3 This is a decoding illustration.

[0031] Figure 4 This is a schematic diagram illustrating the calculation of the inverse Fourier transform normalized loss function. Detailed Implementation

[0032] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] The application provides a contour modeling-based optical remote sensing image arbitrary direction ship target detection method, and improves the detection precision and speed of a convolutional neural network for high-resolution optical remote sensing image arbitrary direction ship target.

[0034] As shown in Figure 1 , the application uses a Fourier contour modeling strategy to complete the overall description of the rotating ship target, wherein the contour information parameters of the application specifically include:

[0035] (1) determining the target center point: in view of the problem of difficult positioning in the ship target detection process, the mean value of the horizontal coordinate (x) and the vertical coordinate (y) of the ship target label coordinates is calculated, that is, the center point (O) of the ship target, so as to obtain the position information of the ship target.

[0036] (2) equiangular sampling: since the ship target has a large aspect ratio and the rotating target has an angle periodicity, the Top-down modeling idea is adopted to take the center point of the ship target as the pole, and the equiangular sampling is performed clockwise to obtain the contour sampling information. The sampling point is the intersection point of the equiangular interval ray and the ship contour, that is, the angle is (θi), wherein i=1,2,…,N, and N is the number of sampling points. The corresponding one-dimensional periodic distribution of the contour sampling point is the distance distribution of the intersection of the ray starting from the pole and the contour, that is, (r i).

[0037] (3) determining the scale factor: since the ship target has a multi-scale feature, the scale information is decoupled, and the maximum value of the distance distribution is calculated, that is, the scale factor of the ship target.

[0038] (4) mapping the spatial information to a two-dimensional periodic distribution: according to the use of the scale factor to complete the one-dimensional periodic distribution normalization, the distribution function of the normalized contour sampling point is established.

[0039] (5) determining the shape factor: the number of sampling points of the ship contour and the accuracy of the contour description are difficult to balance, so the two-dimensional periodic distribution curve is subjected to Fourier transform to convert the distribution function to the Fourier domain, and the Fourier coefficient obtained is:​​​​​​​​​​ Since the high-frequency signal contains the overall contour information, and the low-frequency signal contains the detail information of the contour, 2K+1 frequencies are selected for feature description, including K high-frequency features, K low-frequency features and a zero-frequency feature.

[0040] The application utilizes the newly proposed Fourier contour modeling strategy, avoids decoupling several independent feature parameters from the rotating detection frame for feature regression by regressing Fourier contour description coefficients, and effectively solves the regression negative optimization problem caused by angle periodicity and long-short side exchange.

[0041] The specific detection flowchart is shown in Figure 2 The specific detection flowchart is shown in

[0042] Step 1, multi-scale feature extraction is performed on the input high-resolution optical remote sensing image to obtain a multi-scale feature map.

[0043] In this step, ResNet, ResNeXt, DenseNet, HourglassNet and the like can be used to realize multi-scale feature extraction. In this embodiment, ResNet-18 backbone network is used, which has fast training speed, saves computing resources, prevents model overfitting, has strong generalization ability, and has insufficient feature extraction, so that the effectiveness of modeling is more highlighted. After the input high-resolution optical remote sensing image is subjected to multi-scale feature extraction by the ResNet-18 backbone network, a multi-scale feature map C3, C4 and C5 is generated.

[0044] Step 2, feature fusion is performed on the multi-scale feature map to obtain a fused multi-scale pyramid feature map.

[0045] Considering that the scale of the ship target is large, in order to adapt to such changes and further improve the detection efficiency, the deep and shallow network features in the multi-scale feature map of step 1 are fused, the deep network obtains high-level semantic features and has more invariance features, which is convenient for network classification, and the shallow network has more isovariance features and has rich relationship features between the target and the background, which is convenient for target positioning regression.

[0046] In this step, the feature fusion can adopt FPN, PANet, NAS-FPN, CFPN and the like. In this embodiment, FPN is adopted, specifically, 1x1 convolution filter is used to reduce the channel depth of C5, a 3x3 convolution is applied to obtain P5, the result after the channel reduction of C5 is up-sampled by steps of 2, and the result after the 1x1 convolution filter of C4 is added, the addition result is subjected to a 3x3 convolution to obtain P4, then the result after the channel reduction of C4 is up-sampled by steps of 2, and the result after the 1x1 convolution filter of C3 is added, the addition result is subjected to a 3x3 convolution to obtain P3, and finally P5 is up-sampled by steps of 2 to obtain P6.

[0047] Step 3, respectively sending each layer feature map P3, P4, P5 and P6 of step 2 into a classification prediction network, a position factor regression network and a shape scale factor regression network to obtain a classification prediction result, a position factor prediction result and a shape scale factor prediction result;

[0048] The classification prediction network is composed of four groups of convolution modules and a 3x3 convolution operation with a channel number of 2, the convolution module includes a 3x3 convolution operation, a group normalization (Group Normalization) and a ReLU activation function, and two predicted probability distribution maps, i.e. the classification prediction result, are obtained; the position factor regression network is composed of four groups of convolution modules and a 3x3 convolution operation with a channel number of 2, the convolution module includes a 3x3 convolution operation, a group normalization (Group Normalization) and a ReLU activation function, and a position factor prediction result with a channel number of 2 is obtained; the shape scale factor regression network is composed of four groups of convolution modules and a 3x3 convolution operation with a channel number of 2K+3, wherein K is a Fourier series artificially set to describe the target contour, K=16 in this application, the convolution module includes a 3x3 convolution operation, a group normalization (Group Normalization) and a ReLU activation function, and a shape scale factor prediction result with a channel number of 2K+3 is obtained;

[0049] Step 4, using a loss function to correct the position factor, shape factor and scale factor of the target, and the application proposes an intrinsic geometric information loss function to supervise the network learning of the potential information of the rotating ship target. The calculation method of the intrinsic geometric information loss function includes the following four sub-steps:

[0050] S401, classification loss function L cls : The probability value generated by the softmax function indicates the probability value of being predicted in the ship target area, is the ground truth, and the loss function is SmoothL1:

[0051] S402, localization loss function L loc : is the center point coordinate obtained by regression, which represents the position information of the ship target, is the ground truth center point, and the loss function is SmoothL1:

[0052] S403, contour loss function: the predicted shape factor is converted to the time domain by inverse Fourier transform:

[0053] wherein: is the normalized one-dimensional distance distribution of the predicted ship target, is the predicted Fourier coefficient.

[0054] The shape factor calculated from the ground truth is converted to the time domain by inverse Fourier transform:

[0055]

[0056] As Figure 4 , the loss function is calculated in polar coordinates, is the connection value of the normalized one-dimensional distance distribution shape factor and the position factor, and the loss function is SmoothL1:

[0057] S304, intrinsic geometric information loss function: the classification loss function, the localization loss function and the contour loss function are weighted and summed:

[0058]

[0059]

[0060] wherein, N pos , N neg are the number of contour points predicted to be positive and the number of contour points predicted to be negative, respectively; is the classification loss function, are the probability predicted to be in the ship target area and its ground truth, respectively; is the localization loss function, are the i-th position factor predicted and its ground truth, respectively; is the contour loss function, respectively, are the connection values of the i-th shape factor and scale factor and their labeled true values.

[0061] Step 5, the position factor, scale factor and shape factor prediction results obtained in step 3 are decoded to obtain the results of the ship target detection.

[0062] As shown in Figure 3 , the network predicted information is decoded into three sub-steps:

[0063] S5.1, the network predicted shape factor is inverse Fourier transformed:

[0064] S5.2, the inverse Fourier transformed shape factor is denormalized, i.e. multiplied by the corresponding scale factor:

[0065]

[0066] S5.3, the polar coordinate is converted to the Cartesian coordinate system to obtain the coordinate points of the ship target:

[0067] In summary, the above is only a preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting a ship target in an optical remote sensing image in any direction based on profile modeling, characterized in that, The application relates to a ship target detection method based on optical remote sensing image, which comprises the following steps: Step one: multi-scale feature extraction is performed on an optical remote sensing image to obtain a multi-scale feature map; Step two: classification and regression analysis are performed on the multi-scale feature map; the regression analysis is completed by predicting Fourier contour information parameters, the Fourier contour information parameters comprise target center point position information, a scale factor and a shape factor; the scale factor is obtained by performing equi-angle sampling with the target center position as a pole point, the sampling points are intersection points of equi-angle interval rays and a ship contour, and the maximum value of the distances between the sampling points and the pole point is the scale factor; the shape factor is obtained by normalizing the distances between the sampling points and the pole point by using the scale factor, establishing a distribution function of the normalized sampling points, and performing Fourier transform on the distribution function to obtain high-frequency characteristics, low-frequency characteristics and zero-frequency characteristics; Step three: the Fourier contour information parameters predicted in step two are decoded to obtain a ship target detection result, and the decoding comprises the following steps: S31: inverse Fourier transform is performed on the shape factor predicted in step two to obtain a normalized one-dimensional distance distribution of the predicted ship target; S32: the distance distribution obtained in S1 is denormalized based on the scale factor; S33: the denormalized distance distribution is converted into a Cartesian coordinate system to obtain coordinate points of the ship target.

2. The method of claim 1, wherein, The step one further comprises the following steps: feature fusion is performed on the multi-scale feature map to obtain a fused multi-scale feature map; and the fused multi-scale feature map is used to execute subsequent steps.

3. The method of claim 2, wherein, The feature fusion adopts an FPN, PANet, NAS-FPN or CFPN method.

4. The method of claim 3, wherein, The feature fusion specifically comprises: performing convolution and skip connection on each scale feature map C3~C n 1, and generating corresponding feature pyramids P3~P n n n+1 ; and P3~P n+1 are multi-scale feature maps after fusion.​​​ 5. The method according to any one of claims 1 to 4, characterized in that, In the step one, a ResNet-18 backbone network is used to perform multi-scale feature extraction.

6. The method of claim 1, wherein, In the regression analysis of the step two, supervision learning is performed by using an intrinsic geometric information loss function; the intrinsic geometric information loss function is a weighted sum of a classification loss function, a positioning loss function and a contour loss function.

Images