Ship Detection Method for Optical Remote Sensing Images

Abstract

The present invention belongs to the field of computer vision technology, and particularly relates to a ship detection method for optical remote sensing images. The method includes: S1: obtaining new labels for each ship target; S2: constructing a ship detection network, and obtaining all candidate positive samples based on all pixel points of the multi-scale feature maps obtained by the ship detection network and the new labels; S3: setting a classification threshold and a regression threshold, and obtaining final positive samples and negative samples by using the classification threshold and the regression threshold; S4: calculating a classification loss and a regression loss based on the positive samples and the negative samples, and training the ship detection network according to the loss calculation results to obtain a ship detection model; S5: processing the optical remote sensing image to be detected by using step S1, and inputting the new labels of each ship target of the optical remote sensing image to be detected into the ship detection model to realize ship detection of the optical remote sensing image to be detected. The present invention can accurately detect ship targets in optical remote sensing images.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a ship detection method for optical remote sensing images. Background Art

[0002] In view of the urgent needs of ship detection in practical applications such as fishery management, maritime patrol, and maritime rescue, ship detection has become an important research direction in the field of computer vision. At the same time, with the rapid development of remote sensing technology, the resolution of optical remote sensing images has been continuously improved, providing more abundant detailed information such as color and texture, which creates favorable conditions for ship detection in optical remote sensing images. However, compared with natural images, ships in optical remote sensing images have arbitrary rotation angles. Traditional horizontal bounding boxes cannot accurately represent the direction of ships and will introduce too much background information. The proposed rotated bounding box alleviates the limitations of horizontal bounding boxes and realizes the direction prediction of ships in optical remote sensing images. However, the discontinuous loss problem existing in rotated bounding boxes restricts the improvement of detection accuracy. The essence of the discontinuous loss problem is that the rotation angle is discrete at critical points, while the network is differentiable and continuous. At critical points, represents the same direction. However, when the input changes slightly, the network prediction result does not mutate but decreases continuously, which may lead to incorrect predictions. In addition, label assignment is one of the key processes in ship detection, and general label assignment strategies have two obvious disadvantages in ship detection: (1) Unbalanced sample quantity: Due to the large variation in ship scales in optical remote sensing images, compared with large ships, small ships often match fewer positive samples, which results in a small contribution of small ship samples to the loss. (2) Misalignment of sample features: The classification and localization tasks in ship detection have different spatial sensitivities. This leads to positive samples focusing on different visual features of the input image, and samples at different positions have different prediction accuracies for classification and regression. Samples with accurate localization may not have obvious discriminative information, resulting in their generated detection results being suppressed in NMS. Additionally, features with strong discriminability may not be suitable for localization, resulting in inaccurate detection results. Summary of the Invention

[0003] In view of this, the present invention aims to provide a ship detection method for optical remote sensing images to solve the detection problem of inaccurate detection results in the prior art. The present invention proposes an optical remote sensing image ship detection method based on bounding box representation and label assignment, which can accurately detect ship targets in optical remote sensing images.

[0004] To achieve the above object, the technical solution of the present invention is realized as follows:

[0005] A ship detection method for optical remote sensing images, specifically including the following steps:

[0006] S1: Obtain the true labels of each ship target in the optical remote sensing image, and encode each true label as an inscribed ellipse based on the ellipse feature encoding strategy to obtain new labels for each ship target;

[0007] S2: Construct a ship detection network, obtain the multi-scale feature map of the optical remote sensing image and the classification prediction results and localization prediction results of each pixel point of the multi-scale feature map based on the ship detection network, map each pixel point of the multi-scale feature map to the optical remote sensing image, and calculate the positional relationship between each pixel point mapped to the optical remote sensing image and each new label to obtain all candidate positive samples;

[0008] S3: Calculate the classification quality score and regression quality score of each candidate positive sample in combination with the classification prediction results, localization prediction results, candidate positive samples and the new labels corresponding to the candidate positive samples, adaptively set the classification threshold and regression threshold in combination with the sizes of the ship targets corresponding to each candidate positive sample, and obtain positive samples and negative samples using the classification threshold and regression threshold;

[0009] S4: Calculate the classification loss and regression loss based on the positive samples and negative samples, train the ship detection network according to the loss calculation results to obtain a ship detection model;

[0010] S5: Process the optical remote sensing image to be detected using step S1, and input the new labels of each ship target in the optical remote sensing image to be detected into the ship detection model to realize the ship detection of the optical remote sensing image to be detected.

[0011] Furthermore, in step S2, the ship detection network is composed of a feature extraction network, a feature pyramid and a head network connected in sequence. The feature extraction network and the feature pyramid are used to obtain the multi-scale feature map of the optical remote sensing image, and the head network is used to obtain the classification prediction results and localization prediction results of each pixel point of the optical remote sensing image.

[0012] Furthermore, step S1 specifically includes the following steps:

[0013] S11: Obtain the true label of any ship target j in the optical remote sensing image , the true labels of each ship target are all in the form of rectangles, where, is the center point of the true label of the current ship target j, is the long side of the rectangle of the true label of the current ship target j, is the short side of the rectangle of the true label of the current ship target j, is the included angle between the long side of the rectangle of the true label of the current ship target j and the horizontal direction, ;

[0014] S12: Establish a two-dimensional coordinate system with the center point of the true label corresponding to the current ship target j as the origin, and construct an inscribed ellipse corresponding to the current true label based on the two-dimensional coordinate system:

[0015] ;

[0016] Among them, is the coordinate of any point e on the inscribed ellipse, is the center point of the inscribed ellipse corresponding to the current ship target j, and , , is the semi-major axis of the inscribed ellipse corresponding to the current ship target j, is the semi-minor axis of the inscribed ellipse corresponding to the current ship target j;

[0017] S13: Encode the angle of the true label of the current ship target j:

[0018] ;

[0019] ;

[0020] Among them, is one of the foci of the inscribed ellipse corresponding to the current ship target j, is the focus of the inscribed ellipse corresponding to the current ship target j to the ellipse center distance, is the encoded ellipse focus;

[0021] S14: Encode the semi-minor axis of the inscribed ellipse corresponding to the current ship target j:

[0022] ;

[0023] Among them, is one of the short-axis vertices of the inscribed ellipse corresponding to the current ship target j, is the encoded short-axis vertex;

[0024] S15: Based on the above calculation results, set the new label of the current ship target j as ;

[0025] S16: Repeat steps S11~S15 to obtain the new labels of each ship target in the optical remote sensing image.

[0026] Furthermore, step S2 specifically includes:

[0027] S21: The multi-scale feature map The position of pixel point i on is mapped back to the position of pixel point p in the optical remote sensing image , and based on the new label of the current ship target j, the elliptical sampling region of the current ship target j is modeled as follows:

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] Among them, is the offset in the X direction from to the center point of the current ship target j is the offset in the Y direction from to the center point of the current ship target j is the semi-major axis of the elliptical sampling region, is the semi-minor axis of the elliptical sampling region, is the scale balance factor, is the aspect ratio of the current ship target j, is the feature downsampling stride;

[0036] S22: Model the elliptical full-sample sampling region for the current ship target j:

[0037] ;

[0038] ;

[0039] ;

[0040] Among them, and represent the width and length of the current ship target respectively;

[0041] S23: Take the intersection of the elliptical sampling region of the current ship target j and the elliptical full-sample sampling region as the candidate positive sample:

[0042] ;

[0043] S24: Repeat steps S21 to S23 until all candidate positive samples in the optical remote sensing image are selected.

[0044] Furthermore, step S3 specifically includes:

[0045] S31: Obtain candidate positive samples The location tag of the ship target And the position prediction results of the ship target , get candidate positive samples Classification prediction results , candidate positive samples The regression quality score and classification quality score of are expressed as follows:

[0046] ;

[0047] in, Candidate positive samples The regression quality score of Candidate positive samples The classification quality score, RIOU is the regression similarity function, is the normalization function;

[0048] S32: Repeat step S31 to obtain the classification quality score and regression quality score of each candidate positive sample;

[0049] S33: Based on the data distribution of the regression quality score and classification quality score of each candidate positive sample and the size of the ship target corresponding to each candidate positive sample, the classification threshold and regression threshold are calculated respectively by the following formula:

[0050] ;

[0051] ;

[0052] in, is the area of the current ship target j to be calculated, To adjust the threshold hyperparameters, is the mean of the classification quality scores, is the mean of the regression quality score, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality score, is the scale factor for adjusting the ship area;

[0053] S34: Use the candidate positive samples with classification quality scores greater than the classification threshold as classification positive samples, use the candidate positive samples with classification quality scores less than or equal to the classification threshold as classification negative samples, use the candidate positive samples with regression quality scores greater than the regression threshold as regression positive samples, and use the candidate positive samples with regression quality scores less than or equal to the regression threshold as regression negative samples.

[0054] Further, in step S4, calculate the classification loss between the classification prediction result and the classification positive sample, the classification loss between the classification prediction result and the classification negative sample, and the regression loss between the localization prediction result of the regression positive sample and the new label of the regression positive sample.

[0055] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0056] The ship detection method for optical remote sensing images according to the present invention proposes an elliptical label encoding strategy, an elliptical balanced sampling strategy, and a dynamic decoupled label assignment strategy. The elliptical label encoding strategy proposed by the present invention can obtain the rectangular label of the ship, encode the rectangular label into an inscribed ellipse represented by six parameters, and use this six-parameter representation method as the new label of the ship target, alleviating the problem of discontinuous loss of the rotated box. The elliptical balanced sampling strategy proposed by the present invention realizes the adaptive allocation of shapes by fully learning the geometric characteristics of the ship, balances the number of positive samples among ships of different scales, and improves the contribution of small ship targets in the loss. The dynamic decoupled label assignment strategy proposed by the present invention adaptively evaluates the spatial sensitivity of each sample, and autonomously selects the dominant positive samples for the classification and regression tasks through training, realizing the adaptive decoupling of positive samples and alleviating the problem of feature misalignment between classification and regression. Experimental results show that the ship detection method provided by the present invention can accurately detect ship targets in optical remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0058] Figure 1 is a schematic flowchart of the ship detection method for optical remote sensing images according to the embodiment of the present invention;

[0059] FIG. 2(a) is a schematic diagram of the label box and the prediction box according to the embodiment of the present invention;

[0060] FIG. 2(b) is a schematic diagram of the problem of discontinuous rotated box boundary according to the embodiment of the present invention;

[0061] FIG. 3(a) is a schematic diagram of the problem of unbalanced sample numbers existing in the label assignment according to the embodiment of the present invention;

[0062] FIG. 3(b) is a schematic diagram of the problem of misaligned sample features existing in the label assignment according to the embodiment of the present invention;

[0063] Figure 4 is a schematic diagram of the network structure of the ship detection network for optical remote sensing images according to the embodiment of the present invention;

[0064] Figure 5 is a reference schematic diagram for explaining the elliptical feature coding strategy according to the embodiment of the present invention;

[0065] Figure 6 is a reference schematic diagram for explaining the elliptical balanced sampling strategy according to the embodiment of the present invention;

[0066] Figure 7 is a schematic diagram of the assignment process of the dynamic decoupled label assignment strategy according to the embodiment of the present invention;

[0067] Figure 8 is a schematic diagram of the result of ship detection using the ship detection method for optical remote sensing images according to the embodiment of the present invention. Detailed implementation manners

[0068] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention.

[0069] It should be noted that, without conflict, the embodiments and features in the embodiments of the present invention may be combined with each other.

[0070] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0071] In the description of the present invention, it should be noted that, unless otherwise clearly specified and defined, the terms "mounted", "connected", and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be a direct connection or an indirect connection through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific circumstances.

[0072] The present invention will be described in detail below with reference to the drawings and in conjunction with embodiments.

[0073] Such as Figure 1As shown in the figure, a ship detection method for optical remote sensing images proposed by the present invention specifically includes the following steps: S1: Obtain the true labels of each ship target in the optical remote sensing image, and encode each true label as an inscribed ellipse based on the ellipse feature encoding strategy to obtain new labels for each ship target; S2: Construct a ship detection network, and based on the ship detection network, obtain the multi-scale feature map of the optical remote sensing image and the classification prediction results and localization prediction results of each pixel point of the multi-scale feature map, map each pixel point of the multi-scale feature map to the optical remote sensing image, and calculate the positional relationship between each pixel point mapped to the optical remote sensing image and each new label to obtain all candidate positive samples; S3: Combine the classification prediction results, localization prediction results, candidate positive samples and the new labels corresponding to the candidate positive samples to calculate the classification quality score and regression quality score of each candidate positive sample, adaptively set the classification threshold and regression threshold according to the size of the ship target corresponding to each candidate positive sample, and use the classification threshold and regression threshold to obtain positive samples and negative samples; S4: Calculate the classification loss and regression loss based on the positive samples and negative samples, and train the ship detection network according to the loss calculation results (the sum of the classification loss and the regression loss) to obtain a ship detection model; S5: Use the method in step S1 to process the optical remote sensing image to be detected, and input the new labels of each ship target in the optical remote sensing image to be detected into the ship detection model to realize the ship detection of the optical remote sensing image to be detected.

[0074] First, the optical remote sensing image is sent into the feature extraction network to obtain multi-scale feature maps. Then, classification prediction results and localization prediction results are generated pixel by pixel on the multi-scale feature maps. Subsequently, the classification loss between the classification prediction results and positive and negative samples, as well as the regression loss between the localization prediction results of positive samples and the new labels composed of six parameters, are calculated. Finally, the gradient update of the ship detection network is achieved through the loss function (that is, gradient descent is performed on the ship detection network according to the calculated loss, and iterated repeatedly until the optimal parameters are obtained), thereby promoting the training of the ship detection network and obtaining the ship detection model. For the new labels required in the regression loss calculation, the present invention proposes an elliptical feature encoding strategy for the representation of ship bounding boxes and solves the problem of boundary discontinuity starting from the contradiction between angular discreteness and network continuity. The present invention obtains the rectangular ground truth labels of ship targets and encodes the ground truth labels as inscribed ellipses represented by six parameters using the elliptical feature encoding strategy, and constructs new labels for each ship target using this six-parameter representation method. At the same time, to obtain better positive samples to promote the feature learning ability of the ship detection network, the present invention proposes a two-stage label assignment strategy. In the first stage, each pixel point of the multi-scale feature map is mapped back to the original optical remote sensing image, and its positional relationship with the new label is calculated. The elliptical balanced sampling strategy is used to balance the sampling quantity between ship targets of different scales by combining the shape and size information of ship targets, and the pixels located inside the ellipse are selected as candidate samples. Then, the elliptical balanced sampling strategy is used to achieve the balance of sampling between different-scale targets, and the candidate positive samples of ship targets in the first stage are obtained. In the second stage, the dynamic decoupling label assignment strategy calculates the classification quality score and the regression quality score respectively by combining the prediction results, candidate positive samples, and new labels, and adaptively obtains the classification threshold and the regression threshold for each ship target by combining the size information of each ship target, and selects the points with quality scores greater than the threshold as the final positive samples. That is, the dynamic decoupling label assignment strategy enables the ship detection network to autonomously select advantageous positive samples for the classification task and the regression task through training, achieving the adaptive decoupling of positive samples.

[0075] In some embodiments, in step S2, the ship detection network is composed of a feature extraction network, a feature pyramid, and a head network connected in sequence. The feature extraction network and the feature pyramid are used to obtain multi-scale feature maps of the optical remote sensing image, and the head network is used to obtain the classification prediction results and localization prediction results of each pixel point of the optical remote sensing image.

[0076] It should be noted that the feature extraction network is a convolutional neural network, such as Resnet50.

[0077] In some embodiments, step S1 specifically includes the following steps:

[0078] S11: Obtain the ground truth label of any ship target j in the optical remote sensing image , the true labels of each ship target are all rectangular. Among them, is the center point of the true label of the current ship target j, is the long side of the rectangle of the true label of the current ship target j, is the short side of the rectangle of the true label of the current ship target j, is the included angle between the long side of the rectangle of the true label of the current ship target j and the horizontal direction, ;

[0079] S12: Establish a two-dimensional coordinate system with the center point of the true label corresponding to the current ship target j as the origin, and construct an inscribed ellipse corresponding to the current true label based on the two-dimensional coordinate system:

[0080] ;

[0081] Among them, is the coordinate of any point e on the inscribed ellipse, is the center point of the inscribed ellipse corresponding to the current ship target j, and , , is the semi-major axis of the inscribed ellipse corresponding to the current ship target j, is the semi-minor axis of the inscribed ellipse corresponding to the current ship target j;

[0082] S13: Encode the angle of the true label of the current ship target j:

[0083] ;

[0084] ;

[0085] Among them, is one of the foci of the inscribed ellipse corresponding to the current ship target j, is the focus of the inscribed ellipse corresponding to the current ship target j to the center of the ellipse distance, is the encoded ellipse focus, used to calculate the angle of the current ship target;

[0086] S14: Encode the semi-minor axis of the inscribed ellipse corresponding to the current ship target j:

[0087] ;

[0088] Among them, is one of the short-axis vertices of the inscribed ellipse corresponding to the current ship target j, is the encoded short-axis vertex, used to calculate the length of the semi-minor axis of the inscribed ellipse;

[0089] S15: Based on the above calculation results, set the new label of the current ship target j as ;

[0090] S16: Repeat steps S11 - S15 to obtain the new labels of each ship target in the optical remote sensing image.

[0091] It should be noted that in step S13, to enhance the connection between the real label and the positive sample, the elliptical feature encoding strategy, based on the geometric properties of the ellipse, uses key points to generate the ship detection box (i.e., the new label). The foci of the ellipse are the core concepts and can best reflect the characteristics of the ellipse. Therefore, the angle of the ship target is encoded based on the foci of the ellipse. The elliptical feature encoding strategy provides a continuously varying representation vector for angle prediction based on the foci and the center point of the ellipse, solving the problem of discontinuous boundaries caused by sudden angle changes at critical points.

[0092] Furthermore, in step S14, the shape ratio of the ellipse is jointly determined by the major axis and the minor axis. Given any two of the major axis, minor axis, and focal length parameters, a unique ellipse can be determined. Since the minor axis is more sensitive to the overall shape change of the ellipse, here the semi - minor axis is selected for encoding. The elliptical feature encoding strategy maps the length prediction (the length prediction refers to the length and width of the original label prediction rectangle) to key point prediction, enhancing the connection between the new label and the positive sample. In addition, the elliptical feature encoding strategy is based on key points , to guide the ship detection network to learn.

[0093] In some embodiments, step S2 specifically includes:

[0094] S21: Map the position of pixel point i on the multi - scale feature map back to the position of pixel point p in the optical remote sensing image , and model the elliptical sampling region of the current ship target j based on the new label of the current ship target j : ;

[0095] ;

[0096] ;

[0097] ; [[ID=@46]]

[0098] ;

[0099] ;

[0100] ;

[0101] ;

[0102] Among them, is the offset in the X direction to the center point of the current ship target j . is the offset in the Y direction to the center point of the current ship target j . is the semi-major axis of the elliptical sampling area, is the semi-minor axis of the elliptical sampling area, is the scale balance factor, is the aspect ratio of the current ship target j, is the feature downsampling stride;

[0103] S22: Model the elliptical full-sample sampling area of the current ship target j:

[0104] ;

[0105] ;

[0106] ;

[0107] Among them, and represent the width and length of the current ship target respectively;

[0108] S23: Take the intersection of the elliptical sampling area of the current ship target j and the elliptical full-sample sampling area as the candidate positive sample:

[0109] ;

[0110] S24: Repeat steps S21~S23 until all candidate positive samples in the optical remote sensing image are selected.

[0111] It should be noted that in step S22, and are related to the aspect ratio and scale balance factor of each ship target. Among them, the aspect ratio can adaptively control the shape distribution of the elliptical sampling area and is obtained by decoupling key points. The scale balance factor is used to adjust the range of the elliptical sampling area.

[0112] Further, in step S24, since a ship usually has a geometric feature of being wide in the middle and narrow at both ends. Therefore, similar to the elliptical sampling region , the position of each pixel point on the multi-scale feature map is mapped back to the position of the original optical remote sensing image to model an elliptical full-sample sampling region for each ship target. And and represent the width and length of the current ship target j respectively, which can be obtained by decoding the key points of the new label.

[0113] Additionally, in step S25, for large targets, the elliptical balanced sampling strategy selects the high-quality samples in as candidate positive samples. For small targets, the elliptical balanced sampling strategy selects

[0114] all the samples in

[0115] as candidate positive samples. In some embodiments, step S3 specifically includes: S31: Obtain the position label of the ship target of the candidate positive sample and the position prediction result of the ship target, obtain the classification prediction result

[0116] of the candidate positive sample ;

[0117] Among them, is the regression quality score of the candidate positive sample , is the classification quality score of the candidate positive sample , RIOU is the regression similarity function, is the normalization function;

[0118] S32: Repeat step S31 to obtain the classification quality score and regression quality score of each candidate positive sample;

[0119] S33: Based on the data distribution of the regression quality score and classification quality score of each candidate positive sample and the size of the ship target corresponding to each candidate positive sample, calculate the classification threshold and regression threshold respectively through the following formula:

[0120] ;

[0121] ;

[0122] Among them, is the area of the ship target j to be currently calculated, is the hyperparameter for adjusting the threshold, is the mean of the classification quality scores, is the mean of the regression quality scores, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality scores;

[0123] S34: Take the candidate positive samples with classification quality scores greater than the classification threshold as classification positive samples, take the candidate positive samples with classification quality scores less than or equal to the classification threshold as classification negative samples, take the candidate positive samples with regression quality scores greater than the regression threshold as regression positive samples, and take the candidate positive samples with regression quality scores less than or equal to the regression threshold as regression negative samples.

[0124] It should be noted that in step S31, the dynamic decoupling label assignment strategy obtains the position labels of the ship targets of the candidate positive samples and the position prediction results , and measures the positioning ability of the candidate positive samples by calculating the similarity between and . Then, the dynamic decoupling label assignment strategy obtains the classification prediction results of the candidate positive samples to measure the classification ability of the candidate positive samples. Here, the encoded six parameters and are decoded back to five parameters, and the RIOU between them is calculated to measure the regression similarity. The role of the normalization function is to control the data distribution to maintain a relatively stable state. The dynamic decoupling label assignment strategy enables the ship detection network to autonomously learn the representation ability of samples in different tasks. In step S34, the classification task and the regression task do not share the same positive samples, and all the remaining pixel points in each task are used as negative samples.

[0125] Furthermore, the dynamic decoupling label assignment strategy adaptively obtains the positive sample threshold by combining the data distribution of the quality scores and the ship size information. It sets reasonable thresholds by calculating the mean and standard deviation of the quality scores to screen samples with strong representation ability, and adjusts the threshold size in combination with the size of the ship target, providing a lower threshold for small ship targets, which alleviates the problem of unbalanced sample numbers of ships of different sizes. The classification threshold and the regression threshold are jointly determined by the respective quality scores of the candidate positive samples and the ship area, and they are not the same. The dynamic decoupling label assignment strategy selects the points with quality scores greater than the threshold as the final positive samples, achieving the decoupling of positive samples in different tasks.

[0126] As shown in Figure 2(a), taking the bow position of the ship target as the positive direction, the actual angle deviation between the original true label and the predicted bounding box is very small. At this time, the actual angle of the ship is 89°, the predicted ship angle is 91°, and the loss calculation result is only 2. However, since the angle of the rotated bounding box needs to be preset within a certain range in ship detection, the current mainstream way to define the angle of the rotated bounding box is the angle between the long side of the rectangular box and the horizontal axis, and the range is . At this time, the ship label angle is -89°, the predicted angle output result is 89°, and the loss calculation result is 178. This situation where a large loss value appears due to the ideal predicted result exceeding the predefined range at the angle critical point is called the loss discontinuity problem. The essence of the loss discontinuity problem is shown in Figure 2(b). The output angle range is in . When the output angle exceeds the predefined range, such as 91°, the angle will suddenly change to -89°, so the angle is discrete. However, the detection network is continuously differentiable and will not have sudden changes but will decrease smoothly. Therefore, the essence of the loss discontinuity is the contradiction between the angle discreteness and the network continuity.

[0127] There is a problem of unbalanced sample quantity in label assignment. Most methods regard all pixel points of the ship true label as positive samples. As shown in the left figure of Figure 3(a), a large ship has 10 positive samples, while a small ship only has 4. The right figure shows the statistical results of the number of positive samples of ships of different scales, that is, assuming there are 100 positive samples in an image, 50 are of large ships, 34 are of medium-sized ships, and only 16 belong to small ships. Figure 3(b) shows the problem of misalignment of sample features in label assignment. Here, the feature maps required for classification (the left figure of Figure 3(b)) and regression (the right figure of Figure 3(b)) are visualized, and the darker the color, the better the effect of classification and regression here. It can be seen that the classification task and the localization task have different spatial sensitivities. For the positive sample points at the same position (the white points in Figure 3(a) and Figure 3(b)), this positive sample point is in the red area of the classification feature map, suitable for the classification task, and can produce better classification results. However, if this positive sample point is in the light-colored area of the regression feature map, it indicates that the feature information here is not obvious, which is not conducive to ship regression and is likely to generate inaccurate predicted bounding boxes.

[0128] As Figure 4 shown, the method provided by the present invention mainly consists of three parts: (1) Ellipse feature encoding strategy: responsible for encoding the five-parameter label into a six-parameter label to represent the ship bounding box for subsequent training and prediction. (2) Ellipse balanced sampling strategy: responsible for allocating relatively balanced positive sample sampling quantities for ship targets of different scales. (3) Dynamic decoupled label assignment strategy: responsible for decoupling the positive samples participating in training for the classification and regression tasks to better learn the ship target features.

[0129] AsFigure 5 As shown, assume a ship target with a label of (10, 10, 50, 40, ). Here, (10, 10) is the center point of the rectangle, the long side of the rectangle is 50, the short side is 40, and the rotation angle is . Here, a two-dimensional coordinate system with (10, 10) as the origin is established and the inscribed ellipse of the true label is obtained. The equation of the rotated inscribed ellipse of the rectangle is expressed as follows:

[0130] ;

[0131] Among them, represents the coordinates of any point e on the ellipse. The center point of the ellipse is (10, 10), the semi-major axis of the ellipse is 25, and the semi-minor axis of the ellipse is 20.

[0132] The focus is the core concept of the ellipse and can best reflect its characteristics. Therefore, the ship angle can be encoded as:

[0133] ;

[0134] ;

[0135] Then, we encode the semi-minor axis as:

[0136] ;

[0137] Therefore, the ellipse feature encoding module encodes the label (10, 10, 50, 40, ) into a new label in the form of (10, 10, 20.6, 20.6, -4.1, 24.1).

[0138] As Figure 6 shown, assume a ship target with a label of (10, 10, 50, 40, ). We have obtained a new label in the form of (10, 10, 20.6, 20.6, -4.1, 24.1). First, model the ellipse sampling area as:

[0139] ;

[0140] Among them, the major axis and the minor axis are related to the aspect ratio and the scale balance factor of each ship target. Among them, the aspect ratio is decoupled from the new label, and the scale balance factor is used to adjust the sampling area range. and are mapped back to the original image through the feature downsampling stride . Specifically expressed as:

[0141] ;

[0142] ;

[0143] ;

[0144] Here, in all experiments, is set to 1. The of the feature maps P3, P4, P5, P6, and P7 are 8, 16, 32, 64, and 128 respectively.

[0145] Among them, and are the offsets to each ship center point and are related to . is the coordinate of each pixel position on the feature map mapped back to the original image (optical remote sensing image):

[0146] ;

[0147] ;

[0148] ;

[0149] For example, is the pixel point (1, 1) at the top-left first position on the feature map P3. We know that the downsampling stride of P3 is 8. Therefore, the pixel position on the original image (optical remote sensing image) is (8, 8). We determine whether (8, 8) is in . The of the ship target has been obtained above. Then, we judge point by point layer by layer whether the pixel points are in .

[0150] Similar to , an elliptical full-sample sampling area is modeled for each ship target:

[0151] ;

[0152] Among them, and have the same meaning as above. and represent the width and length of the ship target respectively, and are obtained by decoding the key points of the new label:

[0153] ;

[0154] ;

[0155] Finally, and the intersection of the samples within the area of is the finally selected candidate positive sample, which is specifically expressed as:

[0156] .

[0157] As Figure 7 shown, the candidate positive sample has been obtained from the above steps. First, obtain the position label of and the position prediction result . The position label is (10, 10, 20.6, 20.6, -4.1, 24.1). Assuming the position prediction result is (10, 11, 19, 20, -4.1, 20), decode and back to the five-parameter form, and calculate the RIOU value between the two to measure the regression similarity. Then, obtain the classification prediction result of to measure the classification ability of the candidate positive sample. The regression quality score and classification quality score of

[0158] ;

[0159] Meanwhile, the dynamic decoupled label assignment strategy adaptively obtains the positive sample threshold through the data distribution of the quality scores and the ship size information. The threshold is expressed as follows:

[0160] ;

[0161] ;

[0162] Among them, represents the area of the current ship target j to be calculated, which is obtained as 500 by the length and width decoded from the label. is the scale factor for adjusting the ship area, which is set to 5 here. is the hyperparameter for adjusting the threshold, which is set to 1 here. is the mean of the classification quality scores, is the mean of the regression quality scores, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality score. The dynamic decoupled label assignment strategy selects points with a quality score greater than the threshold as the final positive samples, achieving the decoupling of positive samples in different tasks.

[0163] As Figure 8 shown, for ship targets with extremely large aspect ratios, inaccurate predicted angles and shapes may lead to the problem of missed detection of dense ships. As Figure 8 shown in the first row, ships arranged densely in any direction can be effectively detected. This is due to the fact that we focus on the geometric features of ships and convert discrete angles into continuous vectors to generate more accurate prediction boxes. In addition, while paying attention to the sensitivity of different task spaces of positive samples, the present invention balances the number of positive samples among ship targets of different scales. As Figure 8 shown in the second row, this enables ship targets of different scales to be accurately detected.

[0164] It should be understood that various forms of the processes shown above can be used, reordering, adding, or deleting steps. For example, the steps described in the disclosure of the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and no limitation is made herein.

[0165] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A ship detection method for optical remote sensing images, characterized in that: Specifically, it includes the following steps: S1: Obtain the true labels of each ship target in the optical remote sensing image, and encode each true label as an inscribed ellipse based on the ellipse feature encoding strategy to obtain new labels for each ship target; S11: Obtain the true label of any ship target j in the optical remote sensing image , the true labels of each ship target are all rectangular, where is the center point of the true label of the current ship target j, is the long side of the rectangle of the true label of the current ship target j, is the short side of the rectangle of the true label of the current ship target j, is the included angle between the long side of the rectangle of the true label of the current ship target j and the horizontal direction, ; S12: Establish a two-dimensional coordinate system with the center point of the true label corresponding to the current ship target j as the origin, and construct an inscribed ellipse corresponding to the current true label based on the two-dimensional coordinate system: ； Among them, is the coordinate of any point e on the inscribed ellipse, is the center point of the inscribed ellipse corresponding to the current ship target j, and , , is the semi-major axis of the inscribed ellipse corresponding to the current ship target j, is the semi-minor axis of the inscribed ellipse corresponding to the current ship target j; S13: Encode the angle of the true label of the current ship target j: ； ； Among them, is one of the foci of the inscribed ellipse corresponding to the current ship target j, is the focus of the inscribed ellipse corresponding to the current ship target j to the center of the ellipse distance, is the encoded ellipse focus; S14: Encode the semi-minor axis of the inscribed ellipse corresponding to the current ship target j: ； Among them, is one of the minor axis vertices of the inscribed ellipse corresponding to the current ship target j, is the encoded minor axis vertex; S15: Based on the above calculation results, set the new label of the current ship target j as ; S16: Repeat steps S11 - S15 to obtain new labels for each ship target in the optical remote sensing image; S2: Construct a ship detection network, obtain the multi-scale feature map of the optical remote sensing image and the classification prediction results and localization prediction results of each pixel point of the multi-scale feature map based on the ship detection network, map each pixel point of the multi-scale feature map to the optical remote sensing image, and calculate the positional relationship between each pixel point mapped to the optical remote sensing image and each new label to obtain all candidate positive samples; S21: Map the position of pixel point i on the multi-scale feature map back to the position of pixel point p on the optical remote sensing image , and model the elliptical sampling region of the current ship target j based on the new label of the current ship target j : ​ ； ； ； ； ； ； ； Among them, is the offset in the X direction to the center point of the current ship target j, and is the offset in the Y direction to the center point of the current ship target j, is the semi-major axis of the elliptical sampling area, is the semi-minor axis of the elliptical sampling area, is the scale balance factor, is the aspect ratio of the current ship target j, is the feature downsampling stride;​ S22: Conduct modeling for the elliptical full-sample sampling area of the current ship target j : ； ； ； Wherein, and respectively represent the width and length of the current ship target; S23: Use the intersection of the elliptical sampling region of the current ship target j and the elliptical full-sample sampling region as the candidate positive sample: ； S24: Repeat steps S21 - S23 until all candidate positive samples in the optical remote sensing image are selected; S3: Calculate the classification quality score and regression quality score of each candidate positive sample by combining the classification prediction results, localization prediction results, candidate positive samples and the new labels corresponding to the candidate positive samples, adaptively set the classification threshold and regression threshold according to the size of the ship target corresponding to each candidate positive sample, and obtain positive samples and negative samples using the classification threshold and regression threshold; S31: Obtain candidate positive samples The position label of the ship target And the position prediction result of the ship target , Obtain candidate positive samples The classification prediction result of , Candidate positive samples The regression quality score and classification quality score of are expressed as follows: ； Among them, is the regression quality score of the candidate positive sample , is the classification quality score of the candidate positive sample, RIOU is the regression similarity function , is the normalization function; S32: Repeat step S31 to obtain the classification quality score and regression quality score of each candidate positive sample; S33: Calculate the classification threshold and regression threshold respectively through the following formula based on the data distribution of the regression quality score and classification quality score of each candidate positive sample and the size of the ship target corresponding to each candidate positive sample: ； ； Among them, is the area of the ship target j to be currently calculated, is the hyperparameter for adjusting the threshold, is the mean of the classification quality scores, is the mean of the regression quality scores, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality scores, is the scale factor for adjusting the ship area; S34: Take the candidate positive samples with a classification quality score greater than the classification threshold as classification positive samples, take the candidate positive samples with a classification quality score less than or equal to the classification threshold as classification negative samples, take the candidate positive samples with a regression quality score greater than the regression threshold as regression positive samples, and take the candidate positive samples with a regression quality score less than or equal to the regression threshold as regression negative samples; S4: Calculate the classification loss and regression loss based on the positive samples and negative samples, and train the ship detection network according to the loss calculation result to obtain a ship detection model; S5: Use step S1 to process the optical remote sensing image to be detected, and input the new labels of each ship target in the optical remote sensing image to be detected into the ship detection model to realize ship detection of the optical remote sensing image to be detected.

2. The ship detection method for optical remote sensing images according to claim 1, characterized in that: In step S2, the ship detection network is composed of a feature extraction network, a feature pyramid and a head network connected in sequence. The feature extraction network and the feature pyramid are used to obtain the multi-scale feature map of the optical remote sensing image, and the head network is used to obtain the classification prediction results and localization prediction results of each pixel point of the optical remote sensing image.

3. The ship detection method for optical remote sensing images according to claim 1, characterized in that: In step S4, calculate the classification loss between the classification prediction result and the classification positive sample, the classification loss between the classification prediction result and the classification negative sample, and the regression loss between the localization prediction result of the regression positive sample and the new label of the regression positive sample.

Images