Optical remote sensing image-oriented ship detection method

Abstract

The invention belongs to the technical field of computer vision, and particularly relates to an optical remote sensing image-oriented ship detection method. The method comprises the following steps: S1, obtaining a new label of each ship target; s2, constructing a ship detection network, and obtaining all candidate positive samples based on all pixel points and new labels of the multi-scale feature map obtained by the ship detection network; s3, setting a classification threshold value and a regression threshold value, and obtaining a final positive sample and a final negative sample by using the classification threshold value and the regression threshold value; s4, calculating classification loss and regression loss based on the positive sample and the negative sample, and training the ship detection network according to a loss calculation result to obtain a ship detection model; and S5, processing the to-be-detected optical remote sensing image by using the step S1, and inputting the new label of each ship target of the to-be-detected optical remote sensing image into the ship detection model to realize ship detection of the to-be-detected optical remote sensing image. The ship target in the optical remote sensing image can be accurately detected.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to a ship detection method oriented to optical remote sensing images. Background Art

[0002] Given the urgent need for ship detection in practical applications such as fishery management, maritime patrols, and maritime rescue, ship detection has become an important research direction in the field of computer vision. At the same time, with the rapid development of remote sensing technology, the resolution of optical remote sensing images has been continuously improved, and can provide richer detail information such as color and texture, which creates favorable conditions for ship detection in optical remote sensing images. However, compared with natural images, ships in optical remote sensing images have arbitrary rotation angles. Traditional horizontal bounding boxes cannot accurately represent the direction of the ship and will also introduce too much background information. The introduction of a rotated bounding box alleviates the limitations of the horizontal bounding box and realizes the prediction of the direction of the ship in optical remote sensing images. However, the loss discontinuity problem of the rotated bounding box restricts the improvement of detection accuracy. The essence of the loss discontinuity problem is that the rotation angle At the critical point, it is discrete, while the network is differentiable and continuous. At the critical point, Represents the same direction, however, when the input changes very little, the network prediction result does not change suddenly, but decreases continuously, which leads to the occurrence of wrong predictions. In addition, label assignment is one of the key processes of ship detection, and the general label assignment strategy has two obvious shortcomings in ship detection: (1) Imbalanced sample number: Due to the large scale variation of ships in optical remote sensing images, small ships often match fewer positive samples than large ships, which leads to a small contribution of small ship samples in the loss. (2) Misaligned sample features: The classification and localization tasks in ship detection have different spatial sensitivities. This causes positive samples to focus on different visual features of the input image, and samples at different locations have different prediction accuracies for classification and regression. Samples with accurate localization may not have obvious discriminative information, resulting in the suppression of the detection results generated by them in NMS. In addition, features with strong discriminativeness may not be suitable for localization, resulting in inaccurate detection results. Summary of the invention

[0003] In view of this, the present invention aims to provide a ship detection method for optical remote sensing images to solve the problem of inaccurate detection results in the prior art. The present invention proposes an optical remote sensing image ship detection method based on bounding box representation and label allocation, which can accurately detect ship targets in optical remote sensing images.

[0004] To achieve the above object, the technical solution created by the present invention is implemented as follows: A ship detection method for optical remote sensing images comprises the following steps: S1: Obtain the true label of each ship target in the optical remote sensing image, encode each true label into an inscribed ellipse based on the ellipse feature encoding strategy, and obtain a new label for each ship target; S2: Build a ship detection network, obtain the multi-scale feature map of the optical remote sensing image and the classification prediction results and positioning prediction results of each pixel of the multi-scale feature map based on the ship detection network, map each pixel of the multi-scale feature map to the optical remote sensing image, and calculate the positional relationship between each pixel mapped to the optical remote sensing image and each new label to obtain all candidate positive samples; S3: Calculate the classification quality score and regression quality score of each candidate positive sample by combining the classification prediction results, positioning prediction results, candidate positive samples and new labels corresponding to the candidate positive samples, and adaptively set the classification threshold and regression threshold based on the size of the ship target corresponding to each candidate positive sample, and use the classification threshold and regression threshold to obtain positive samples and negative samples; S4: Calculate the classification loss and regression loss based on the positive samples and negative samples, train the ship detection network according to the loss calculation results, and obtain the ship detection model; S5: Process the optical remote sensing image to be detected by using step S1, and input the new label of each ship target in the optical remote sensing image to be detected into the ship detection model to realize ship detection in the optical remote sensing image to be detected.

[0005] Furthermore, in step S2, the ship detection network is composed of a feature extraction network, a feature pyramid and a head network connected in sequence. The feature extraction network and the feature pyramid are used to obtain a multi-scale feature map of the optical remote sensing image, and the head network is used to obtain classification prediction results and positioning prediction results of each pixel point of the optical remote sensing image.

[0006] Furthermore, step S1 specifically includes the following steps: S11: Get the true label of any ship target j in the optical remote sensing image , the true labels of each ship target are all represented as rectangles, where is the center point of the true label of the current ship target j, is the long side of the rectangle of the real label of the current ship target j, is the short side of the rectangle of the real label of the current ship target j, is the angle between the long side of the rectangle of the real label of the current ship target j and the horizontal direction, ; S12: Establish a two-dimensional coordinate system with the center point of the real label corresponding to the current ship target j as the origin, and construct an inscribed ellipse corresponding to the current real label based on the two-dimensional coordinate system: ; in, are the coordinates of any point e on the inscribed ellipse, is the center point of the inscribed ellipse corresponding to the current ship target j, and , , is the semi-major axis of the inscribed ellipse corresponding to the current ship target j, is the semi-minor axis of the inscribed ellipse corresponding to the current ship target j; S13: Encode the angle of the true label of the current ship target j: ; ; in, is one of the foci of the inscribed ellipse corresponding to the current ship target j, is the inscribed ellipse focus corresponding to the current ship target j To the center of the ellipse The distance is the encoded ellipse focus; S14: Encode the semi-minor axis of the inscribed ellipse corresponding to the current ship target j: ; in, is one of the short axis vertices of the inscribed ellipse corresponding to the current ship target j, is the encoded short axis vertex; S15: Based on the above calculation results, the new label of the current ship target j is set to ; S16: Repeat steps S11 to S15 to obtain new labels for each ship target in the optical remote sensing image.

[0007] Furthermore, step S2 specifically includes: S21: Multi-scale feature map The position of pixel i on Mapping back to the position of the pixel point p of the optical remote sensing image , and based on the new label of the current ship target j, the elliptical sampling area of ​​the current ship target j To model: ; ; ; ; ; ; ; in, for To the center point of the current ship target j The X-direction offset, for To the center point of the current ship target j The Y-direction offset, is the semi-major axis of the elliptical sampling area, is the semi-minor axis of the elliptical sampling area, is the scale balance factor, is the aspect ratio of the current ship target j, is the feature downsampling stride; S22: Perform elliptical full sample sampling for the current ship target j Modeling: ; ; ; in, and Respectively represent the width and length of the current ship target; S23: The elliptical sampling area of ​​the current ship target j and elliptical full sample sampling area The intersection of is taken as a candidate positive sample: ; S24: Repeat steps S21 to S23 until all candidate positive samples in the optical remote sensing image are selected.

[0008] Furthermore, step S3 specifically includes: S31: Obtain candidate positive samples The location tag of the ship target And the position prediction results of the ship target , get candidate positive samples The classification prediction results , candidate positive samples The regression quality score and classification quality score of are expressed as follows: ; in, Candidate positive samples The regression quality score of Candidate positive samples The classification quality score, RIOU is the regression similarity function, is the normalization function; S32: Repeat step S31 to obtain the classification quality score and regression quality score of each candidate positive sample; S33: Based on the data distribution of the regression quality score and the classification quality score of each candidate positive sample and the size of the ship target corresponding to each candidate positive sample, the classification threshold and the regression threshold are calculated respectively by the following formulas: ; ; in, is the area of ​​the current ship target j to be calculated, To adjust the threshold hyperparameters, is the mean of the classification quality scores, is the mean of the regression quality score, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality score; S34: taking the candidate positive samples whose classification quality scores are greater than the classification threshold as classification positive samples, taking the candidate positive samples whose classification quality scores are less than or equal to the classification threshold as classification negative samples, taking the candidate positive samples whose regression quality scores are greater than the regression threshold as regression positive samples, and taking the candidate positive samples whose regression quality scores are less than or equal to the regression threshold as regression negative samples.

[0009] Furthermore, in step S4, the classification loss between the classification prediction result and the classified positive sample, the classification loss between the classification prediction result and the classified negative sample, and the regression loss between the positioning prediction result of the regression positive sample and the new label of the regression positive sample are calculated.

[0010] Compared with the prior art, the invention can achieve the following beneficial effects: The present invention creates the ship detection method for optical remote sensing images, and proposes an ellipse label encoding strategy, an ellipse balance sampling strategy and a dynamic decoupling label allocation strategy. The ellipse label encoding strategy proposed in the present invention can obtain the rectangular label of the ship, and encode the rectangular label into an inscribed ellipse represented by six parameters, and use this six-parameter representation as a new label for the ship target, thereby alleviating the loss discontinuity problem of the rotating frame. The ellipse balance sampling strategy proposed in the present invention achieves adaptive shape allocation by fully learning the geometric characteristics of the ship, balances the number of positive samples between ships of different scales, and improves the contribution of small ship targets in the loss. The dynamic decoupling label allocation strategy proposed in the present invention adaptively evaluates the spatial sensitivity of each sample, autonomously selects dominant positive samples for classification and regression tasks through training, achieves adaptive decoupling of positive samples, and alleviates the problem of feature misalignment of classification and regression. Experimental results show that the ship detection method provided by the present invention can accurately detect ship targets in optical remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A schematic diagram of a process flow of a ship detection method for optical remote sensing images according to an embodiment of the present invention; FIG. 2 (a) is a schematic diagram of a label frame and a prediction frame according to an embodiment of the present invention; FIG2( b ) is a schematic diagram of the problem of discontinuity of the boundary of the rotating frame according to an embodiment of the present invention; FIG3 (a) is a schematic diagram of the problem of sample quantity imbalance in label allocation according to an embodiment of the present invention; FIG3 (b) is a schematic diagram of the sample feature misalignment problem existing in the label allocation according to the embodiment of the present invention; Figure 4 A schematic diagram of the network structure of a ship detection network for optical remote sensing images according to an embodiment of the present invention; Figure 5 A reference schematic diagram for explaining the ellipse feature encoding strategy described in the embodiment of the present invention; Figure 6 A reference schematic diagram for explaining the elliptical balance sampling strategy described in the embodiment of the present invention; Figure 7 A schematic diagram of the allocation process of the dynamic decoupling label allocation strategy described in the embodiment of the present invention; Figure 8 A schematic diagram of the results of ship detection using the ship detection method for optical remote sensing images described in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0013] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0014] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0015] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0016] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0017] like Figure 1As shown in the figure, a ship detection method for optical remote sensing images proposed in the present invention specifically includes the following steps: S1: obtaining the true label of each ship target in the optical remote sensing image, encoding each true label as an inscribed ellipse based on the ellipse feature encoding strategy, and obtaining a new label for each ship target; S2: constructing a ship detection network, obtaining a multi-scale feature map of the optical remote sensing image and a classification prediction result and a positioning prediction result of each pixel point of the multi-scale feature map based on the ship detection network, mapping each pixel point of the multi-scale feature map to the optical remote sensing image, and calculating the position relationship between each pixel point mapped to the optical remote sensing image and each new label, and obtaining all candidate positive samples; S3: combining the classification prediction result and the positioning prediction result The classification quality score and regression quality score of each candidate positive sample are calculated based on the result, candidate positive samples and the new labels corresponding to the candidate positive samples, and the classification threshold and regression threshold are adaptively set in combination with the size of the ship target corresponding to each candidate positive sample, and the classification threshold and regression threshold are used to obtain positive samples and negative samples; S4: the classification loss and regression loss are calculated based on the positive samples and negative samples, and the ship detection network is trained according to the loss calculation result (the sum of the classification loss and the regression loss) to obtain a ship detection model; S5: the optical remote sensing image to be detected is processed by using step S1, and the new labels of each ship target in the optical remote sensing image to be detected are input into the ship detection model to realize ship detection in the optical remote sensing image to be detected.

[0018] First, the optical remote sensing image is sent to the feature extraction network to obtain a multi-scale feature map, and then the classification prediction results and positioning prediction results are generated pixel by pixel on the multi-scale feature map, and then the classification loss between the classification prediction result and the positive and negative samples and the regression loss between the positioning prediction result of the positive sample and the new label composed of six parameters are calculated. Finally, the gradient update of the ship detection network is realized through the loss function (that is, the ship detection network is gradient-decreased according to the calculated loss, and it is iterated repeatedly until the optimal parameters are obtained), thereby promoting the training of the ship detection network and obtaining the ship detection model. For the new label required in the regression loss calculation, the present invention proposes an elliptical feature encoding strategy for the representation of the ship boundary box, and solves the boundary discontinuity problem from the contradiction between the angle discreteness and the network continuity. The present invention obtains the rectangular true label of the ship target, and adopts the elliptical feature encoding strategy to encode the true label into an inscribed ellipse represented by six parameters, and uses this six-parameter representation to construct the new label of each ship target. At the same time, in order to obtain better-performing positive samples to promote the feature learning ability of the ship detection network, the present invention proposes a two-stage label assignment strategy. In the first stage, each pixel of the multi-scale feature map is mapped back to the original optical remote sensing image, and its positional relationship with the new label is calculated. The elliptical balanced sampling strategy is used in combination with the shape and size information of the ship target to achieve the balance of the number of samples between ship targets of different scales, and the pixels located inside the ellipse are selected as candidate samples. Then, the elliptical balanced sampling strategy is used to achieve the balance of sampling between targets of different scales, and the candidate positive samples of the ship target in the first stage are obtained. In the second stage, the dynamic decoupling label assignment strategy combines the prediction results, candidate positive samples and new labels to calculate the classification quality score and regression quality score respectively, and combines the size information of each ship target to adaptively obtain the classification threshold and regression threshold for each ship target, and selects the point with a quality score greater than the threshold as the final positive sample, that is, the dynamic decoupling label assignment strategy allows the ship detection network to autonomously select positive samples with advantages for classification tasks and regression tasks through training, and realizes the adaptive decoupling of positive samples.

[0019] In some embodiments, in step S2, the ship detection network is composed of a feature extraction network, a feature pyramid and a head network connected in sequence, the feature extraction network and the feature pyramid are used to obtain a multi-scale feature map of the optical remote sensing image, and the head network is used to obtain the classification prediction results and positioning prediction results of each pixel point of the optical remote sensing image.

[0020] It should be noted that the feature extraction network is a convolutional neural network, such as Resnet50.

[0021] In some embodiments, step S1 specifically includes the following steps: S11: Get the true label of any ship target j in the optical remote sensing image , the true labels of each ship target are all represented as rectangles, where is the center point of the true label of the current ship target j, is the long side of the rectangle of the real label of the current ship target j, is the short side of the rectangle of the real label of the current ship target j, is the angle between the long side of the rectangle of the real label of the current ship target j and the horizontal direction, ; S12: Establish a two-dimensional coordinate system with the center point of the real label corresponding to the current ship target j as the origin, and construct an inscribed ellipse corresponding to the current real label based on the two-dimensional coordinate system: ; in, are the coordinates of any point e on the inscribed ellipse, is the center point of the inscribed ellipse corresponding to the current ship target j, and , , is the semi-major axis of the inscribed ellipse corresponding to the current ship target j, is the semi-minor axis of the inscribed ellipse corresponding to the current ship target j; S13: Encode the angle of the true label of the current ship target j: ; ; in, is one of the foci of the inscribed ellipse corresponding to the current ship target j, is the inscribed ellipse focus corresponding to the current ship target j To the center of the ellipse The distance is the encoded ellipse focus, which is used to calculate the angle of the current ship target; S14: Encode the semi-minor axis of the inscribed ellipse corresponding to the current ship target j: ; in, is one of the short axis vertices of the inscribed ellipse corresponding to the current ship target j, is the encoded minor axis vertex, which is used to calculate the length of the semi-minor axis of the inscribed ellipse; S15: Based on the above calculation results, the new label of the current ship target j is set to ; S16: Repeat steps S11 to S15 to obtain new labels for each ship target in the optical remote sensing image.

[0022] It should be noted that in step S13, in order to enhance the connection between the true label and the positive sample, the ellipse feature encoding strategy is based on the geometric properties of the ellipse and uses key points to generate the ship detection frame (i.e., the new label). The focus is the core concept of the ellipse and best reflects its ellipse characteristics. Therefore, the angle of the ship target is encoded based on the focus of the ellipse. The ellipse feature encoding strategy provides a continuously changing representation vector for angle prediction based on the focus and center point of the ellipse, solving the boundary discontinuity problem caused by angle mutation at the critical point.

[0023] Furthermore, in step S14, the shape ratio of the ellipse is determined by the major axis and the minor axis. If any two parameters among the major axis, the minor axis and the focal length are known, a unique ellipse can be determined. Since the minor axis is more sensitive to the overall shape change of the ellipse, the semi-minor axis is chosen to be encoded here. The ellipse feature encoding strategy maps the length prediction (the length prediction refers to the length and width of the original label prediction rectangle) to the key point prediction, which enhances the connection between the new label and the positive sample. In addition, the ellipse feature encoding strategy is based on the key point , Guide the learning of ship inspection network.

[0024] In some embodiments, step S2 specifically includes: S21: Multi-scale feature map The position of pixel i on Mapping back to the position of the pixel point p of the optical remote sensing image , and based on the new label of the current ship target j, the elliptical sampling area of ​​the current ship target j To model: ; ; ; ; ; ; ; in, for To the center point of the current ship target j The X-direction offset, for To the center point of the current ship target j The Y-direction offset, is the semi-major axis of the elliptical sampling area, is the semi-minor axis of the elliptical sampling area, is the scale balance factor, is the aspect ratio of the current ship target j, is the feature downsampling stride; S22: Perform elliptical full sample sampling for the current ship target j Modeling: ; ; ; in, and Respectively represent the width and length of the current ship target; S23: The elliptical sampling area of ​​the current ship target j and elliptical full sample sampling area The intersection of is taken as a candidate positive sample: ; S24: Repeat steps S21 to S23 until all candidate positive samples in the optical remote sensing image are selected.

[0025] It should be noted that, in step S22, and Aspect ratio and scale balance factor for each ship target Among them, the aspect ratio can adaptively control the shape distribution of the elliptical sampling area, which is obtained by decoupling the key points. Used to adjust the range of the elliptical sampling area.

[0026] Further, in step S24, since ships usually have a geometric characteristic of being wide in the middle and narrow at both ends, Similarly, the multi-scale feature map The position of each pixel on Mapping back to the original optical remote sensing image position , an elliptical full sample sampling area is modeled for each ship target. and Represent the width and length of the current ship target j respectively, which can be obtained by decoding the key points of the new label.

[0027] In addition, in step S25, for large targets, the elliptical balance sampling strategy selects The high-quality samples in are used as candidate positive samples. For small targets, the elliptical balanced sampling strategy selects All samples in are regarded as candidate positive samples.

[0028] In some embodiments, step S3 specifically includes: S31: Obtain candidate positive samples The location tag of the ship target And the position prediction results of the ship target , get candidate positive samples The classification prediction results , candidate positive samples The regression quality score and classification quality score of are expressed as follows: ; in, Candidate positive samples The regression quality score of Candidate positive samples The classification quality score, RIOU is the regression similarity function, is the normalization function; S32: Repeat step S31 to obtain the classification quality score and regression quality score of each candidate positive sample; S33: Based on the data distribution of the regression quality score and the classification quality score of each candidate positive sample and the size of the ship target corresponding to each candidate positive sample, the classification threshold and the regression threshold are calculated respectively by the following formulas: ; ; in, is the area of ​​the current ship target j to be calculated, To adjust the threshold hyperparameters, is the mean of the classification quality scores, is the mean of the regression quality score, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality score; S34: taking the candidate positive samples whose classification quality scores are greater than the classification threshold as classification positive samples, taking the candidate positive samples whose classification quality scores are less than or equal to the classification threshold as classification negative samples, taking the candidate positive samples whose regression quality scores are greater than the regression threshold as regression positive samples, and taking the candidate positive samples whose regression quality scores are less than or equal to the regression threshold as regression negative samples.

[0029] It should be noted that in step S31, the dynamic decoupling label allocation strategy obtains candidate positive samples The location tag of the ship target And the location prediction results , and by calculating and The similarity between them is used to measure the positioning ability of the candidate positive samples. Then, the dynamic decoupling label assignment strategy is used to obtain the candidate positive samples. Classification prediction results , to measure the classification ability of candidate positive samples. Here, the six parameters encoded and Decode back to five parameters and calculate the RIOU between them to measure the regression similarity. Normalization function The role of is to control the data distribution to maintain a relatively stable state. The dynamic decoupling label assignment strategy enables the ship detection network to autonomously learn the representation ability of samples in different tasks. In step S34, the classification task and the regression task do not share the same positive samples, and all the remaining pixels in each task are used as negative samples.

[0030] Furthermore, the dynamic decoupled label assignment strategy combines the data distribution of the quality score and the ship size information to adaptively obtain the positive sample threshold. It sets a reasonable threshold by calculating the mean and standard deviation of the quality score to screen samples with strong representation ability, and adjusts the threshold size based on the size of the ship target to provide a lower threshold for small ship targets, which alleviates the problem of unbalanced sample numbers of ships of different sizes. The classification threshold and regression threshold are jointly determined by the quality score and ship area of ​​each candidate positive sample, and they are not the same. The dynamic decoupled label assignment strategy selects points with a quality score greater than the threshold as the final positive sample, realizing the decoupling of positive samples in different tasks.

[0031] As shown in Figure 2 (a), with the bow position of the ship target as the positive direction, the actual angle deviation between the original true label and the predicted box is very small. At this time, the actual angle of the ship is 89°, the predicted angle of the ship is 91°, and the loss calculation result is only 2. However, since the angle of the rotating box in ship detection needs to be preset in advance, the current mainstream angle limit method of the rotating box is the angle between the long side of the rectangular box and the horizontal axis, which is within the range of At this time, the ship tag angle is -89°, the predicted angle output result is 89°, and the loss calculation result is 178. This situation where a large loss value occurs at the critical angle because the ideal prediction result exceeds the predefined range is called the loss discontinuity problem. The essence of the loss discontinuity problem is shown in Figure 2 (b). The output angle range is , when the output angle exceeds the predefined range, for example 91°, the angle will suddenly change to -89°, so the angle is discrete. However, the detection network is continuously differentiable and will not suddenly change, but will decrease steadily. Therefore, the essence of loss discontinuity is the contradiction between angle discreteness and network continuity.

[0032] There is an imbalance in the number of samples in label assignment. Most methods regard all pixels of the ship's true label as positive samples. As shown in the left figure of Figure 3 (a), there are 10 positive samples for large ships and only 4 for small ships. The right figure shows the statistical results of the number of positive samples of ships of different scales. That is, assuming that an image has 100 positive samples, 50 are large ships, 34 are medium-sized ships, and only 16 are small ships. Figure 3 (b) shows the problem of sample feature misalignment in label assignment. Here, the feature maps required for classification (Figure 3 (b) left) and regression (Figure 3 (b) right) are visualized. The darker the color, the better the effect of classification and regression. It can be seen that the classification task and the positioning task have different spatial sensitivities. For the positive sample points at the same position (the white points in Figure 3 (a) and Figure 3 (b)), the positive sample point is in the red area of ​​the classification feature map, which is suitable for the classification task and can produce good classification results. However, if the positive sample point is in the light-colored area of ​​the regression feature map, it means that the feature information here is not obvious, which is not conducive to ship regression and it is easy to generate inaccurate prediction boxes.

[0033] like Figure 4 As shown in the figure, the method provided by the present invention mainly consists of three parts: (1) Ellipse feature encoding strategy: responsible for encoding the five-parameter label into a six-parameter label to represent the ship bounding box for subsequent training and prediction. (2) Ellipse balanced sampling strategy: responsible for allocating a relatively balanced number of positive sample samples to ship targets of different scales. (3) Dynamic decoupled label allocation strategy: responsible for decoupling the positive samples involved in training for classification and regression tasks, so as to better learn the characteristics of ship targets.

[0034] like Figure 5 As shown, given a label (10, 10, 50, 40, ), (10, 10) is the center point of the rectangle, the long side of the rectangle is 50, the short side is 40, and the rotation angle is Here, a two-dimensional coordinate system with (10, 10) as the origin is established and the inscribed ellipse of the real label is obtained. The equation of the rotated inscribed ellipse of the rectangular frame is expressed as follows: ; in, Represents the coordinates of any point e on the ellipse, the center point of the ellipse is (10, 10), the semi-major axis of the ellipse is 25, and the semi-minor axis of the ellipse is 20.

[0035] The focus is the core concept of an ellipse and best describes its characteristics. Therefore, the ship angle can be encoded as: ; ; We then encode the semi-minor axis as: ; Therefore, the ellipse feature encoding module converts the labels (10, 10, 50, 40, ) is encoded into a new label of the form (10, 10, 20.6, 20.6, -4.1, 24.1).

[0036] like Figure 6 As shown, given a label (10, 10, 50, 40, ), we have obtained new labels in the form of (10, 10, 20.6, 20.6, -4.1, 24.1). First, model the elliptical sampling area for: ; Among them, the long axis and short axis Aspect ratio and scale balance factor for each ship target Among them, the aspect ratio is decoupled by the new label, and the scale balance factor Used to adjust the sampling area range. and By feature downsampling stride Map back to the original image. Specifically expressed as: ; ; ; In all experiments here, are all set to 1. The feature maps P3, P4, P5, P6, and P7 They are 8, 16, 32, 64 and 128 respectively.

[0037] in, and for To each ship center point The offset of related. The feature map Each pixel position on Mapping back to the coordinates of the original image (optical remote sensing image): ; ; ; For example, It is the pixel (1, 1) at the first position in the upper left corner of the feature map P3. We know the downsampling stride of P3. is 8, so The pixel position on the original image (optical remote sensing image) is (8, 8). We determine whether (8, 8) is in The ship's target It has been obtained above, and then, layer by layer and point by point, it is determined whether the pixel point is middle.

[0038] and Similarly, an elliptical full sample sampling area is modeled for each ship target : ; in, and Same meaning as above, and Represent the width and length of the ship target respectively, and are obtained by decoding the key points of the new label: ; ; at last, and The intersection of the samples in the region is the candidate positive sample finally selected, which is specifically expressed as: .

[0039] like Figure 7 As shown, the candidate positive sample Obtained from the above steps, first, obtain Location Tags And the location prediction results , location tag Assuming the position prediction result is (10, 10, 20.6, 20.6, -4.1, 24.1), is (10, 11, 19, 20, -4.1, 20), and Decode back to five parameters form, and calculate the RIOU value between the two to measure the regression similarity. Then, obtain The classification prediction results , which is used to measure the classification ability of candidate positive samples. The regression quality score and classification quality score of are expressed as follows: ; At the same time, the dynamic decoupled label assignment strategy adaptively obtains the positive sample threshold through the data distribution of the quality score and the ship size information. The threshold is expressed as follows: ; ; in, It represents the area of ​​the current ship target j to be calculated, which is 500 obtained by decoding the length and width of the tag. To adjust the scale factor of the ship area, it is set to 5 here. is a hyperparameter for adjusting the threshold, which is set to 1 here. is the mean of the classification quality scores, is the mean of the regression quality score, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality score. The dynamic decoupled label assignment strategy selects points with quality scores greater than the threshold as the final positive samples, realizing the decoupling of positive samples in different tasks.

[0040] like Figure 8 As shown in Figure 2, for ship targets with a large aspect ratio, inaccurate prediction angles and shapes may lead to missed detection of densely packed ships. Figure 8 As shown in the first row, ships in any direction and densely arranged can be effectively detected. This is because we focus on the geometric features of the ship and convert discrete angles into continuous vectors to generate more accurate prediction boxes. In addition, while paying attention to the spatial sensitivity of different tasks of positive samples, the present invention balances the number of positive samples between ship targets of different scales. Figure 8 As shown in the second row, this enables ship targets of different scales to be accurately detected.

[0041] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0042] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A ship detection method for optical remote sensing images, characterized in that: The specific steps include: S1: Obtain the true label of each ship target in the optical remote sensing image, encode each true label into an inscribed ellipse based on the ellipse feature encoding strategy, and obtain a new label for each ship target; S2: constructing a ship detection network, obtaining a multi-scale feature map of the optical remote sensing image and a classification prediction result and a positioning prediction result of each pixel point of the multi-scale feature map based on the ship detection network, mapping each pixel point of the multi-scale feature map to the optical remote sensing image, and calculating a positional relationship between each pixel point mapped to the optical remote sensing image and each new label, to obtain all candidate positive samples; S3: Calculate the classification quality score and regression quality score of each candidate positive sample by combining the classification prediction results, positioning prediction results, candidate positive samples and new labels corresponding to the candidate positive samples, and adaptively set the classification threshold and regression threshold based on the size of the ship target corresponding to each candidate positive sample, and use the classification threshold and regression threshold to obtain positive samples and negative samples; S4: Calculate the classification loss and regression loss based on the positive samples and negative samples, train the ship detection network according to the loss calculation results, and obtain the ship detection model; S5: Process the optical remote sensing image to be detected by using step S1, and input the new label of each ship target in the optical remote sensing image to be detected into the ship detection model to realize ship detection in the optical remote sensing image to be detected.

2. The ship detection method for optical remote sensing images according to claim 1, characterized in that: In step S2, the ship detection network is composed of a feature extraction network, a feature pyramid and a head network connected in sequence, the feature extraction network and the feature pyramid are used to obtain a multi-scale feature map of the optical remote sensing image, and the head network is used to obtain a classification prediction result and a positioning prediction result of each pixel point of the optical remote sensing image.

3. The ship detection method for optical remote sensing images according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: Get the true label of any ship target j in the optical remote sensing image , the true labels of each ship target are all represented as rectangles, where is the center point of the true label of the current ship target j, is the long side of the rectangle of the real label of the current ship target j, is the short side of the rectangle of the real label of the current ship target j, is the angle between the long side of the rectangle of the real label of the current ship target j and the horizontal direction, ; S12: Establish a two-dimensional coordinate system with the center point of the real label corresponding to the current ship target j as the origin, and construct an inscribed ellipse corresponding to the current real label based on the two-dimensional coordinate system: ； in, are the coordinates of any point e on the inscribed ellipse, is the center point of the inscribed ellipse corresponding to the current ship target j, and , , is the semi-major axis of the inscribed ellipse corresponding to the current ship target j, is the semi-minor axis of the inscribed ellipse corresponding to the current ship target j; S13: Encode the angle of the true label of the current ship target j: ； ； in, is one of the foci of the inscribed ellipse corresponding to the current ship target j, is the inscribed ellipse focus corresponding to the current ship target j To the center of the ellipse The distance is the encoded ellipse focus; S14: Encode the semi-minor axis of the inscribed ellipse corresponding to the current ship target j: ； in, is one of the short axis vertices of the inscribed ellipse corresponding to the current ship target j, is the encoded short axis vertex; S15: Based on the above calculation results, the new label of the current ship target j is set to ; S16: Repeat steps S11 to S15 to obtain new labels for each ship target in the optical remote sensing image.

4. The ship detection method for optical remote sensing images according to claim 3 is characterized in that: Step S2 specifically includes: S21: Multi-scale feature map The position of pixel i on Mapping back to the position of the pixel point p of the optical remote sensing image , and based on the new label of the current ship target j, the elliptical sampling area of ​​the current ship target j To model: ； ； ； ； ； ； ； in, for To the center point of the current ship target j The X-direction offset, for To the center point of the current ship target j The Y-direction offset, is the semi-major axis of the elliptical sampling area, is the semi-minor axis of the elliptical sampling area, is the scale balance factor, is the aspect ratio of the current ship target j, is the feature downsampling stride; S22: Perform elliptical full sample sampling for the current ship target j Modeling: ； ； ； in, and Respectively represent the width and length of the current ship target; S23: The elliptical sampling area of ​​the current ship target j and elliptical full sample sampling area The intersection of is taken as a candidate positive sample: ； S24: Repeat steps S21 to S23 until all candidate positive samples in the optical remote sensing image are selected.

5. The ship detection method for optical remote sensing images according to claim 1, characterized in that: Step S3 specifically includes: S31: Obtain candidate positive samples The location tag of the ship target And the position prediction results of the ship target , get candidate positive samples The classification prediction results , candidate positive samples The regression quality score and classification quality score of are expressed as follows: ； in, Candidate positive samples The regression quality score of Candidate positive samples The classification quality score, RIOU is the regression similarity function, is the normalization function; S32: Repeat step S31 to obtain the classification quality score and regression quality score of each candidate positive sample; S33: Based on the data distribution of the regression quality score and the classification quality score of each candidate positive sample and the size of the ship target corresponding to each candidate positive sample, the classification threshold and the regression threshold are calculated respectively by the following formulas: ； ； in, is the area of ​​the current ship target j to be calculated, To adjust the threshold hyperparameters, is the mean of the classification quality scores, is the mean of the regression quality score, is the standard deviation of the classification quality scores, is the standard deviation of the regression quality score; S34: taking the candidate positive samples whose classification quality scores are greater than the classification threshold as classification positive samples, taking the candidate positive samples whose classification quality scores are less than or equal to the classification threshold as classification negative samples, taking the candidate positive samples whose regression quality scores are greater than the regression threshold as regression positive samples, and taking the candidate positive samples whose regression quality scores are less than or equal to the regression threshold as regression negative samples.

6. The ship detection method for optical remote sensing images according to claim 1, characterized in that: In step S4, the classification loss between the classification prediction result and the classification positive sample, the classification loss between the classification prediction result and the classification negative sample, and the regression loss between the positioning prediction result of the regression positive sample and the new label of the regression positive sample are calculated.

Images