A method and apparatus for detecting ship target rotation in optical remote sensing images

Abstract

This invention discloses a method and apparatus for detecting ship targets rotation in optical remote sensing images. The method includes: acquiring image data to be detected; wherein the image data to be detected includes a port image to be detected and corresponding label information of the port image; inputting the image data to be detected into a preset network model to obtain a feature map of the port image to be detected; calculating the optimal detection weight of the network model based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected; and obtaining the ship target detection result based on the optimal detection weight. This invention improves the detection accuracy of ship targets in optical remote sensing images, focusing on solving the difficult problems of low detection accuracy and low detection efficiency of large-scale targets in complex scenarios such as near-shore ports.

Description

Technical Field

[0001] This invention relates to the field of remote sensing application technology, and in particular to a method and apparatus for detecting the rotation of ship targets in optical remote sensing images. Background Technology

[0002] In recent decades, with the rapid development of programming languages ​​and GPU computing power, deep learning methods based on neural networks have been widely applied in various industries, such as facial recognition, disaster monitoring, and target detection. Their outstanding detection performance and open-source code have garnered widespread acclaim. However, ship targets in optical remote sensing images exhibit characteristics such as arbitrary orientation, large aspect ratios, and dense distribution in specific scenes. This presents new challenges and significant difficulties for deep learning-based ship target detection methods in optical remote sensing images.

[0003] Currently, in the field of deep learning-based ship target detection, mainstream methods mostly employ rotated bounding box detection. These methods generate rotatable rectangular boxes by predicting additional angles to fit the ship target. Clearly, compared to horizontal detection methods, rotated detection can better fit the ship target by relying on the rotational properties of the bounding box, thus solving the problem of arbitrary target orientation and achieving better detection accuracy. However, these methods do not consider scenarios where a large number of ship targets are densely packed, such as when many ships are moored in near-shore ports. Therefore, current rotated detection methods generally have low detection accuracy in complex scenarios such as near-shore areas. At the same time, ship targets in visible light remote sensing images exhibit a large aspect ratio, a characteristic that traditional deep learning methods based on the Intersection-over-Union (IoU) metric do not consider, resulting in poor detection performance.

[0004] To effectively address the challenge of low detection accuracy in complex scenes, an attention mechanism has emerged that allows the network to focus on important regions. Its key feature is that it increases the weight of key regions and critical locations containing the target object in the feature image, thereby reducing interference from background information and irrelevant clutter, thus improving the detection accuracy of ship targets. Generally, attention mechanisms can be categorized into three modes: spatial attention, channel attention, and hybrid attention. Spatial attention focuses on the two-dimensional spatial characteristics of the input image, guiding the network to extract spatial information more accurately based on the target's positional relationships within the image, forming a feature image with more accurate positional information. Channel attention focuses on the channel characteristics of the image, guiding the network to utilize channel information more accurately based on the global correlations between channels, forming a feature map with richer scale information. Hybrid attention combines both approaches, leveraging their respective advantages. Typically, deep learning network architectures consist of three stages: Backbone, Neck, and Head. The Backbone stage primarily extracts two-dimensional feature information from an image to generate a feature image. The Neck stage fuses and processes the feature images generated by the Backbone, making reasonable use of the channel information of the feature images to ultimately output a feature image containing specific information. The Head stage calculates the loss on the feature image output from the Neck stage to obtain the final detection result. Clearly, the Backbone stage focuses on the spatial characteristics of the image, the Neck stage pays more attention to the channel characteristics, and the Head stage includes the design and calculation of the loss function. Current research often employs a hybrid attention approach, placing it in the Neck stage of the network to guide the network in accurately utilizing the extracted feature information, thereby improving detection accuracy. However, simply using hybrid attention in the Neck stage does not fully utilize the main functions and operating characteristics of different stages of the network, thus failing to achieve the expected performance gains and leading to a sharp increase in network complexity and computational overhead.

[0005] Furthermore, existing deep learning network architectures based on the IoU metric do not consider the large aspect ratio of ship targets. To address this issue, there are two main improvement strategies: one is to use SkewIoU as the metric, which mitigates the negative impact of ignoring aspect ratio by sensing angular changes during rotation detection; the second is to use CompleteIntersectionOverUnion (CIoU) as the metric, which incorporates the target's aspect ratio as a penalty term into the IoU, thereby improving detection accuracy. However, SkewIoU, as a compromise, still fundamentally does not consider the impact of aspect ratio. While CIoU quantifies the impact of aspect ratio, it is insensitive to angular changes during rotation detection, thus affecting detection efficiency. Therefore, existing methods still struggle to achieve accurate detection of targets with large aspect ratios during rotation. Summary of the Invention

[0006] The present invention aims to at least partially solve one of the technical problems in the related art.

[0007] Therefore, the purpose of this invention is to propose a method for rotating ship targets in optical remote sensing images, which can improve the detection accuracy of ship targets in optical remote sensing images and focus on solving the difficult problems of low detection accuracy and low detection efficiency of large and wide targets in complex scenarios such as near-shore ports.

[0008] Another objective of this invention is to provide a device for detecting the rotation of ship targets in optical remote sensing images.

[0009] To achieve the above objectives, this invention proposes a method for detecting the rotation of ship targets in optical remote sensing images, comprising:

[0010] Acquire the image data to be detected; wherein, the image data to be detected includes the port image to be detected and the corresponding label information of the port image to be detected;

[0011] The image data to be detected is input into a preset network model to obtain the feature map of the port image to be detected. The optimal detection weight of the network model is obtained by calculating the loss based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected.

[0012] The ship target detection result of the image data to be detected is obtained based on the optimal detection weight.

[0013] In addition, the optical remote sensing image ship target rotation detection method according to the above embodiments of the present invention may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present invention, the network model is a dual-path segmentation attention network DSA-Net. The step of inputting the port image data to be detected into the preset network model to obtain the port image feature map includes: in the Backbone stage of the DSA-Net, the network utilizes the spatial location information of the port image to be detected to form a multi-scale feature map by adding a context location module (CLM); and in the Neck stage of the DSA-Net, the network fuses the channel information of the multi-scale feature map to obtain the final feature map by adding a global channel module (GCM).

[0015] Furthermore, in one embodiment of the present invention, the step of extracting ship target feature information contained in the port image to be detected by adding a context location module (CLM) in the Backbone stage of DSA-Net to form a multi-scale feature map includes: in the Backbone stage of DSA-Net, inputting a port image to be detected of a preset size; and through a preset pooling operation of a convolutional neural network and by utilizing the context location module (CLM) according to the context location relationship, enabling the network to obtain the multi-scale feature map using the spatial location information of the port image to be detected of the preset size.

[0016] Furthermore, in one embodiment of the present invention, the step of fusing the channel information of the multi-scale feature map to obtain the final feature map by adding a Global Channel Module (GCM) in the Neck stage of DSA-Net includes: in the Neck stage of DSA-Net, using an interpolation method and the added Global Channel Module (GCM), the network performs a fusion operation on the channel information of the multi-scale feature map to obtain the final feature map.

[0017] Furthermore, in one embodiment of the present invention, the step of calculating the optimal detection weights of the network model based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected includes: adding a skew cross intersection and comparing SkewCIoU in the Head stage of the DSA-Net; using the SkewCIoU loss function to calculate the loss between the feature information of the extracted final feature map and the corresponding label information of the port image to be detected; obtaining the difference result based on the loss calculation, and obtaining the optimal detection weights of the network model based on the difference result.

[0018] To achieve the above objectives, another aspect of the present invention provides a device for detecting the rotation of ship targets in optical remote sensing images, comprising:

[0019] The data acquisition module is used to acquire image data to be detected; wherein, the image data to be detected includes a port image to be detected and corresponding label information of the port image to be detected;

[0020] The weight acquisition module is used to input the image data to be detected into a preset network model to obtain the feature map of the port image to be detected, and to calculate the optimal detection weight of the network model based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected.

[0021] The image detection module is used to obtain the ship target detection result of the image data to be detected based on the optimal detection weight.

[0022] The optical remote sensing image ship target rotation detection method and apparatus of the present invention introduce a dual-path segmentation attention mechanism and oblique full intersection-to-union loss to improve the detection accuracy of ship targets in optical remote sensing images, and focus on solving the difficulties of low detection accuracy and low detection efficiency of large and wide targets in complex scenarios such as near-shore ports.

[0023] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 This is a flowchart of a method for detecting the rotation of ship targets in optical remote sensing images according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of an optical remote sensing image ship target rotation detection architecture according to an embodiment of the present invention;

[0027] Figure 3 These are attention heatmaps illustrating the effects of DSA (Distraction and Analysis) on attention without and with DSA, according to embodiments of the present invention.

[0028] Figure 4 This is a comparison chart of experimental results at different metric scales according to an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of the PR curves of various methods in different scenarios according to embodiments of the present invention;

[0030] Figure 6 This is a schematic diagram of the detection results according to an embodiment of the present invention;

[0031] Figure 7This is a schematic diagram of the structure of an optical remote sensing image ship target rotation detection device according to an embodiment of the present invention. Detailed Implementation

[0032] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] The following description, with reference to the accompanying drawings, describes a method and apparatus for detecting the rotation of ship targets in optical remote sensing images according to embodiments of the present invention.

[0035] Figure 1 This is a flowchart of a method for detecting the rotation of ship targets in optical remote sensing images according to an embodiment of the present invention.

[0036] like Figure 1 As shown, the method includes, but is not limited to, the following steps:

[0037] S1, acquire the image data to be detected; wherein, the image data to be detected includes the port image to be detected and the corresponding label information of the port image to be detected;

[0038] S2, input the image data to be detected into the preset network model to obtain the feature map of the port image to be detected, and calculate the loss based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected to obtain the optimal detection weight of the network model;

[0039] S3, based on the optimal detection weights, obtain the ship target detection results of the image data to be detected.

[0040] Specifically, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0041] This invention proposes a dual-separation attention network (DSA-Net) architecture based on SkewComplete Intersection-over-Union (SkewCIoU) for rotational detection of ship targets in optical remote sensing images. Specifically, for the spatial characteristics of the network backbone stage, a Contextual Location Module (CLM) is proposed as spatial attention to specifically guide the network to accurately locate and extract image feature information; for the channel characteristics of the network neck stage, a Global Channel Module (GCM) is used as channel attention to specifically guide the network to fully utilize the inter-channel correlation information of the feature image. Because it can accurately adapt to the working characteristics of different stages of the network, the DSA strategy can not only effectively improve the network's detection performance but also avoid redundant computation, achieving a gain in detection accuracy without significantly increasing computational complexity. At the same time, this invention also introduces a differentiable SkewCIoU loss, enabling the network to achieve more accurate and efficient detection of ship targets with large length and width bit dimensions. Specifically, firstly, in the existing SkewIoU scale, a center point distance penalty term and an aspect ratio penalty term are introduced to quantify the influence of distance and aspect ratio. Simultaneously, addressing the operational challenge that SkewCIoU cannot be differentiated and therefore cannot directly participate in backpropagation, this invention introduces an angle-dependent SkewCIoU constant factor during the regression process to ensure the reliability of the loss calculation.

[0042] As an example, Figure 2 This is a diagram of the DSA-Net network architecture according to an embodiment of the present invention, such as... Figure 2 As shown. The green, red, and black arrows represent downsampling, upsampling, and concatenation operations, respectively. and These represent the addition and multiplication of corresponding elements within the tensor, respectively. The gray cuboids represent feature maps at different scales, and C×H×W represent the number of channels, length, and width of the feature map, respectively.

[0043] Specifically, in the Backbone stage of DSA-Net, the addition of the Context Location Module (CLM) enables the network to utilize the spatial location information of the port image to be detected to form a multi-scale feature map; and in the Neck stage of DSA-Net, the addition of the Global Channel Module (GCM) enables the network to fuse the channel information of the multi-scale feature maps to obtain the final feature map.

[0044] It is understood that this embodiment of the invention uses the publicly available dataset HRSC2016 as both the training and testing datasets. The HRSC2016 dataset contains 1070 images and 2976 ship targets. 632 images are selected as the training dataset, and the remaining 438 images are used as the testing dataset. To verify the detection performance of this invention in complex near-shore scenes, the 438 test images are further subdivided into 398 near-shore images and 40 marine images, serving as two supplementary test datasets.

[0045] As an example, in the Backbone stage of DSA-Net, a port image to be detected of a preset size is input; through the pooling operation of a preset convolutional neural network and the use of the Context Position Module (CLM) based on the context position relationship, the network uses the spatial position information of the port image to be detected of the preset size to obtain a multi-scale feature map.

[0046] Specifically, based on the image size requirements of the network, the input image size is uniformly adjusted to 608×608 before being input into the network.

[0047] In the Backbone stage, the input image is downsampled to form feature maps of different scales.

[0048] Specifically, in the Backbone stage, the input image is downsampled through pooling operations of the convolutional neural network to obtain feature images at different scales.

[0049] The designed CLM modules are placed sequentially at the end of each downsampling step.

[0050] Specifically, the specific structure of CLM is as follows: Figure 2 As shown, the CLM consists of two 1×1 convolutional layers, one 3×3 convolutional layer, and a sigmoid activation function. From top to bottom, the first 1×1 convolutional layer compresses the channel dimension of the feature map, the 3×3 convolutional layer increases the network's receptive field, and the second 1×1 convolutional layer adjusts the channel scale of the feature map. The sigmoid activation function is used to calculate the spatial weights of the feature map. This process can be calculated as follows:

[0051]

[0052] Among them, F CLM This represents the spatial feature map obtained after CLM operations, where F represents the initial spatial feature map. and These represent element-wise multiplication and addition within the tensor, respectively. is a convolution operator, representing convolution computation, where n represents the kernel size, m represents the computation order, and σ represents the Sigmoid function computation. Specifically, it can be represented as:

[0053]

[0054] As an example, in the Neck stage of DSA-Net, interpolation methods and the added Global Channel Module (GCM) enable the network to fuse the channel information of multi-scale feature maps to obtain the final feature map.

[0055] Understandably, during the Neck stage, feature maps of different scales are fused and utilized through upsampling and concatenation operations.

[0056] Specifically, in the Neck stage, the extracted feature images are upsampled using interpolation, and then feature maps of the same scale are merged to complete the fusion and utilization of the feature maps.

[0057] Add the GCM module at the end of the Neck phase and before entering the Head phase.

[0058] Specifically, the specific structure of GCM is as follows: Figure 2 As shown, GCM consists of three 1×1 convolutional layers, one Softmax activation function, and one LayerNorm operation and one ReLU activation function. In top-to-bottom order, the first 1×1 convolutional layer adjusts the channel dimensions, the Softmax activation function calculates the channel weights, the second 1×1 convolutional layer passes the channel weights, the LayerNorm operation acts as a regularization term to improve the network's generalization performance, the ReLU operation preserves positive values ​​in the matrix after convolution, reducing overall computation, and the third 1×1 convolutional layer adjusts the channel scale and passes it to the initial feature map. This process can be calculated as follows:

[0059]

[0060] F GCM This represents the channel feature map after GCM operation. This represents the initial channel feature map. It is a with Convolution computations with different weights are also convolution operators. δ represents the softmax function computation, which can be specifically expressed as:

[0061]

[0062] The RELU calculation process can be represented as:

[0063] RELU(x) = max(0,x)

[0064] The LN calculation process can be represented as follows:

[0065]

[0066] Where E[x] and Var[x] represent the expectation and variance of the input x, respectively, ∈ is a constant representing skewness, and the parameters γ and β are equal to the number of embedded elements.

[0067] As an example, a skew cross intersection and SkewCIoU comparison are added to the Head stage of DSA-Net; the SkewCIoU loss function is used to calculate the loss between the feature information of the extracted final feature map and the corresponding label information of the port image to be detected; the difference result is obtained according to the loss calculation, and the optimal detection weight of the network model is obtained based on the difference result.

[0068] In the Head phase, a rotating rectangle is designed to fit the real ship target.

[0069] Specifically, five parameters (x, y, w, h, θ) are used to describe a rotating rectangle in any direction. Here, x and y represent the x and y coordinates of the rectangle's center, respectively; w and h are the width and height of the rectangle, respectively; and θ represents the angle between the rectangle and the x-axis, with a range of [missing information]. The process of generating the rotated frame is as follows:

[0070]

[0071] t θ =θ-θ a

[0072]

[0073] t′ θ =θ′-θ a

[0074] Where t is the migration parameter, which represents the difference between the actual ship target and the predicted rotating frame, and x, x a x' represents the x-coordinate of the center of the true target bounding box, the x-coordinate of the center of the preset anchor box, and the x-coordinate of the center of the predicted box, respectively. The same logic applies to the four parameters y, w, h, and θ. Therefore, the goal of this embodiment is to find a suitable preset anchor box that is equal to the predicted rotated box and coincides with the true target bounding box as much as possible.

[0075] Design the SkewCIoU loss function, perform regression calculations to obtain the final detection results, and save the corresponding network weights.

[0076] Specifically, the Head stage of this network obtains the final detection result by designing a reasonable loss function. The loss function of this network is designed as follows:

[0077]

[0078] N represents the number of anchor boxes generated, and ∈′ and ∈ represent the predicted offset vector and the ground truth offset vector, respectively, both containing the predicted values ​​(x, y, w, h, θ). n It is a binary value, equal to 1 indicating a foreground target (a ship target within the anchor point box), and equal to 0 indicating a background target (no ship target within the anchor point box). C n ′ and C n p represents the predicted confidence value and the true confidence value, respectively. n ′ and p n Let λ1, λ2, λ3, λ4 represent the probability values ​​calculated using the Softmax function and the true probability values, respectively. {λ1, λ2, λ3, λ4} are four hyperparameters affecting computational performance, with values ​​of (5, 5, 0.5, 1), respectively. reg The regression calculation for the rotated frame can be represented as:

[0079]

[0080] in:

[0081]

[0082] Where θ′ and θ represent the predicted angle offset and the true angle offset, respectively, x represents the input, and in this invention, x = θ′ - θ. CIoU represents the crossover ratio, which can be calculated using the following formula:

[0083]

[0084] Where IoU(R,R) gt ) represents the rectangular box R and its truth box R. gt The intersection-union ratio:

[0085]

[0086] Area represents the area, ∪ and ∩ represent the union and the bag set respectively, ρ() represents the distance between two points, and o and o gt Represent the center point of the rectangle R and its truth box R, respectively. gt The center point of R. c represents the area containing R and R. gt The diagonal length of the smallest enclosed region. v is used to measure the impact of the target's aspect ratio and can be calculated as:

[0087]

[0088] w and h represent width and height, respectively. α is a balancing parameter used to bridge the relationship between IoU and aspect ratio, and can be calculated as:

[0089]

[0090] In loss calculation, SkewCIoU is the proposed skew cross union ratio, which is defined as follows:

[0091]

[0092] Where SkewIoU represents the oblique intersection-to-union ratio, which can be calculated as:

[0093]

[0094] Where I represents the relationship between R and R gt The region obtained by triangulation of the intersection of the two regions. Obviously, the SkewCIoU proposed in this invention introduces center point distance and aspect ratio penalty terms on the basis of SkewIoU, realizing an effective combination of CIoU and SkewIoU. In addition, a constant factor representing angle change is introduced into SkewCIoU, achieving good angle perception performance in rotation detection.

[0095] In addition, this invention introduces SkewCIoU into the loss calculation in the form of a modulus, which effectively solves the problem of SkewCIoU being nondifferentiable, allowing it to directly participate in the backpropagation of the network to solve for the optimal weights.

[0096] Furthermore, the saved network weights are loaded, the test set is tested, recall, precision, F1 score, mean precision are calculated, and PR curves are plotted.

[0097] Specifically, weights adapted to the network are obtained through network loss calculation, loaded and saved for testing. The final evaluation metrics are: Recall, Precision, F1 score, Average Precision (AP), and Average AP for each class (mAP). In single-class object detection, the mAP value is equal to the AP value. AP... 0.5 and AP 0.75These represent the AP values ​​when CIoU is 0.5 and 0.75, respectively. To measure the above criteria, we predefine the following concepts: During the detection process, if the network correctly identifies a sample as a ship (the sample is indeed a ship), the sample is called a true positive sample (TP); if the network incorrectly identifies a sample as a ship (the sample is not a ship), the sample is called a false positive sample (FP); if the network correctly identifies a sample as not a ship (the sample is not a ship), the sample is called a true negative sample (TN); if the network incorrectly identifies a sample as not a ship (the sample is a ship), the sample is called a false negative sample (FN). The above evaluation metrics can then be calculated using the following formula:

[0098]

[0099]

[0100]

[0101] The curve drawn with Recall on the x-axis and Precision on the y-axis is called the PR curve. The area under the curve is equal to the AP value. Therefore, AP can be calculated as:

[0102]

[0103] Furthermore, in order to verify the authenticity and effectiveness of the present invention, tests were performed on a publicly available test set and compared with other existing advanced methods.

[0104] DSA strategy effectiveness verification experiment. To visually demonstrate the performance of the dual-path segmentation attention mechanism in this invention, attention heatmaps were extracted for both the absence and inclusion of the DSA strategy, as shown below. Figure 3As shown, from left to right, the images are the original image, the attention heatmap without the DSA strategy, and the attention heatmap with the DSA strategy. As the color deepens, it indicates that the network pays more attention to that area. It can be clearly seen that without DSA, the network's focus is not on the ship target, while with DSA, the network pays more attention to the area encompassing the ship target, thus improving accuracy. Furthermore, to further verify and quantify the specific impact of the proposed DSA strategy on the network architecture's performance, four comparative experiments were conducted using the YOLOv4-Oriented model as a benchmark: 1. Adding the CLM module only in the Backbone stage; 2. Adding the GCM module only in the Neck stage; 3. Using both CLM and GCM in the Neck stage as a hybrid attention; 4. Using the DSA strategy, i.e., adding CLM in the Backbone stage and GCM in the Neck stage. The specific detection results are shown in Table 1. It can be seen that adding CLM and GCM separately improved the network's detection accuracy by 1.13% and 2.70%, respectively, demonstrating the effectiveness of the CLM and GCM modules designed in this invention. Furthermore, hybrid attention and DSA improved the network's detection accuracy by 3.63% and 4.78%, respectively, indicating that the DSA strategy performs better than the traditional method of adding hybrid attention in the Neck phase. In summary, the dual-path segmentation attention mechanism proposed in this invention allows the network to focus more intently on the detected ship target, offsetting interference from background or clutter, thereby improving detection accuracy. Moreover, compared to the traditional hybrid attention method, the dual-path segmentation attention mode proposed in this invention performs better.

[0105] Table 1

[0106]

[0107] SkewCIoU Effectiveness Comparison. To verify the effectiveness of the SkewCIoU metric proposed in this invention, networks with different metric scales were trained independently, and the trained weights were then tested. Specific results are as follows: Figure 4 As shown, from left to right, the values ​​are the training loss and the detection accuracy. Figure 4As can be seen, in terms of training loss, the SkewCIoU proposed in this invention exhibits the best training performance compared to traditional SkewIoU and CIoU. This is reflected in two aspects: firstly, the overall oscillation amplitude during training is minimized; and secondly, the final convergence loss value is minimized. Regarding testing accuracy, detection results were selected at epochs of 50, 100, 150, and 200. It can be seen that SkewCIoU maintains good accuracy throughout, especially when the network converges, i.e., at epoch = 200, where the detection accuracy of SkewCIoU is maximized, indicating optimal network performance.

[0108] Detection and experimental comparison results. This invention was compared with some industry-recognized high-performance methods on the publicly available optical remote sensing image dataset HRSC2016, specifically designed for ship detection, and two scenarios were distinguished: nearshore and offshore. The detection results for different scenarios are shown in Table 2. It can be seen that this invention achieves higher Precision, F1 score, and AP in the nearshore scenario. 0.75 AP 0.5 The values ​​are all the highest. In offshore scenarios, the AP of this invention... 0.75 and AP 0.5 The value is the highest. Therefore, the overall performance of this invention is good, especially in nearshore scenarios, where its performance advantage is more obvious. PR curves for different scenarios are as follows: Figure 5 As shown, from left to right, the PR curves of each method are for nearshore scenarios and offshore scenarios. It can be seen that, regardless of whether it's a nearshore or offshore scenario, the curve corresponding to this invention has the largest area under the curve, indicating that this invention has the highest detection accuracy. Meanwhile, test experiments were conducted on the total test dataset, and the results are shown in Table 3. It can be seen that this invention has the highest detection accuracy (90.41%) and ranks second in speed (27.8 fps). Therefore, compared to other advanced methods, this invention not only has the highest detection accuracy but also ranks highly in detection speed. Furthermore, visualizations of some detection results of this invention can be found in [reference needed]. Figure 6 .

[0109] Table 2

[0110]

[0111] Table 3

[0112]

[0113] The method for detecting rotating ship targets in optical remote sensing images according to embodiments of the present invention can improve the detection accuracy of ship targets in optical remote sensing images, and focuses on solving the difficult problems of low detection accuracy and low detection efficiency of large and wide targets in complex scenarios such as near-shore ports.

[0114] To achieve the above embodiments, such as Figure 7 As shown, this embodiment also provides an optical remote sensing image ship target rotation detection device 10, which includes: a data acquisition module 100, a weight acquisition module 200 and an image detection module 300.

[0115] The data acquisition module 100 is used to acquire the image data to be detected; wherein, the image data to be detected includes the port image to be detected and the corresponding label information of the port image to be detected;

[0116] The weight acquisition module 200 is used to input the image data to be detected into the preset network model to obtain the feature map of the port image to be detected, and to calculate the loss based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected to obtain the optimal detection weight of the network model.

[0117] The image detection module 300 is used to obtain the ship target detection result of the image data to be detected based on the optimal detection weight.

[0118] Furthermore, the network model is a dual-path segmentation attention network DSA-Net, and the aforementioned weight acquisition module 200 is specifically used for:

[0119] In the backbone stage of DSA-Net, the added Context Location Module (CLM) enables the network to utilize the spatial location information of the port image to be detected, forming multi-scale feature maps; and,

[0120] In the Neck stage of DSA-Net, the addition of the Global Channel Module (GCM) enables the network to fuse channel information from multi-scale feature maps to obtain the final feature map.

[0121] Furthermore, the aforementioned weight acquisition module 200 is also used for:

[0122] In the Backbone stage of DSA-Net, a port image of a preset size is input;

[0123] By using the pooling operation of a pre-defined convolutional neural network and the context location module (CLM) based on the context location relationship, the network can obtain multi-scale feature maps using the spatial location information of the port image to be detected of a pre-defined size.

[0124] Furthermore, the aforementioned weight acquisition module 200 is also used for:

[0125] In the Neck stage of DSA-Net, interpolation methods and the added Global Channel Module (GCM) are used to enable the network to fuse the channel information of multi-scale feature maps to obtain the final feature map.

[0126] Furthermore, the aforementioned weight acquisition module 200 is also used for:

[0127] Add skew-crossing to the head stage of DSA-Net and compare SkewCIoU;

[0128] The SkewCIoU loss function is used to calculate the loss between the feature information of the extracted final feature map and the corresponding label information of the port image to be detected;

[0129] The difference is calculated based on the loss, and the optimal detection weights of the network model are obtained based on the difference.

[0130] The optical remote sensing image ship target rotation detection device according to the present invention can improve the detection accuracy of ship targets in optical remote sensing images, and focuses on solving the difficult problems of low detection accuracy and low detection efficiency of large and wide targets in complex scenarios such as near-shore ports.

[0131] It should be noted that the foregoing explanation of the embodiment of the ship target rotation detection method based on optical remote sensing images also applies to the optical remote sensing image ship target rotation detection device of this embodiment, and will not be repeated here.

[0132] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0133] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0134] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting the rotation of ship targets in optical remote sensing images, characterized in that, Includes the following steps: Acquire the image data to be detected; wherein, the image data to be detected includes the port image to be detected and the corresponding label information of the port image to be detected; The image data to be detected is input into a preset network model to obtain the feature map of the port image to be detected. The optimal detection weight of the network model is obtained by calculating the loss based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected. The ship target detection result of the image data to be detected is obtained based on the optimal detection weight; The network model is a dual-path segmentation attention network (DSA-Net). The step of inputting the port image data to be detected into the preset network model to obtain the port image feature map includes: In the backbone stage of the DSA-Net, the added Context Location Module (CLM) enables the network to utilize the spatial location information of the port image to be detected to form a multi-scale feature map; and, In the Neck stage of the DSA-Net, the addition of the Global Channel Module (GCM) enables the network to fuse the channel information of the multi-scale feature maps to obtain the final feature map. The step of calculating the optimal detection weights of the network model based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected includes: In the Head stage of the DSA-Net, a skew-crossover loop is added and compared with SkewCIoU; The SkewCIoU loss function is used to calculate the loss between the feature information of the extracted final feature map and the corresponding label information of the port image to be detected; The difference result is obtained based on the loss calculation, and the optimal detection weight of the network model is obtained based on the difference result; The head stage of the network obtains the final detection result by designing a reasonable loss function. The network's loss function is designed as follows: N represents the number of anchor boxes generated. and These represent the predicted cancellation vector and the true cancellation vector, respectively, both containing the predicted values ​​(x, y, w, h, ...). x and y represent the horizontal and vertical coordinates of the center of the rectangle, respectively, and w and h represent the width and height, respectively. Indicates the actual angle offset; It is a binary value, where 1 represents a foreground object and 0 represents a background object. and These represent the predicted confidence level and the actual confidence level, respectively. and These represent the probability value calculated using the Softmax function and the true probability value, respectively. These are four hyperparameters that affect computational performance, with values ​​of (5, 5, 0.5, 1). The regression calculation for the rotated frame is represented as: in: in, and These represent the actual angle offset and the predicted angle offset, respectively. It is input, that is = CIoU represents the crossover union ratio, which is calculated using the following formula: CIoU in Represents a rectangular frame and its truth box The intersection-union ratio: Area refers to the area. and Let them represent the union and the package, respectively. This represents the calculation of the distance between two points. and Represent the center point of rectangle R and its truth box, respectively. The center point; Indicates that it contains R and The length of the diagonal of the smallest enclosed region; It is used to measure the impact of the target's aspect ratio and is calculated as follows: w and h represent the width and height, respectively. It is a balancing parameter used to bridge the relationship between IoU and aspect ratio, and is calculated as follows: In loss calculation, SkewCIoU is the proposed skew cross union ratio, which is defined as follows: Where SkewIoU represents the oblique intersection-to-union ratio, which is calculated as: Where I represents the pair and The region obtained by triangulation of the intersection of the two regions.

2. The method according to claim 1, characterized in that, In the Backbone stage of the DSA-Net, the added Context Location Module (CLM) enables the network to extract ship target feature information contained in the port image to be detected, forming a multi-scale feature map, including: In the Backbone stage of the DSA-Net, a port image of a preset size is input; By using the pooling operation of a pre-defined convolutional neural network and the context location module (CLM) based on the context location relationship, the network obtains the multi-scale feature map using the spatial location information of the port image to be detected of the pre-defined size.

3. The method according to claim 1, characterized in that, In the Neck stage of the DSA-Net, the addition of the Global Channel Module (GCM) enables the network to fuse the channel information of the multi-scale feature maps to obtain the final feature map, including: In the Neck stage of the DSA-Net, interpolation and the added Global Channel Module (GCM) are used to enable the network to fuse the channel information of the multi-scale feature maps to obtain the final feature map.

4. A device for detecting the rotation of ship targets in optical remote sensing images using the method described in claim 1, characterized in that, include: The data acquisition module is used to acquire image data to be detected; wherein, the image data to be detected includes a port image to be detected and corresponding label information of the port image to be detected; The weight acquisition module is used to input the image data to be detected into a preset network model to obtain the feature map of the port image to be detected, and to calculate the optimal detection weight of the network model based on the feature map of the port image to be detected and the corresponding label information of the port image to be detected. The image detection module is used to obtain the ship target detection result of the image data to be detected based on the optimal detection weight.

5. The apparatus according to claim 4, characterized in that, The network model is a dual-path segmentation attention network DSA-Net, and the weight acquisition module includes: In the backbone stage of DSA-Net, the added Context Location Module (CLM) enables the network to utilize the spatial location information of the port image to be detected to form a multi-scale feature map; and, In the Neck stage of DSA-Net, the addition of the Global Channel Module (GCM) enables the network to fuse the channel information of the multi-scale feature maps to obtain the final feature map.

6. The apparatus according to claim 5, characterized in that, The weight acquisition module is also used for, In the Backbone stage of DSA-Net, a port image of a preset size is input; By using the pooling operation of a pre-defined convolutional neural network and the context location module (CLM) based on the context location relationship, the network obtains the multi-scale feature map using the spatial location information of the port image to be detected of the pre-defined size.

7. The apparatus according to claim 6, characterized in that, The weight acquisition module is also used for, In the Neck stage of DSA-Net, interpolation and the added Global Channel Module (GCM) are used to enable the network to fuse the channel information of the multi-scale feature maps to obtain the final feature map.

8. The apparatus according to claim 6, characterized in that, The weight acquisition module is also used for: In the Head stage of the DSA-Net, a skew-crossover loop is added and compared with SkewCIoU; The SkewCIoU loss function is used to calculate the loss between the feature information of the extracted final feature map and the corresponding label information of the port image to be detected; The difference result is obtained based on the loss calculation, and the optimal detection weight of the network model is obtained based on the difference result.

Images