A remote sensing image rotating target detection method based on key feature learning, medium and equipment

Abstract

This invention discloses a method for detecting rotating targets in remote sensing images based on key feature learning, comprising the following steps: S1, screening the input remote sensing image to obtain high-quality samples, and regressing the position of the rotating target based on the high-quality samples; S2, extracting key features from the high-quality samples to obtain classification features; S3, optimizing the loss of the classification features to obtain optimized classification features; S4, calculating the category of the rotating target based on the optimized classification features; S5, obtaining the detection result of the rotating target based on the position and category of the rotating target. This invention can allocate sufficient positive samples for objects with a high degree of diversity, and through the design of the regression loss function, it can change the form of the regression loss function based on the proposed statistics, allowing more attention to high-quality samples during training; moreover, it can extract key features for the classification task based on the refined sampling position of the accurate location.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, medium, and device for detecting rotating targets in remote sensing images based on key feature learning. Background Technology

[0002] Remote sensing image target detection aims to locate targets of interest in remote sensing images and identify their categories using rotated bounding boxes. Currently, remote sensing image target detection is widely used in various fields such as military reconnaissance, urban planning, disaster early warning, and resource exploration. Deep learning-based single-stage remote sensing image target detection algorithms, with their simple and efficient network design and excellent performance, have become the mainstream method in the field of rotated target detection. The main steps of the aforementioned algorithm are as follows:

[0003] First, the initial image is processed by a feature extraction network to extract multi-scale features, and the sample generation algorithm generates dense initial samples based on the multi-scale features.

[0004] Secondly, multi-scale features generate offset predictions from samples to the ground truth through convolution operations, and the refinement algorithm corrects the initial samples based on the predicted offsets.

[0005] Next, based on the location information of the refined samples, the feature extraction network extracts the features of the region of interest, and predicts the offset of the refined samples from the ground truth and the corresponding category through subsequent convolution operations.

[0006] Finally, the sample allocation algorithm selects a certain number of positive and negative samples to calculate the loss, so that the network learns the offset features and category features from the sample space to the detection ground truth, and thus identifies the target in the image.

[0007] Although the above algorithm has achieved good results in rotating target detection, it is difficult to achieve high-performance target detection. A key reason for this is that when detecting remote sensing images of objects with irregular shapes or arbitrary orientations, key features are easily mismatched. This problem mainly stems from the following two aspects:

[0008] On the one hand, sample allocation algorithms need to provide the network with sufficient positive samples to regress accurate detection boxes. However, since the intersection-union ratio (IU) of irregularly shaped or arbitrarily oriented objects with samples is usually very small, existing sample allocation strategies based on fixed thresholds can hardly guarantee that such objects receive sufficient positive samples, thus affecting the accuracy of location regression.

[0009] On the other hand, accurate location regression does not necessarily mean accurate object detection. The network needs to capture key features for classifying objects to achieve accurate detection. However, irregularly shaped or arbitrarily oriented objects prevent the network from effectively extracting discriminative key features, and inaccurate classification also affects detection performance.

[0010] Therefore, high-performance detection can only be achieved by accurately locating objects and capturing samples that capture the key features required for object recognition. To achieve this, the sample allocation strategy should employ dynamic metrics for objects with significant variations, rather than fixed metrics. Recently, many excellent methods have adopted dynamic sample selection strategies. Some sample selection methods construct dynamic metrics based on IoU. Although these methods are more efficient than fixed allocation strategies, they suffer from parameter sensitivity. To address this issue, some sample selection methods model the samples and ground truth as Gaussian distributions and use the Gaussian-Wasestein distance (GWD) to measure their differences. GWD-based methods can describe the scale and angular features of arbitrarily oriented objects, but the form of GWD is complex and cannot measure the distance between square targets. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the prior art and provide a remote sensing image rotation target detection method, medium and device based on critical feature learning. This invention captures the key features of objects with great diversity through the critical feature learning (CFL) method, and can refine the critical feature positions by selecting high-quality samples with fewer prior settings and using accurate positions.

[0012] The first objective of this invention is to provide a method for detecting rotating targets in remote sensing images based on key feature learning, comprising the following steps:

[0013] S1. Filter the input remote sensing images to obtain high-quality samples, and use the high-quality samples to regress the position of the rotating target;

[0014] S2. Extract the key features from the high-quality samples obtained in step S1 to obtain classification features;

[0015] S3. Perform loss optimization on the classification features obtained in step S2 to obtain optimized classification features;

[0016] S4. Based on the optimized classification features obtained in step S3, the category of the rotating target is calculated using a convolutional neural network;

[0017] S5. Based on the position of the rotating target obtained in step S1 and the category of the rotating target obtained in step S4, obtain the detection result of the rotating target.

[0018] Preferably, step S1 includes the following steps:

[0019] S11. Extract feature maps of different sizes from the input image through multi-layer convolution, and fuse the feature maps of different sizes with the feature pyramid network to obtain multi-scale feature maps.

[0020] S12. Multiple anchor points are laid on the multi-scale feature map obtained in step S11. Multiple anchor points are used as the center points of the boxes, and multiple horizontal anchor boxes with a length and width of 4 pixels are set as training samples.

[0021] S13. Obtain the detection ground truth g and anchor frame a position information from the training samples obtained in step S12, and obtain the initial IoU threshold based on the detection ground truth g and anchor frame a position information. Then based on the initial IoU threshold Shape adjustment factor and angle adjustment factor To obtain the final IoU threshold ;

[0022] S14. The final IoU threshold obtained in step S13. The training samples obtained in step S12 are screened to obtain high-quality samples, and the position of the rotating target is obtained by regression based on the high-quality samples.

[0023] Preferably, in step S13, the initial IoU threshold is obtained by the following method:

[0024] S131. Obtain the position information of the ground truth value g and the anchor frame a from the training samples obtained in step S12. The position information of the ground truth value g and the anchor frame a are denoted as follows:

[0025]

[0026] Among them, B g and B a The positions of the ground truth value g and the anchor frame a are respectively, x g and y g Let w represent the coordinates of the center point of the true detection value g, respectively. g and h g Let θ represent the width and height of the true value g, respectively. g The angle representing the true value of the detection; x a and y a Let w represent the coordinates of the center point of anchor frame a, respectively. a and h a Let represent the width and height of anchor frame 'a', respectively. The angle of anchor frame a;

[0027] S132. Based on the position information of the detection ground value g and the anchor frame a obtained in step S131, for each detection ground value g on the multi-scale feature map obtained in step S11, the Euclidean distance between the center point of the detection ground value g and the center point of the anchor frame a is calculated as follows:

[0028]

[0029] Where i and j represent the i-th ground truth detection and the j-th anchor box, respectively, x a and y a Let x and x represent the coordinates of the center point of anchor frame a, respectively. g and y g These represent the coordinates of the center point of the true value g;

[0030] S133. Based on the Euclidean distance calculated in step S132, select the K anchor boxes that are closest to the Euclidean distance of the true detection value g.

[0031] S134. Calculate the intersection-union ratio (IUU) of the true detection value g with the K anchor frames selected in step S133. The formula for calculating the IUU is as follows:

[0032]

[0033] Among them, B g and B a These are the position information of the detection ground value g and the anchor frame a, respectively;

[0034] S135. The mean value m of the cross-union ratio obtained in step S134. g And variance υ g Calculations were performed to obtain the mean m. g And variance υ g and the mean m g And variance υ g The sum is denoted as the initial IoU threshold. Mean m g Variance υ g and initial IoU threshold The calculation formula is:

[0035]

[0036] Among them, I i,j For intersection and union ratio, m g υ is the mean of the crossover ratio. g The variance of the intersection-union ratio, is the initial IoU threshold, and N is the number of candidate samples.

[0037] Preferably, in step S13, the final IoU threshold Obtained through the following methods:

[0038] S1301, The difference between the true value g and the aspect ratio of the K anchor frames selected in step S133. and the difference in angle Calculations were performed to obtain the difference in aspect ratio. and the difference in angle Difference in aspect ratio and the difference in angle The calculation formula is as follows:

[0039]

[0040]

[0041] in, The aspect ratio of the anchor frame. To detect the aspect ratio of the true value, For the angle of the anchor frame, Angle for detecting the true value;

[0042] S1302, Based on the difference in aspect ratio obtained in step S1301 and the difference in angle The shape adjustment factor was calculated. and angle adjustment factor ， Shape adjustment factor and angle adjustment factor The calculation formula is as follows:

[0043]

[0044]

[0045] Where ε is a hyperparameter, Due to the difference in aspect ratio, Due to differences in angle;

[0046] S1303, Based on the shape adjustment factor obtained in step S1302 and angle adjustment factor and the initial IoU threshold obtained in step S135 The final IoU threshold is calculated. The final IoU threshold The calculation formula is as follows:

[0047]

[0048] in, ε is a hyperparameter. and These are the shape adjustment factor and the angle adjustment factor, respectively. This is the initial IoU threshold.

[0049] Preferably, step S14 includes the following steps:

[0050] S141. Filter the samples whose center point falls in the anchor box a of the true value g in the training samples obtained in step S12 to obtain dynamic samples.

[0051] S142. Select dynamic samples obtained in step S141 whose IoU with the true detection value g is greater than... The anchor frame is used as a high-quality sample, and the position of the rotating target is obtained by regression based on the high-quality sample.

[0052] Preferably, step S2 includes the following steps:

[0053] S21. Obtain the location information of the target of interest from the high-quality sample obtained in step S1. p ;

[0054] S22. Based on the location information of the target of interest obtained in step S21, B p The initial sampling positions of key features are obtained. ;

[0055] S23. Obtain the initial sampling position P relative to the key sampling position in step S22. init The offset is calculated to obtain the sampling position offset S;

[0056] S24. Based on the initial sampling position P of the key features obtained in step S22 init The sampling position P of the key feature is calculated using the sampling position offset S obtained in step S23. ref Key feature sampling location P ref The calculation formula is as follows:

[0057]

[0058] Where, x init and y init These are the x-axis and y-axis coordinates of the initial sampling position, respectively. and The components of the sampling position offset S on the x and y axes are w and h, respectively, which are the width and height of the target of interest.

[0059] S25. Calculate the transformation matrix T from the image coordinate system to the convolution coordinate system. The transformation matrix T is expressed as:

[0060]

[0061] Where w, h and θ are the changes in width, height and angle from the image coordinate system to the convolution coordinate system, respectively, and x and y are the changes in the center point coordinates from the image coordinate system to the convolution coordinate system.

[0062] S26. Using the transformation matrix T obtained in step S25, the key feature sampling positions P obtained in step S24 are... ref Sampling points P converted to deformable convolution dcn ;

[0063] S27. Based on sampling point P dcn Extract the key features from the multi-scale feature map obtained in step S11, and concatenate them pixel by pixel with the initial features to obtain the final classification features.

[0064] Preferably, step S3 includes the following steps:

[0065] S31. Calculate the total loss for multiple tasks. The calculation formula is as follows:

[0066]

[0067] in, The total loss for multiple tasks, hyperparameter and Control tradeoffs are set, with a default value of 1, where N represents the number of anchors. It is a binary value ( Used as a background (Used for the foreground, but not for the background, no regression is observed). Labels indicating the target. It is the probability distribution of various categories calculated by the sigmoid function. Labels indicating the true value of the detection. The label representing the prediction, L reg For regression loss, L cls For classification loss;

[0068] S32. Use the total loss of the multi-task obtained in step S31 to optimize the classification features obtained in step S2, and obtain optimized classification features.

[0069] Preferably, step S4 includes the following steps:

[0070] S41. Based on the optimized classification features obtained in step S3, generate a feature map with K×A channels through multi-layer convolution operations, where K is the number of categories of the target of interest and A is the number of selected samples.

[0071] S42. Select the channel with the largest value in the feature map obtained in step S41, which is the category corresponding to the rotating target.

[0072] A second objective of this invention is to provide a computer-readable storage medium comprising a stored computer program; wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the steps of the aforementioned remote sensing image rotation target detection method based on key feature learning.

[0073] A third objective of this invention is to provide a terminal device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the steps of the aforementioned remote sensing image rotation target detection method based on key feature learning.

[0074] Compared with the prior art, the advantages of this invention are as follows:

[0075] This invention constructs a dynamic metric that can allocate sufficient positive samples to objects with high diversity. Furthermore, through the design of SC-Loss, it can modify the form of the regression loss function based on the proposed statistics, allowing for greater focus on high-quality samples during training. Moreover, it can extract key features for the classification task by refining the sampling location based on accurate location. The proposed method is validated using three challenging remote sensing datasets, and the validation results show that the detection method of this invention outperforms existing state-of-the-art methods. Attached Figure Description

[0076] Figure 1 A flowchart of the remote sensing image rotation target detection method based on key feature learning provided by the present invention;

[0077] Figure 2 This is a flowchart of the spatial transformation selection strategy in this invention;

[0078] Figure 3 This is a flowchart of the key feature sampling strategy in this invention;

[0079] Figure 4 This is an overview diagram of the key feature sampling strategy in this invention;

[0080] Figure 5 The scaling variation between predicted bounding boxes and ground truth detections at different training stages;

[0081] Figure 6 A visualization of the input image (first column), shared features (second column), and classification features generated by CFS (third column);

[0082] Figure 7 Visualization results of the detection method and benchmark method provided in this invention on the DOTA dataset and HRSC2016 dataset;

[0083] Different colored bounding boxes represent different categories of targets;

[0084] Figure 8 The detection results of the detection method and benchmark method provided in this invention on the FAIR1M-1.0 dataset are visualized. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] Example 1

[0087] like Figures 1-3 As shown, this invention provides a remote sensing image rotation target detection method based on key feature learning, consisting of a backbone network, a feature pyramid network (FPN), an initial detection stage with CFS, and a fine detection stage. The initial detection stage generates high-quality rotated anchor boxes and extracts aligned features for the fine detection stage. The fine detection stage extracts orientation-invariant features and performs classification and localization to produce the final detection. The proposed STS strategy is implemented in the initial detection stage, selecting high-quality samples for objects with irregular shapes and complex rotations. To achieve accurate classification, the proposed CFS module uses the detection results from the initial detection stage to refine the classification features of the sampled locations. SC-Loss, designed for anchor-based bounding box regression, introduces statistical information from the predicted boxes to change the form of the regression loss function. The proposed SC-Loss makes the training process more focused on high-quality samples.

[0088] This invention provides a method for detecting rotating targets in remote sensing images based on key feature learning, specifically including the following steps:

[0089] S1. The input remote sensing images are filtered to obtain high-quality samples. The position of the rotating target is obtained by regression based on the high-quality samples. This step is the Spatial Transform Selection (STS) strategy, which includes the following steps:

[0090] S11. Extract feature maps of different sizes from the input image through multi-layer convolution, and fuse the feature maps of different sizes with the feature pyramid network to obtain multi-scale feature maps.

[0091] S12. On the multi-scale feature map obtained in step S11, anchor points are laid with step sizes of 8, 16, 32, 64 and 128 pixels respectively. Multiple anchor points are used as the center points of the boxes. Horizontal anchor boxes with a length and width of 4 pixels are set as training samples.

[0092] S13. Obtain the detection ground truth g and anchor frame a position information from the training samples obtained in step S12, and obtain the initial IoU threshold based on the detection ground truth g and anchor frame a position information. Then based on the initial IoU threshold Shape adjustment factor and angle adjustment factor To obtain the final IoU threshold ;

[0093] Specifically, the initial IoU threshold Obtained through the following methods:

[0094] S131. Obtain the position information of the ground truth value g and the anchor frame a from the training samples obtained in step S12. The position information of the ground truth value g and the anchor frame a are denoted as follows:

[0095]

[0096] Among them, B g and B a The positions of the ground truth value g and the anchor frame a are respectively, x g and y g Let w represent the coordinates of the center point of the true detection value g, respectively. g and h g Let θ represent the width and height of the true value g, respectively. g The angle representing the true value of the detection; x a and y a Let w represent the coordinates of the center point of anchor frame a, respectively. a and h a Let represent the width and height of anchor frame 'a', respectively. The angle of the anchor frame;

[0097] S132. Based on the position information of the detection ground value g and the anchor frame a obtained in step S131, for each detection ground value g on the multi-scale feature map obtained in step S11, the Euclidean distance between the center point of the detection ground value g and the center point of the anchor frame a is calculated as follows:

[0098]

[0099] Where i and j represent the i-th ground truth detection and the j-th anchor box, respectively, x a and y a Let x and x represent the coordinates of the center point of anchor frame a, respectively.g and y g These represent the coordinates of the center point of the true value g;

[0100] S133. Based on the Euclidean distance calculated in step S132, select the K anchor boxes that are closest to the Euclidean distance of the true detection value g.

[0101] S134. Calculate the intersection-union ratio (IUU) of the true detection value g with the K anchor frames selected in step S133. The formula for calculating the IUU is as follows:

[0102]

[0103] Among them, B g and B a These are the position information of the detection ground value g and the anchor frame a, respectively;

[0104] S135. The mean value m of the cross-union ratio obtained in step S134. g And variance υ g Calculations were performed to obtain the mean m. g And variance υ g and the mean m g And variance υ g The sum is denoted as the initial IoU threshold. Mean m g Variance υ g and initial IoU threshold The calculation formula is:

[0105]

[0106] Among them, I i,j For intersection and union ratio, m g υ is the mean of the crossover ratio. g The variance of the intersection-union ratio, is the initial IoU threshold, and N is the number of candidate samples.

[0107] The final IoU threshold Obtained through the following methods:

[0108] S1301, The difference between the true value g and the aspect ratio of the K anchor frames selected in step S133. and the difference in angle Calculations were performed to obtain the difference in aspect ratio. and the difference in angle Difference in aspect ratio and the difference in angle The calculation formula is as follows:

[0109]

[0110]

[0111] in, The aspect ratio of the anchor frame. To detect the aspect ratio of the true value, For the angle of the anchor frame, Angle for detecting the true value;

[0112] S1302, Based on the difference in aspect ratio obtained in step S1301 and the difference in angle The shape adjustment factor was calculated. and angle adjustment factor Shape adjustment factor and angle adjustment factor The calculation formula is as follows:

[0113]

[0114]

[0115] Where ε is a hyperparameter used to adjust the aspect ratio adjustment factor according to the distribution of the dataset. Due to the difference in aspect ratio, Due to differences in angle;

[0116] S1303, Based on the shape adjustment factor obtained in step S1302 and angle adjustment factor and the initial IoU threshold obtained in step S135 The final IoU threshold is calculated. The final IoU threshold The calculation formula is as follows:

[0117]

[0118] in, It is used to adaptively adjust according to the distribution of instances in the dataset, where ε is a hyperparameter. and These are the shape adjustment factor and the angle adjustment factor, respectively. This is the initial IoU threshold.

[0119] S14. The final IoU threshold obtained in step S13. The training samples obtained in step S12 are screened to obtain high-quality samples. The position of the rotating target is obtained by regression based on the high-quality samples. The specific steps include:

[0120] S141. Filter the samples whose center point falls in the anchor box a of the true value g in the training samples obtained in step S12 to obtain dynamic samples.

[0121] S142. Select dynamic samples obtained in step S141 whose IoU with the true detection value g is greater than... The anchor frame is used as a high-quality sample, and the position of the rotating target is obtained by regression based on the high-quality sample.

[0122] S2. Extract the key features from the high-quality samples obtained in step S1 to obtain classification features. This step is the Key Feature Sampling (CFS) strategy. Figure 4 As shown, the process includes fine-grained sampling location generation and key feature sampling. For a given refined anchor point (blue solid line), this invention reduces its boundary by 0.3 to obtain a reduced anchor point (blue dashed line), and uses the center point, vertex, and midpoint as initial sampling locations (green dots). The fine-grained sampling locations (yellow dots) can be obtained from the initial feature sampling locations and the learnable offset field (yellow arrows). For a given initial prediction, this invention first refines the sampling locations for classification based on the initial prediction box and the learnable offset S. Secondly, the key feature sampling process is applied, calculating the offset S at each location in the DCN based on the fine-grained sampling locations. Finally, the offset S is fed into the DCN to extract discriminative features for classification, specifically including the following steps:

[0123] S21. Obtain the location information of the target of interest from the high-quality sample obtained in step S1. p B p =(x,y,w,h,θ); where x and y represent the coordinates of the center point of the prediction box, w and h represent the width and height of the prediction box, respectively, and θ represents the angle of the ground truth detection g;

[0124] S22. Based on the location information of the target of interest obtained in step S21, B p The initial sampling positions of key features are obtained. Specifically, this refers to: location information B of the target of interest. p The midpoint remains unchanged, while the lengths of the four sides are reduced to 0.3 of their original length. The midpoint, four vertices, and the midpoint of each side—a total of nine points—are taken as the initial sampling locations for the key features. ;

[0125]

[0126] Where, x init and y init These are the x-axis and y-axis coordinates of the initial sampling position, respectively.

[0127] S23. Obtain the initial sampling position relative to the key sampling position in step S22. The offset is calculated to obtain the sampling position offset S, specifically: in Figure 1 A branch is added to the regression network of the detection head to calculate the offset of the key sampling position relative to the initial sampling position. The structure of the regression branch is as follows:

[0128]

[0129] Where δ is the activation function, conv r and conv s These are two branches of conv0, used to obtain the positional features R and the offset features S. The number of channels representing positional features and offset features, respectively;

[0130] S24. Based on the initial sampling position P of the key features obtained in step S22 init The sampling position P is obtained by combining the sampling position offset S obtained in step S23. ref Key feature sampling location P ref The calculation formula is as follows:

[0131]

[0132] Where, x init and y init These are the x-axis and y-axis coordinates of the initial sampling position, respectively. and The components of the sampling position offset S on the x and y axes are w and h, respectively, which are the width and height of the target of interest.

[0133] S25. Calculate the transformation matrix T from the image coordinate system to the convolution coordinate system. The transformation matrix T is expressed as:

[0134]

[0135] Where w, h and θ are the changes in width, height and angle from the image coordinate system to the convolution coordinate system, respectively, and x and y are the changes in the center point coordinates from the image coordinate system to the convolution coordinate system.

[0136] S26. Using the transformation matrix T obtained in step S25, the key feature sampling positions P obtained in step S24 are... ref Sampling points P converted to deformable convolution dcn ;

[0137] S27. Based on sampling point P dcn Extract the key features from the multi-scale feature map obtained in step S11, and concatenate them pixel by pixel with the initial features to obtain the final classification features.

[0138] S3. Perform loss optimization on the classification features obtained in step S2 to obtain optimized classification features, including the following steps:

[0139] S31. Calculate the total loss for multiple tasks. The calculation formula is as follows:

[0140]

[0141] in, The total loss for multiple tasks, hyperparameter and Control tradeoffs are set, with a default value of 1, where N represents the number of anchors. It is a binary value ( Used as a background (Used for the foreground, but not for the background, no regression is observed). Labels indicating the target. It is the probability distribution of various categories calculated by the sigmoid function. Labels indicating the true value of the detection. The label representing the prediction, L reg For regression loss, L cls For classification loss;

[0142] S32. Use the total loss of the multi-task obtained in step S31 to optimize the classification features obtained in step S2, and obtain optimized classification features.

[0143] Figure 5 This is a scale transformation map between predicted bounding boxes and ground truth detections at different training stages, achieved through... Figure 5 This illustrates the changes in the scale during the training process. This invention uses S... pre / S gt To describe scale changes during the training process. Figure 5 It can be seen that the scale variation between the predicted value and the ground truth value is reduced during the training process, which means that the quality of the samples is improving.

[0144] S4. Based on the optimized classification features obtained in step S3, the category of the rotating target is calculated using a convolutional neural network, specifically including the following steps:

[0145] S41. Based on the optimized classification features obtained in step S3, generate a feature map with K×A channels through multi-layer convolution operations, where K is the number of categories of the target of interest and A is the number of selected samples.

[0146] S42. Select the channel with the largest value in the feature map obtained in step S41, which is the category corresponding to the rotating target;

[0147] S5. Based on the position of the rotating target obtained in step S1 and the category of the rotating target obtained in step S4, obtain the detection result of the rotating target.

[0148] Example 2

[0149] This invention provides a computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the steps of the remote sensing image rotation target detection method based on key feature learning proposed in Embodiment 1.

[0150] Example 3

[0151] This invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the remote sensing image rotation target detection method based on key feature learning proposed in Embodiment 1.

[0152] The following is a verification of the remote sensing image rotation target detection method based on key feature learning provided in Embodiment 1 of the present invention.

[0153] This invention was tested on three well-known public aerial imagery benchmarks: DOTA, FAIR1M-1.0, and HRSC2016. The rotating target detection method proposed in Embodiment 1 of this invention was implemented using MMdetection on an NVIDIA GeForce RTX 3090 server with 24GB of memory, as detailed below:

[0154] 1. Dataset

[0155] DOTA is a challenging remote sensing image orientation target detection dataset containing 2,806 satellite optical images ranging in size from 800×800 to 4,000×4,000 pixels, with a total of 188,282 instances. These instances are annotated into 15 common target categories.

[0156] The target categories include airplanes, baseball fields, bridges, ships, harbors, roundabouts, water tanks, tennis courts, small vehicles, large vehicles, basketball courts, athletic fields, soccer fields, swimming pools, and helicopters. Half of the entire dataset is randomly selected as the training set, one-sixth as the validation set, and one-third as the test set. Due to the large image size in DOTA, this invention divides the images in the training and validation sets into 1024×1024 image blocks with a stride of 200 and inputs them into the detection network. The detection results are then used to verify the performance using officially provided tools.

[0157] FAIR1M-v1.0 is a recently released large-scale remote sensing dataset. It consists of 15,266 images from different platforms, with resolutions ranging from 300 to 800, and contains over 1 million instances for fine-grained object recognition. In the FAIR1M1.0 dataset, all instances are labeled with directed bounding boxes, and there are 5 classes and 37 subclasses.

[0158] HRSC2016 is a high-resolution, fine-grained ship detection dataset. The dataset contains 1061 images from six well-known ports, with resolutions ranging from 300 to 1500 to 900 pixels. All instances in the images are labeled with directed bounding boxes. All images are resized to 800×512 and divided into three parts: 436 images for training, 181 images for validation, and 444 images for testing.

[0159] 2. Implementation details

[0160] S 2 A-Net_C uses normal convolutions instead of aligned convolutions, which is S 2 One of the baseline structures of A-Net. This invention uses S... 2 A-Net_C is used as the baseline to implement the method proposed in Embodiment 1 of this invention, which makes it easier to verify the performance of the method proposed in Embodiment 1 of this invention. Unless otherwise specified, this invention defaults to using ResNet-101 with FPN as the backbone. For each scale feature, there is only one rectangular anchor box at the corresponding location. This invention uses standard image processing methods in the detection pipeline, such as resizing, random flipping, normalization, and random rotation. All models are trained on 4 GPUs with a batch size of 8 by the SGD optimizer, where the initial learning rate, momentum, and weight decay are set to 0.01, 0.9, and 0.0001, respectively. The models are trained on the DOTA, FAIR1M-1.0, and HRSC2016 datasets for 12, 12, and 36 epochs, respectively.

[0161] 3. Ablation test

[0162] The present invention conducted a series of comparative experiments on DOTA and HRSC2016 to verify the effectiveness of the method proposed in Embodiment 1 of the present invention.

[0163] (1) Hyperparameters of Spatial Transformation Sample Assignment Strategy (STS)

[0164] To find suitable STS hyperparameter settings, this invention conducted parameter sensitivity experiments on the DOTA dataset. The experimental results are shown in Table 1 below. Table 1 demonstrates that on the DOTA dataset, STS performance is optimal when the weighting parameter γ is set to 5. Considering the distribution differences between the aforementioned datasets, since the FAIR1M-1.0 dataset has a similar distribution to the DOTA dataset, the weighting parameter γ for the STS on FAIR1M-1.0 is also set to 5. For the HRSC2016 dataset, which has a more exaggerated aspect ratio, the weighting parameter γ is set to 6.

[0165] Table 1. Experimental results of parameter sensitivity

[0166]

[0167] (2) The role of STS

[0168] The performance of the method in Embodiment 1 of this invention was compared for each structure in the DOTA dataset. The results are shown in Table 2 below. As can be seen from the results in Table 2, after implementing STS, the mAP50 and mA75 of the method proposed in Embodiment 1 of this invention are 73.06% and 37.23%, respectively, representing improvements of 1.17% and 4.71%. In particular, the method in Embodiment 1 of this invention achieves SH, BR, LV, and HA of 79.38%, 52.99%, 75.03%, and 66.92% for the DOTA dataset, respectively. These categories in the DOTA dataset exhibit significant differences in scale or have large aspect ratios.

[0169] Table 2 shows the performance comparison of the method of Embodiment 1 of the present invention for each structure in the DOTA dataset.

[0170]

[0171] The performance of the method proposed in Embodiment 1 of the present invention was compared on the HRSC2016 dataset under different standards. The results are shown in Table 3 below. As can be seen from the results in Table 3, the AP50 and AP75 of the method proposed in Embodiment 1 of the present invention on HRSC2016 are 89.66% and 66.78%, respectively, which are improvements of 4.06% and 25.92%.

[0172] The experimental results in Tables 2 and 3 show that the detection performance of the method proposed in Example 1 of this invention is significantly improved on different datasets.

[0173] Table 3. Performance comparison of the method of the present invention on different standards on the HRSC2016 dataset.

[0174]

[0175] The performance of the method proposed in Embodiment 1 of this invention was compared with other advanced tag allocation methods, and the results are shown in Table 4 below. As can be seen from the results in Table 4, compared with other dynamic tag allocation methods (such as ATSS and SA-S), the tag allocation proposed in Embodiment 1 of this invention improves the mAP on DOTA to 73.06%. When t=1, the STS proposed in Embodiment 1 of this invention achieves 72.22% mAP on DOTA, which is also superior to other tag allocation methods. For irregularly shaped objects, such as bridges, large cars, and basketball courts, the performance of STS reaches 52.99%, 75.03%, and 84.25%, respectively, and the 3-mAP is also significantly improved to 70.76%.

[0176] Table 4. Performance comparison results of the method of the present invention with other advanced tag allocation methods.

[0177]

[0178] (3) The role of the number of anchors in STS

[0179] The number of anchors is an important metric for validating label assignment. To explore the impact of the number of anchors, this invention uses different benchmark methods (rotated RetinaNet and S...) on the HRSC2016 dataset. 2 Ablation experiments were conducted using A-Net_C. The results are shown in Table 5 below. Table 5 shows that using the MaxIoU-based rotated RetinaNet method with 9 anchors per location, the mAP reached 82.67%. When using the STS method, the accuracy decreased by 5.07%. Furthermore, when this invention sets one anchor per location for STS, the accuracy is significantly improved, reaching 88.59% mAP. These results indicate that excessive prior information (the number of anchors) is redundant for STS. The experimental results also show that carefully designed label assignments can achieve competitive performance with fewer anchors.

[0180] Table 5 shows the experiments using different anchor point settings with different benchmark methods on the HRSC2016 dataset.

[0181]

[0182] (4) The role of the key feature sampling strategy (CFS)

[0183] Table 2 compares the performance of the proposed CFS in DOTA. The results in Table 2 show that, with CFS implemented, the method proposed in Example 1 of this invention achieves 73.09% mAP50 and 36.32% mAP75 in DOTA, representing improvements of 1.20% and 3.80%, respectively. Even under the more stringent criteria in Table 2, the proposed method maintains good accuracy, with AP75 decreasing only from 90.01% to 79.52%. This result demonstrates that accurate localization can refine the detector, extract more accurate classification features, and improve detector performance. Figure 6 As shown, taking airplanes and ships as examples, the discriminative features are usually distributed in various parts of the object, such as the middle of an airplane or the stern and bow of a ship. The heat map generated by CFS can accurately classify sensitive key features.

[0184] (5) Impact of Scale Control Loss (SC-Loss)

[0185] This invention also verifies the impact of SC-Loss. The results of the method proposed in Example 1 of this invention on the DOTA dataset are shown in Table 2. As can be seen from the results in Table 2, the SC-Loss proposed in Example 1 of this invention achieves 72.31% mAP50, with improvements of 8.79%, 4.23%, 7.05%, and 2.57% for ships, bridges, large vehicles, and ports, respectively. As can be seen from the results in Table 3, the AP50, AP60, AP70, and AP75 of the SC-Loss strategy proposed in Example 1 of this invention increase by 1.70%, 0.76%, 1.82%, and 9.15%, respectively. The experimental results on HRSC2016 demonstrate that the SC-Loss proposed in Example 1 of this invention can improve the scale adaptability of the detector.

[0186] 4. Comparison with the most advanced detectors

[0187] (1) The result of DOTA

[0188] The method proposed in Embodiment 1 of this invention was compared with the state-of-the-art method in the DOTA dataset, and the results are shown in Table 6 below.

[0189] Table 6 compares the method proposed in Example 1 of this invention with the most advanced methods in the DOTA dataset.

[0190]

[0191] As shown in Table 6, the method proposed in Embodiment 1 of this invention achieves 75.35% mAP without any data augmentation (e.g., random rotation). Compared to the baseline, the method of this invention improves mAP by 1.23%. It is worth noting that the CFL method was proposed to address the challenges of objects with irregular shapes and complex rotations. The gray area in Table 6 shows the difference between the CFL method and the baseline S. 2 Performance improvements compared to A-Net. For objects with irregular shapes and complex rotations in the DOTA dataset, such as baseball fields (3.42%), bridges (5.45%), athletic fields (4.20%), large vehicles (4.18%), traffic roundabouts (4.14%), and helicopters (5.42%), the above data demonstrates that the method proposed in Embodiment 1 of this invention has significant performance improvements.

[0192] Data augmentation is a crucial way to improve object detection performance. With the increase in data, the proposed method using R-101-FPN as the backbone network achieves 77.30% mAP, which is 1.19% higher than the state-of-the-art two-stage detectors based on R-CNN. ReDet, using a rotationally equivariant ResNet (RER) backbone network to model rotation-invariant features, achieves 80.10% mAP under multi-scale training and testing methods. Meanwhile, Swin-Tiny is also a powerful feature extraction network. The method of this invention, using SwinTiny as the backbone network, achieves the best result (81.24% mAP), outperforming state-of-the-art two-stage detectors and exceeding ReDet's 1.14% mAP.

[0193] Faced with the challenging DOTA dataset, the method proposed in this invention achieves state-of-the-art results across seven categories (bolded in Table 6): boats, bridges, athletic fields, cars, football fields, water tanks, and helicopters. Specifically, this invention achieves these results by using only one anchor point at each feature point (without redundant scale and aspect ratio). Baseline methods (i.e., S...) 2 The qualitative detection results of A-Net and the method proposed in Embodiment 1 of this invention are as follows: Figure 7 As shown. (Through) Figure 7 As can be seen, compared with the criterion method, the method proposed in this invention can achieve more accurate detection and fewer false positives and false negatives, indicating that when faced with objects with a huge variety of objects, the method proposed in the embodiments of this invention produces a more accurate location.

[0194] (2) Results of FAIR1M-1.0

[0195] FAIR1M-11.0 is a challenging multi-class dataset. Instances in FAIR1M-1.0 are always surrounded by complex environments, such as blurry images, complex backgrounds, and cloud or fog occlusion, which can lead to false positives or false negatives.

[0196] The method proposed in Embodiment 1 of this invention was compared with six other models on the FAIR1M-v1.0 dataset, and the comparison results are shown in Table 7 below. As can be seen from the results in Table 7, the method proposed in Embodiment 1 of this invention achieves 46.84% mAP, which is the best performing among the methods mentioned above.

[0197] Table 7 compares the method proposed in Example 1 of this invention with the state-of-the-art methods on the FAIR1M-v1.0 dataset.

[0198]

[0199] This invention also includes Figure 8 Visual comparisons were performed, with the red dashed line area representing the primary region of interest. Compared to the baseline, the proposed method detected most targets in the scene, achieving fewer missed detections and false detections, demonstrating its stronger feature capture capability in complex environments.

[0200] (3) Results of HRSC2016

[0201] The HRSC2016 dataset contains many elongated and arbitrarily oriented ship objects, posing a significant challenge to detection. The method proposed in Example 1 of this invention was compared with other state-of-the-art methods on the PASCALVOC2007 and PASCALVOC2012 metrics. The results are shown in Table 8. Table 8 shows that the performance on the PASCALVOC2007 metric is lower than that on the PASCALVOC2012 metric. Using RetinaNet as a baseline, the method proposed in Example 1 of this invention achieved 89.32% AP, significantly improving the baseline performance by 6.43%. Meanwhile, using S… 2 Using A-Net as a baseline, the method proposed in Embodiment 1 of this invention is superior to other methods, achieving 90.48% AP (07) and 96.90% AP (12) respectively, which is 1.89% higher than the baseline.

[0202] Table 8 compares the method of Example 1 of the present invention with other state-of-the-art methods under the PASCALVOC2007 and PASCALVOC2012 indices.

[0203]

[0204] (4) Efficiency comparison

[0205] On the DOTA dataset, comparisons were also made of mAP (%), FPS (Task / s), FLOPS (G), and parameters (M). All experiments were based on R-50-FPN and used a single 3090 GPU with a batch size of 1 and an input image resolution of 1024×1024. Table 9 shows the detailed experimental results of different methods. (S...) 2 A-Net has 36.21 parameters, approximately 172.48 GFLOPs, and achieves 74.12% mAP and 15.3 FPS.

[0206] Table 9. Comparison of parameters for different methods

[0207]

[0208] This invention is in S 2 The CFL method proposed in this invention was implemented on A-Net, achieving 75.35% mAP and 14.2 FPS. Compared to the baseline, this invention incurs a slight increase in computational cost (36.88 #Params vs. 36.21 #Params). As shown in Table 9, the S... 2 A-Net can achieve better performance with a smaller computational cost.

[0209] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting rotating targets in remote sensing images based on key feature learning, characterized in that, Includes the following steps: S1. Filter the input remote sensing images to obtain high-quality samples, and use the high-quality samples to regress the position of the rotating target, including the following steps: S11. Extract feature maps of different sizes from the input image through multi-layer convolution, and fuse the feature maps of different sizes with the feature pyramid network to obtain multi-scale feature maps. S12. Multiple anchor points are laid on the multi-scale feature map obtained in step S11. Multiple anchor points are used as the center points of the boxes, and multiple horizontal anchor boxes with a length and width of 4 pixels are set as training samples. S13. Obtain the detection ground truth g and anchor frame a position information from the training samples obtained in step S12, and obtain the initial IoU threshold based on the detection ground truth g and anchor frame a position information. Then based on the initial IoU threshold Shape adjustment factor and angle adjustment factor To obtain the final IoU threshold ; The initial IoU threshold is obtained through the following method: S131. Obtain the position information of the ground truth value g and the anchor frame a from the training samples obtained in step S12. The position information of the ground truth value g and the anchor frame a are denoted as follows: ； Among them, B g and B a The positions of the ground truth value g and the anchor frame a are respectively, x g and y g Let w represent the coordinates of the center point of the true detection value g, respectively. g and h g Let θ represent the width and height of the true value g, respectively. g The angle representing the true value of the detection; x a and y a Let w represent the coordinates of the center point of anchor frame a, respectively. a and h a Let represent the width and height of anchor frame 'a', respectively. The angle of anchor frame a; S132. Based on the position information of the detection ground value g and the anchor frame a obtained in step S131, for each detection ground value g on the multi-scale feature map obtained in step S11, the Euclidean distance between the center point of the detection ground value g and the center point of the anchor frame a is calculated as follows: ； Where i and j represent the i-th ground truth detection and the j-th anchor box, respectively, x a and y a Let x and x represent the coordinates of the center point of anchor frame a, respectively. g and y g These represent the coordinates of the center point of the true value g; S133. Based on the Euclidean distance calculated in step S132, select the K anchor boxes that are closest to the Euclidean distance of the true detection value g. S134. Calculate the intersection-union ratio (IUU) of the true detection value g with the K anchor frames selected in step S133. The formula for calculating the IUU is as follows: ； Among them, B g and B a These are the position information of the detection ground value g and the anchor frame a, respectively; S135. The mean value m of the cross-union ratio obtained in step S134. g And variance υ g Calculations were performed to obtain the mean m. g And variance υ g and the mean m g And variance υ g The sum is denoted as the initial IoU threshold. Mean m g Variance υ g and initial IoU threshold The calculation formula is: ； Among them, I i,j For intersection and union ratio, m g υ is the mean of the crossover ratio. g The variance of the intersection-union ratio, The initial IoU threshold is N, where N is the number of candidate samples. The final IoU threshold Obtained through the following methods: S1301, The difference between the true value g and the aspect ratio of the K anchor frames selected in step S133. and the difference in angle Calculations were performed to obtain the difference in aspect ratio. and the difference in angle Difference in aspect ratio and the difference in angle The calculation formula is as follows: ； ； in, The aspect ratio of the anchor frame. To detect the aspect ratio of the true value, For the angle of the anchor frame, Angle for detecting the true value; S1302, Based on the difference in aspect ratio obtained in step S1301 and the difference in angle The shape adjustment factor was calculated. and angle adjustment factor ， Shape adjustment factor and angle adjustment factor The calculation formula is as follows: ； ； in, For hyperparameters, Due to the difference in aspect ratio, Due to differences in angle; S1303, Based on the shape adjustment factor obtained in step S1302 and angle adjustment factor and the initial IoU threshold obtained in step S135 The final IoU threshold is calculated. The final IoU threshold The calculation formula is as follows: ； in, ε is a hyperparameter. and These are the shape adjustment factor and the angle adjustment factor, respectively. This is the initial IoU threshold; S14. The final IoU threshold obtained in step S13. The training samples obtained in step S12 are filtered to obtain high-quality samples. The position of the rotating target is obtained by regression based on the high-quality samples, which includes the following steps: S141. Filter the samples whose center point falls in the anchor box a of the true value g in the training samples obtained in step S12 to obtain dynamic samples. S142. Select dynamic samples obtained in step S141 whose IoU with the true detection value g is greater than... The anchor frame is used as a high-quality sample, and the position of the rotating target is obtained by regression based on the high-quality sample; S2, the key features of the high-quality sample obtained in step S1 are extracted to obtain the classification features; S3. Perform loss optimization on the classification features obtained in step S2 to obtain optimized classification features; S4. Based on the optimized classification features obtained in step S3, the category of the rotating target is calculated using a convolutional neural network; S5. Based on the position of the rotating target obtained in step S1 and the category of the rotating target obtained in step S4, obtain the detection result of the rotating target.

2. The remote sensing image rotation target detection method based on key feature learning according to claim 1, characterized in that, Step S2 includes the following steps: S21. Obtain the location information of the target of interest from the high-quality sample obtained in step S1. p ; S22. Based on the location information of the target of interest obtained in step S21, B p The initial sampling position P of the key feature is obtained. init ; S23. Obtain the initial sampling position P relative to the key sampling position in step S22. init The offset is calculated to obtain the sampling position offset S; S24. Based on the initial sampling position P of the key features obtained in step S22 init The sampling position P of the key feature is calculated using the sampling position offset S obtained in step S23. ref Key feature sampling location P ref The calculation formula is as follows: ； Where, x init and y init These are the x-axis and y-axis coordinates of the initial sampling position, respectively. and The components of the sampling position offset S on the x and y axes are w and h, respectively, which are the width and height of the target of interest. S25. Calculate the transformation matrix T from the image coordinate system to the convolution coordinate system. The transformation matrix T is expressed as: ； Where w, h and θ are the changes in width, height and angle from the image coordinate system to the convolution coordinate system, respectively, and x and y are the changes in the center point coordinates from the image coordinate system to the convolution coordinate system. S26. Using the transformation matrix T obtained in step S25, the key feature sampling positions P obtained in step S24 are... ref Sampling points P converted to deformable convolution dcn ; S27. Based on sampling point P dcn Extract the key features from the multi-scale feature map obtained in step S11, and concatenate them pixel by pixel with the initial features to obtain the final classification features.

3. The remote sensing image rotation target detection method based on key feature learning according to claim 1, characterized in that, Step S3 includes the following steps: S31. Calculate the total loss for multiple tasks. The calculation formula is as follows: ； in, The total loss for multiple tasks, hyperparameter and Control tradeoffs are set, with a default value of 1, where N represents the number of anchors. It is a binary value. Labels representing objects, It is the probability distribution of various categories calculated by the sigmoid function. The target vector representing the actual ground condition. L represents the predicted offset vector. reg For regression loss, L cls For classification loss; S32. Use the total loss of the multi-task obtained in step S31 to optimize the classification features obtained in step S2, and obtain optimized classification features.

4. The remote sensing image rotation target detection method based on key feature learning according to claim 1, characterized in that, Step S4 includes the following steps: S41. Based on the optimized classification features obtained in step S3, generate a feature map with K×A channels through multi-layer convolution operations, where K is the number of categories of the target of interest and A is the number of selected samples. S42. Select the channel with the largest value in the feature map obtained in step S41, which is the category corresponding to the rotating target.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the steps of the remote sensing image rotation target detection method based on key feature learning as described in any one of claims 1-4.

6. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the steps of the remote sensing image rotation target detection method based on key feature learning as described in any one of claims 1-4.

Images