Fine-grained object detection method of fusion point-by-point frequency domain attention and image-level to instance-level supervision from coarse to fine progressive learning

Abstract

The application provides a fine-grained object detection method based on coarse-to-fine gradual learning by fusing point-by-point frequency domain attention and image-level to instance-level supervision, comprising the following steps: sending spatial features output by a feature pyramid backbone network into a PFA-DWT module, fusing the spatial features and frequency domain features point by point, and sending the fused features into a C2FIm-FDE module to enhance the discriminability of the features; sending the enhanced backbone network features into a directional RPN to generate directional candidate boxes, obtaining the pooling features of each candidate box through a rotation region of interest alignment operation, and sending the pooling features into a C2FIn detection head to predict the category and position offset of the candidate box; training the constructed fine-grained object detection model to obtain a trained fine-grained object detection model; and inputting an image to be detected into the trained fine-grained object detection model to obtain a final detection result after non-maximum suppression. Compared with a popular FGOD model, the application has superior detection precision and generalization ability.

Description

Technical Field

[0001] This invention relates to the technical field of fine-grained object detection, and more particularly to a fine-grained object detection method that integrates point-by-point frequency domain attention with image-level to instance-level supervision, employing a coarse-to-fine progressive learning approach. Background Technology

[0002] Traditional object detection models are typically used to detect coarse-grained targets, such as aircraft, ships, and vehicles. Fine-grained object detection (FGOD), on the other hand, requires the ability to detect subclasses of coarse-grained targets, such as C919, A321, and Boeing 737 aircraft. FGOD based on remote sensing images has broad application prospects in areas such as Earth observation, urban monitoring, and disaster control.

[0003] Fine-grained object detection builds upon directed object detection (OOD) by further identifying similar subclasses. First, we will briefly review the relevant work on directed object detection, and then introduce some classic works in the field of fine-grained object detection.

[0004] Current OOD models can be broadly categorized into three types: one-stage OOD models based on anchor frames, two-stage OOD models based on anchor frames, and OOD models without anchor frames.

[0005] The one-stage OOD model based on anchor boxes pre-defines multiple anchor points for each feature point and uses a fully convolutional network to simultaneously predict the class confidence score and position offset of all anchor boxes. To address the issue of mismatch between the pre-define anchor boxes and targets with extreme aspect ratios, S... 2 A-Net [J. Han, J. Ding, J. Li, and G.-S. Xia, “Align deep features for oriented object detection,” IEEE Transactions on Geoscience and RemoteSensing, vol. 60, pp. 1–11, 2022.] proposes an aligned convolutional module to generate high-quality anchor boxes. 3Det [X. Yang, J. Yan, Z. Feng, and T. He, “R3det: Refined single-stage detector with feature refinement for rotating object,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 4, 2021, pp.3163–3171.] proposed a feature refinement module to more accurately extract features from targets with extreme aspect ratios and to extract an improved SkewIoU loss to more accurately estimate the target's rotation angle. Other classic works include DAL [Q. Ming, Z. Zhou, L. Miao, H. Zhang, and L. Li, “Dynamic anchor learning for arbitrary-oriented object detection,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 3, 2021, pp. 2355–2363.] and HRDet [Y. Yao, G. Cheng, C. Lang, X. Yuan, X. Xie, and J. Han, “Hierarchical mask promotion and robust integrated regression for oriented object detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 12, pp. 13 071–13 084, 2024.]. These methods offer advantages such as end-to-end training and high inference efficiency, but they suffer some loss in detection accuracy.

[0006] The two-stage directed object detection model first uses a Region Proposal Network (RPN) to generate higher-quality candidate boxes based on predefined anchor boxes, and then predicts the class confidence score and position offset of the candidate boxes. The RoI transformer [J. Ding, N. Xue, Y. Long, G.-S. Xia, and Q. Lu, “Learning roi transformer for oriented object detection in aerial images,” in 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2844–2853.] first generates horizontal candidate boxes based on predefined anchor boxes, and then uses the proposed Region of Interest (RoI) learner to convert the horizontal candidate boxes into directed candidate boxes. Oriented R-CNN [X.Xie, G. Cheng, J. Wang, X. Yao, and J. Han, “Oriented r-cnn for object detection,” in Proceedings of the IEEE / CVF international conference on computer vision, 2021, pp. 3520–3529.] proposed a directed RPN, which can directly generate directed candidate boxes based on predefined anchor boxes, significantly improving the computational efficiency of the model while ensuring the quality of the candidate boxes. AOPG [G.Cheng, J. Wang, K. Li, X. Xie, C. Lang, Y. Yao, and J. Han, “Anchor freeoriented proposal generator for object detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–11, 2022.] first uses a sub-model without anchor boxes to generate high-quality anchor boxes, and then generates high-quality directed candidate boxes based on these.Other classic methods include Strip R-CNN [X. Yuan, Z. Zheng, Y. Li, X. Liu, L. Liu, X. Li, Q. Hou, and M.-M. Cheng, “Strip r-cnn: Large strip convolution for remote sensing object detection,” 2025. [Online]. Available: https: / / arxiv.org / abs / 2501.03775] and Info-FPN [S. Chen, J. Zhao, Y. Zhou, H. Wang, R. Yao, L. Zhang, and Y. Xue, “Info-fpn: An informative feature pyramid network for object detection in remote sensing images,” Expert Systems with Applications, vol.214, p. 119132, 2023. [Online]. Available: https: / / www.sciencedirect.com / science / article / pii / S0957417422021509]. These methods typically have high detection accuracy but slow inference speed.

[0007] Anchor-free directed object detection models do not use pre-set anchor boxes. Instead, they directly obtain the position and class of the directed box by predicting the distance from each feature point to the four sides of the directed box, the angle of the directed box, and the class confidence score. DRN [X. Pan, Y. Ren, K. Sheng, W. Dong, H. Yuan, X. Guo, C. Ma, and C. Xu, “Dynamicrefinement network for oriented and densely packed object detection,” in 2020IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11 204–11 213.] adaptively selects between traditional convolutional features and strip convolutional features to address the feature extraction problem of irregular targets. On the other hand, it introduces a dynamic convolutional kernel generator into the object detection head to extract target-level features. Oriented RepPoints [W. Li, Y. Chen, K. Hu, and J. Zhu, “Oriented reppoints for aerial object detection,” in 2022 IEEE / CVF Conferenceon Computer Vision and Pattern Recognition (CVPR), 2022, pp. 1819–1828.] represents a target as an adaptively adjustable set of points that can flexibly change according to the target's geometric features. By dynamically adjusting the point set, deformable convolutions are guided to achieve accurate feature extraction of irregular targets.Other classic methods include DFDet [X. Xie, G. Cheng, C. Rao, C. Lang, and J. Han, “Oriented object detection via contextual dependence mining and penalty-incentive allocation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–10, 2024] and R2IPoint [X. Yao, H. Shen, X. Feng, G. Cheng, and J. Han, “R²ipoints: Pursuing rotation-insensitive point representation for aerial object detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–12, 2022]. These methods have strong generalization ability and high accuracy in detecting small targets, but their detection accuracy is affected by the distribution of feature points.

[0008] Existing fine-grained object detection models mainly focus on improving the loss function of the object detection head and the feature attention module, which will be introduced separately below.

[0009] The feature attention module still primarily uses spatial or channel attention. SFRNet [G. Cheng, Q. Li, G. Wang, X. Xie, L. Min, and J. Han, “Sfrnet: Fine grained oriented object recognition via separate feature refinement,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp.1–10, 2023.] uses multi-head attention and local hashing algorithms to compute spatial and channel attention in the spatial and channel dimensions respectively, and then adds the two together to enhance the features. PETDet [W.Li, D. Zhao, B. Yuan, Y. Gao, and Z. Shi, “Petdet: Proposal enhancement for two-stage fine-grained object detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–14, 2024.] enhances the feature representation capability of RPN by concatenating channels and spatial attention, thereby generating higher-quality candidate boxes. On the other hand, it utilizes bilinear fusion of adjacent channels to more effectively utilize the spatial detail information of low-level features to distinguish similar subclasses. EIRNet [Y. Han, X. Yang, T. Pu, and Z. Peng, “Fine-grained recognition for oriented ship against complex scenes in optical remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–18, 2022.] designed a ship-oriented FGOD model. It first generates pixel-level ground truth maps of the ship's bow, stern, and overall region, and uses these to supervise the prediction of the corresponding spatial attention maps. The two maps are then added together to obtain a more accurate and complete spatial attention map, which enhances the discriminative power of the backbone network features.Other important related works include CMAM-FAM [W. Xiong, Z. Xiong, and Y. Cui, “An explainable attention network for fine-grained ship classification using remote-sensing images,”[IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–14, 2022.], P-CNN [J. Han, X. Yao, G. Cheng, X. Feng, and D. Xu, “P-cnn: Part-based convolutional neural networks for fine-grained visual categorization,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 2, pp. 579–590, 2022.], etc.

[0010] Existing FGOD models typically use loss functions that aim to maximize the difference between classes. DRNet [X. Xie, G. Cheng, W. Li, C. Lang, P. Zhang, Y. Yao, and J. Han, “Learning discriminative representation for fine-grained object detection in remote sensing images,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 8, pp. 8197–8208, 2025.] proposes a confusion-minimizing loss. It defines the separability of samples, treats samples with low separability as hard samples, and assigns them higher weights in the loss function. This approach minimizes the confusion of the FGOD model regarding similar subclasses. PCLDet [L. Ouyang, G.Guo, L. Fang, P. Ghamisi, and J. Yue, “Pcldet: Prototypical contrastive learning for fine-grained object detection in remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–11, 2023.] constructs corresponding prototype features for each subclass. A loss function is built by maximizing the similarity between the prototype features and features of samples from the same class and minimizing the similarity between the prototype features and features of samples from different classes. The prototype features are iteratively updated as training progresses. During the inference phase, the category of the test image is determined by comparing the similarity between the features of the test image and the prototype features of each class.Other classic works include BSNet [X. Li, J. Wu, Z. Sun, Z. Ma, J. Cao, and J.-H. Xue, "Bsnet: Bisimilarity network for few-shot fine-grained imageclassification," IEEE Transactions on Image Processing, vol. 30, pp. 1318–1331, 2021], ISCL [L. Zeng, H. Guo, W. Yang, H. Yu, L. Yu, P. Zhang, and T.Zou, “Instance switching-based contrastive learning for fine-grained airplane detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp.1–16, 2022.] et al.

[0011] Therefore, existing object detection models can be broadly categorized into three types: one-stage object detection models based on anchor boxes, two-stage object detection models based on anchor boxes, and object detection models without anchor boxes. Due to the significant inter-class similarity among subclasses, this places high demands on the model's detection capabilities. Consequently, most existing methods employ the two-stage object detection model based on anchor boxes, which offers higher detection accuracy. With the rapid development of deep learning technology, the detection capabilities of the FGOD model have been greatly improved; however, existing methods still suffer from the following two problems.

[0012] The first problem is that the insufficient discriminative power of the backbone network features leads to misidentification of similar subclasses of targets. To address this confusion, most research focuses on improving the structure of the target detection head or the loss function, with only a few efforts addressing the backbone network itself. Improvements to the target detection head have achieved good results due to the direct guidance of instance-level supervision signals. Conversely, lacking direct supervision, existing research on the backbone network typically relies on the designer's experience to make appropriate modifications to its structure, resulting in limited improvement to the discriminative power of the backbone network features. Considering that backbone network features are the common foundation of both the region proposal network and the target detection head, this situation is detrimental to improving the quality of candidate boxes and resolving misclassification issues between subclasses. Figure 1As shown in (a), the remote sensing images were selected from the FAIR1M-1.0 dataset. The baseline model using the traditional backbone network misidentified two Boeing 737 aircraft as A220s, which led to the easy misidentification of similar subclasses of targets.

[0013] The second problem is that the backbone network's insufficient ability to represent the direction of a directed target leads to errors in target localization. Existing FGOD models typically use Convolutional Neural Networks (CNNs) to extract spatial features from images. However, the horizontal rectangular convolutional kernels used in CNNs are severely limited in extracting features from targets facing any direction. In contrast, the frequency domain features of images contain rich directional information, which can support the accurate representation of the angle of a directed target. Existing frequency-domain-based attention modules typically use trigonometric functions to transform the spatial feature map in the frequency domain, obtaining channel attention by convolving multi-band feature maps, and then enhancing the spatial feature map accordingly. However, using trigonometric functions can only perform global frequency domain transformations and lacks local spatial localization capabilities, which is why existing frequency-domain-based attention modules usually only generate channel attention. In other words, most frequency-domain-based attention modules cannot effectively extract the frequency domain features of local targets. Figure 1 As shown in (b), the baseline model using the CNN backbone network has a deviation in the angular positioning of the ship, which will lead to inaccurate target positioning of the directed box.

[0014] Therefore, fine-grained object detection models are dedicated to detecting different subclasses of objects within the same major category. Two-stage object detection architectures are widely adopted by existing FGOD models due to their high detection accuracy, but they still face the following two problems: First, the insufficient discriminative power of the backbone network features leads to misidentification of similar subclasses of objects. Second, the insufficient ability of the backbone network features to express directional targets leads to errors in target localization. Summary of the Invention

[0015] To address the technical problems of existing fine-grained target detection models, such as the lack of direct guidance from supervision signals during the training of backbone network features and insufficient discriminative ability leading to misidentification of targets in similar subclasses, this invention proposes a fine-grained target detection method that integrates point-by-point frequency domain attention with image-level to instance-level supervision, employing a coarse-to-fine progressive learning approach. This method exhibits superior detection accuracy and generalization ability.

[0016] To achieve the above objectives, the technical solution of this invention is as follows: a fine-grained target detection method that integrates point-by-point frequency domain attention with image-level to instance-level supervision, employing a coarse-to-fine progressive learning approach, comprising the following steps:

[0017] Step 1: Construct a fine-grained target detection model: The spatial features output from each layer of the feature pyramid backbone network are fed into the PFA-DWT module, where the input spatial features and the obtained frequency domain features are fused point-by-point. The fused features are then fed into the C2FIm-FDE module to enhance their discriminative power. The enhanced backbone network features are fed into the directed RPN to generate directed candidate boxes, and the pooling features of each directed candidate box are obtained through rotational alignment of the region of interest. The pooling features of the directed candidate boxes are then fed into the C2FIn detection head to predict the category and position offset of the candidate boxes.

[0018] Step 2: Introduce binary cross-entropy loss and task consistency focus loss to train the constructed fine-grained object detection model, and obtain the trained fine-grained object detection model;

[0019] Step 3: Input the image to be detected into the trained fine-grained target detection model, and obtain the final detection result after non-maximum suppression.

[0020] Preferably, the PFA-DWT module performs DWT transformation on the spatial features output by each layer of the feature pyramid backbone network to obtain four frequency domain components; adaptively fuses the four frequency domain components to obtain a fused feature map; processes the fused feature map through deconvolution to generate a pointwise frequency domain attention map; and then processes the pointwise frequency domain attention map... The spatial features are precisely fused point by point with the input spatial features to obtain a feature map that integrates spatial and frequency domain information.

[0021] Preferably, the C2FIm-FDE module outputs the feature map from the PFA-DWT module. As input features, coarse-grained image-level features are extracted through parallel global average pooling and global max pooling via fully connected layers. These coarse-grained features are then processed using fully connected layers and activation functions to obtain fine-grained image-level features. Feature map output by PFA-DWT module The enhanced backbone network features are obtained by fusion.

[0022] Preferably, the C2FIn detection head processes the pooling features of the directed candidate boxes through a fully connected layer to obtain coarse-grained category features of the directed candidate boxes, and then processes the coarse-grained category features through a fully connected layer and an activation function to obtain fine-grained image-level features. ; Utilizing fully connected layers for fine-grained image-level features Processing to obtain fine-grained class confidence scores for predicted directed candidate boxes and rotation angle Pooling features of directed candidate boxes Obtain the center coordinates, width, and height offsets of the directed candidate box, and then determine the position, center coordinates, width, and height offsets, as well as the rotation angle of the directed candidate box. Determine the final location of the predicted target.

[0023] Preferably, the feature map that integrates spatial and frequency domain information ;in, Represents the Hartmann product. This represents element-wise addition. Spatial features representing the input;

[0024] The backbone network features ,in, represent Activation function This indicates element-wise multiplication;

[0025] Predict the final location of the target ;in, This represents the location of the directed candidate box predicted by the directed RPN. Here are the coordinates of the center point of the directed candidate box. , These are the offsets of the width and height of the directed candidate box, respectively. The angle is the rotation angle.

[0026] Preferably, the fused feature map ;in, and Representing spatial feature maps The low-frequency components, horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components, where H, W, and C represent the height, width, and number of channels of the spatial feature, respectively. This represents a splicing operation along the channel direction. This represents a convolution operation with a kernel size of 1×1. represent Activation function;

[0027] The point-by-point frequency domain attention map ;in, Represents the deconvolution operation. represent Activation function;

[0028] The coarse-grained image-level features ;in, and These represent the global average pooling operation function and the global max pooling operation function, respectively. The convolution operation function representing a fully connected layer;

[0029] The fine-grained image-level features ;

[0030] The coarse-grained category features The fine-grained category features The center point coordinates, width, and height offsets of the directed candidate box. ;in, This indicates a convolution operation with a kernel size of 3×3. This represents the convolution operation performed by the Strip module.

[0031] Preferably, the overall loss function ;in, The loss function for a directed RPN; The loss function of the C2FIm-FDE module is implemented using binary cross-entropy loss; It provides instance-level category and location loss functions and is implemented using coarse-grained instance-level classification loss and TF loss.

[0032] Preferably, the loss function of the C2FIm-FDE module

[0033] ;

[0034] in, Represents coarse-grained image-level labels, where NC represents the number of coarse-grained categories; Represents fine-grained image-level labels, where NF represents the number of fine-grained categories; and These represent the coarse-grained and fine-grained class confidence scores of the sample images, respectively. This represents the binary cross-entropy loss function; For coarse-grained image-level features, For fine-grained image-level features, This represents the convolution operation function for a fully connected layer.

[0035] Preferably, the loss function for instance-level category and location ;

[0036] And coarse-grained instance-level classification loss function ;

[0037] TF loss ;

[0038] in, This represents the task consistency focus classification loss. This represents the task consistency focus regression loss. This represents the intersection-union ratio (IoU) of the predicted directed boxes and their corresponding ground truth boxes. To predict the final location of the target, Represents location label, Represents the intersection-union ratio function; The coarse-grained class confidence score representing the directed candidate box is obtained through the convolution operation function of the fully connected layer. coarse-grained category features Obtained through processing; The fine-grained category confidence scores representing directed candidate boxes are obtained through the convolution operation function of the fully connected layer. Fine-grained category features Obtained through processing; and These represent the coarse-grained instance-level category label and the fine-grained instance-level category label corresponding to the directed candidate box, respectively.

[0039] Preferably, the coarse-grained image-level features and fine-grained image-level features The generation of these tags yielded guidance for both coarse-grained and fine-grained image-level labels; the coarse-grained image-level labels... Fine-grained image-level labels are derived from coarse-grained instance-level labels. Inferred from fine-grained instance-level tags;

[0040] The fine-grained category confidence score and the final location of the predicted target Prediction requires joint guidance from instance-level category labels and location labels;

[0041] If the sample image contains an instance of a coarse-grained category, the coarse-grained image-level label is 1; otherwise, the coarse-grained image-level label is 0. If the sample image contains an instance of a fine-grained category, the fine-grained image-level label is 1; otherwise, the fine-grained image-level label is 0.

[0042] The category confidence score and The convolution operation function of the fully connected layer is used to process coarse-grained image-level features respectively. and fine-grained image-level features Obtained through processing;

[0043] Task consistency focus classification loss = ;

[0044] Task consistency focus classification loss ;

[0045] Task consistency focus regression loss ;

[0046] in, This represents the directed candidate boxes generated by the directed RPN. and Let CE(·) represent the sets of positive and negative samples, respectively, and let CE(·) represent the cross-entropy loss function. This represents the Smooth L1 loss used for regression. Indicates focal loss. This represents the intersection-union ratio (IoU) of the predicted directed boxes and their corresponding ground truth boxes. Indicates the predicted final location of the target. Represents location label, and These represent the focus classification loss respectively. and focus regression loss Hyperparameters in [the context].

[0047] The beneficial effects of this invention are as follows: This invention proposes a Coarse-to-Fine Image-Level Supervised Feature Discriminability Enhancement (C2FIm-FDE) module based on coarse-to-fine progressive learning under image-level supervision. The C2FIm-FDE module infers coarse-grained and fine-grained image-level supervision signals from coarse-grained and fine-grained instance-level supervision signals, respectively. Based on the generated coarse-grained and fine-grained image-level category labels, it guides the backbone network features to gradually enhance their ability to discriminate image-level categories. This guides the progressive improvement of the discriminative ability of the image-level features generated by the backbone network in a coarse-to-fine process, effectively solving the problem of confusion between similar subclasses. This invention proposes a Point-Wise Frequency Attention Based on Discrete Wavelet Transform (PFA-DWT) module. The PFA-DWT module leverages the dual spatial and frequency domain localization advantages of DWT, extracting frequency components containing global target structure, texture, and angular information, while addressing the limitation of trigonometric function-based frequency transformations in extracting local target frequency features. The PFA-DWT module adaptively fuses features extracted from various frequency bands by DWT and fully utilizes DWT's dual spatial and frequency domain localization advantages to generate point-wise attention that combines spatial and channel attention advantages. By precisely fusing frequency domain features containing local target orientation information with spatial features through point-wise attention, the backbone network's ability to represent oriented targets is enhanced, enabling accurate target localization. To further enhance the FGOD model's ability to distinguish similar subclasses, this invention proposes a detection head based on coarse-to-fine instance-level supervised learning (C2FIn). The C2FIn detection head introduces coarse-grained and fine-grained instance-level labels sequentially, guiding the detection head to gradually focus on the differences between subclasses within the same coarse-grained category, thus improving the accuracy of similar subclass identification. The C2FIn detection head adds coarse-grained and fine-grained instance-level supervision signals sequentially to the detection head, enabling it to progressively focus on the differences between subclasses within the coarse-grained category, thus progressively improving the detection head's discriminative ability. Ablation experiments validate the effectiveness of the C2FIm-FDE module, the PFA-DWT module, and the C2FIn detection head, as well as any combination thereof. Quantitative comparisons with the popular FGOD model on the large-scale public datasets FAIRM-1.0 and FAIRM-2.0 demonstrate that the FGOD model of this invention exhibits superior detection accuracy and generalization ability.

[0048] The main contributions of this invention are as follows:

[0049] 1) To address the problem of insufficient feature discrimination capability of the backbone network in existing models, a C2FIm-FDE module is proposed and embedded into the backbone network. The C2FIm-FDE module uses the generated image-level supervision signal as a guide to progressively improve the discrimination capability of the image-level features generated by the backbone network in a coarse-to-fine process.

[0050] 2) To address the problem that the backbone network features of existing models are insufficient in expressing the directional target direction, a PFA-DWT module is proposed. The PFA-DWT module fully leverages the advantages of DWT's dual spatial and frequency domain localization. It can extract each frequency domain component containing global target structure, texture and angle information and perform adaptive fusion, and it can also extract the frequency domain features of local targets. Finally, it generates point-by-point attention of the input feature map, achieving accurate fusion of frequency domain features and input spatial features.

[0051] 3) A C2FIn detection head is proposed to further improve the FGOD model's ability to distinguish between similar subclasses. The C2FIn detection head focuses more on the differences between subclasses within the coarse-grained category by progressively adding coarse-grained and fine-grained instance-level supervision signals. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a visualization of the research motivation of this invention, where (a) is the misclassification problem caused by insufficient feature discrimination ability of the backbone network, and (b) is the target localization deviation caused by insufficient feature expression ability of the FGOD model in the directional target direction.

[0054] Figure 2 This is a flowchart of the present invention.

[0055] Figure 3 This is a flowchart of the calculation process of the PFA-DWT module of the present invention.

[0056] Figure 4 This is a flowchart of the calculation process of the C2FIm-FDE module of the present invention.

[0057] Figure 5The following is a visualization of the results of this invention, wherein (a) is the FAIR1M-1.0 dataset and (b) is the FAIR1M-2.0 dataset. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] like Figure 2 As shown, a fine-grained object detection method that integrates point-by-point frequency domain attention with image-level to instance-level supervision and coarse-to-fine progressive learning includes the following steps:

[0060] Step 1: Construct a fine-grained object detection model: The spatial features output from each layer of the Feature Pyramid Network (FPN) backbone are fed into the PFA-DWT module, where the input spatial features and the obtained frequency domain features are fused point-by-point. The fused features are then fed into the C2FIm-FDE module to enhance their discriminative power. Finally, the enhanced backbone network features are fed into the directed RPN to generate directed candidate boxes, and the features of each candidate box are obtained through rotation and region of interest alignment operations. Features of candidate boxes The C2FIn detector head is fed in to predict the category and position offset of the candidate box, and the final detection result is obtained after non-maximum suppression.

[0061] The overall architecture of the pointwise frequency domain attention module based on discrete wavelet transform, namely the PFA-DWT module, is as follows: Figure 3 As shown. The FPN backbone network upsampling from high-level features not only utilizes the high semantic, low-resolution information of the top layers (aiding classification), but also utilizes the shallow, low semantic, high-resolution information (aiding localization). First, the spatial features output by each layer of the FPN backbone network are subjected to DWT transform to obtain four frequency domain components:

[0062] ; (1)

[0063] in, This represents the input spatial feature map, where H, W, and C represent the height, width, and number of channels of the spatial feature map, respectively. Represents discrete wavelet transform. and Representing spatial feature maps The low-frequency components, the high-frequency components in the horizontal direction, the high-frequency components in the vertical direction, and the high-frequency components in the diagonal direction.

[0064] Secondly, the four frequency domain components are adaptively fused using the following formula to obtain the fused feature map:

[0065] ; (2)

[0066] in, Represents the fused feature map. This represents a splicing operation along the channel direction. This represents a convolution operation with a kernel size of 1×1. represent Activation function.

[0067] Subsequently, feature maps are generated. Pointwise frequency domain attention map (PFA), labeled as This process can be represented by the following formula:

[0068] ; (3)

[0069] in, Represents the deconvolution operation. represent Activation function.

[0070] Finally, the spatial features and frequency domain information are precisely fused point by point using the following formula to obtain a feature map that fuses the spatial and frequency domain information:

[0071] ; (4)

[0072] in, Feature maps representing the fusion of spatial and frequency domain information, Represents the Hartmann product. This represents element-wise addition.

[0073] In summary, the PFA-DWT module fully leverages the advantages of DWT's dual spatial and frequency domain localization. On one hand, it extracts the target's angular information through frequency domain transformation, compensating for the shortcomings of spatial features in this area. By decomposing features into frequency sub-bands in different directions (LH, HL, and HH), it achieves implicit modeling of target orientation information. Specifically, targets in different directions exhibit significant differences in response intensity across each sub-band. For example, horizontal structures show a stronger response in the HL sub-band, while vertical structures are more pronounced in the LH sub-band, and tilted structures are mainly reflected in the HH sub-band. Therefore, through joint learning of features from multiple directional sub-bands, the model can effectively capture the target's directional distribution characteristics, thereby enhancing its ability to represent oriented targets. On the other hand, PFA achieves precise fusion of spatial and frequency domain information. Figure 1 As shown in (b), the baseline model can accurately locate the ship after incorporating the PFA-DWT module. The PFA-DWT module is merely an attention enhancement module and has no loss.

[0074] The overall architecture of the C2FIm-FDE module, a feature discrimination enhancement module based on coarse-to-fine progressive learning under image-level supervision, is as follows: Figure 5 As shown. First, the feature map output by the PFA-DWT module. The features are used as input features to the C2FIm-FDE module, and the feature map is extracted using the following formula. The coarse-grained image-level features are labeled as coarse-grained image-level features. :

[0075] ; (5)

[0076] in, and These represent the global average pooling operation function and the global max pooling operation function, respectively. This represents the convolution operation function for a fully connected layer.

[0077] Secondly, in coarse-grained image-level features Based on this, fine-grained image-level features are further extracted using the following formula and labeled as fine-grained image-level features. :

[0078] ; (6)

[0079] Finally, in fine-grained image-level features Based on this, the enhanced backbone network features are obtained using the following formula and labeled as backbone network features. :

[0080] ; (7)

[0081] in, This indicates element-wise multiplication.

[0082] It is worth noting that the above-mentioned coarse-grained image-level features and fine-grained image-level features The generation of images was guided by both coarse-grained and fine-grained image-level labels, and the corresponding loss function is shown in the following equation:

[0083] ; (8)

[0084] in, The loss function representing the C2FIm-FDE module, Represents coarse-grained image-level labels, where NC represents the number of coarse-grained categories. This can be inferred from coarse-grained instance-level labels, that is: if a sample image (in the public datasets, i.e., the FAIR1M-1.0 dataset and the FAIR1M-1.0 dataset) contains the first... For instances of a coarse-grained category, then ,otherwise ; Represents fine-grained image-level labels, where NF represents the number of fine-grained categories. It can be inferred from fine-grained instance-level labels, and the reasoning process is the same as that for coarse-grained image-level labels. Similarly, and These represent the coarse-grained and fine-grained class confidence scores of the sample image, respectively. The coarse-grained image-level features are processed by the convolution operation function of the fully connected layer. and fine-grained image-level features The result is obtained through processing. This represents the binary cross-entropy loss. Since fine-grained categories often have higher appearance similarity and stronger semantic correlation, the BCE loss can impose constraints on each fine-grained category in a category-independent manner, effectively mitigating the problem of mutual interference between categories and enhancing the discriminative power of features at the fine-grained image level.

[0085] In summary, the C2FIm-FDE module generates fine-grained image-level features under the sequential guidance of coarse-grained and fine-grained image-level labels. Based on this, the characteristics of the backbone network were enhanced (using the idea of ​​residual connections), which resulted in enhanced backbone network characteristics. It exhibits strong discriminative power at the image-level category level, laying a solid foundation for subsequent instance-level category recognition. For example... Figure 1 As shown in (a), the baseline model can accurately identify all aircraft after integrating the C2FIm-FDE module.

[0086] Enhanced backbone network features The directed candidate boxes are fed into a directed RPN, and the pooling features of each candidate box are obtained by aligning it with the region of interest through rotation. Rotating the region of interest is used to extract pooling features.

[0087] The overall architecture of the C2FIn detection head, a detection head based on coarse-to-fine progressive learning under instance-level supervised conditions, is as follows: Figure 2 The light green area is shown. First, the coarse-grained category features of the directed candidate boxes are obtained using the following formula, and labeled as... :

[0088] ; (9)

[0089] in, Pooling features represent directed candidate boxes, and N represents coarse-grained category features. The dimension of.

[0090] Secondly, using coarse-grained category features Based on this, fine-grained category features are further obtained using the following formula, and labeled as follows. :

[0091] ; (10)

[0092] Inspired by Strip R-CNN, this invention utilizes fine-grained image-level features. Simultaneously predict the fine-grained class confidence score and rotation angle of the directed candidate boxes, labeled as follows: and The results are obtained from formulas (15) and (12). This is simply the process of predicting the classification score and angle for object detection.

[0093] Next, the center coordinates, width, and height offsets of the directed candidate box can be obtained using the following formula, denoted as follows: , , :

[0094] ; (11)

[0095] in, This indicates a convolution operation with a kernel size of 3×3. This represents the convolution operation performed by the Strip module.

[0096] Finally, the final location of the predicted target is obtained using the following formula and marked as follows. :

[0097] ;(12)

[0098] in, This represents the location of the directed candidate box predicted by the directed RPN.

[0099] The final detection result obtained after non-maximum suppression operation [Neubeck, Alexander and Luc Van Gool. “Efficient Non-Maximum Suppression.” 18th International Conference on Pattern Recognition (ICPR'06) 3 (2006): 850-855.].

[0100] Step 2: Introduce coarse-grained instance-level classification loss and task consistency focus loss to train the constructed fine-grained object detection model, and obtain the trained fine-grained object detection model.

[0101] It is worth noting that, in order to enhance consistency between classification and regression tasks, the fine-grained category confidence scores in the above inference process are... and the final location of the predicted target The prediction requires joint guidance from instance-level class labels and location labels; therefore, the relevant loss function is denoted as:

[0102] (13)

[0103] , (14)

[0104] , (15)

[0105] (16)

[0106] in, This represents the intersection-union ratio (IoU) of the predicted directed boxes and their corresponding ground truth boxes. To predict the final location of the target, Represents location label, Represents the intersection-union ratio function. The coarse-grained class confidence score representing the directed candidate box is obtained through the convolution operation function of the fully connected layer. coarse-grained category features Obtained through processing; The fine-grained category confidence scores representing directed candidate boxes are obtained through the convolution operation function of the fully connected layer. Fine-grained category features Obtained through processing; and These represent the coarse-grained instance-level category label and the fine-grained instance-level category label, respectively, for the directed candidate box. For each directed candidate box, its corresponding ground truth instance is determined by matching it with the ground truth box (usually based on rotation IoU). The fine-grained instance-level category label is directly derived from the original category label of the ground truth box, i.e., the fine-grained category label provided in the dataset. Based on this, the coarse-grained instance-level category label is obtained by merging the fine-grained label through a predefined category hierarchy mapping relationship. Specifically, semantically similar fine-grained categories are mapped to the same high-level category, thereby constructing the coarse-grained label. This represents a coarse-grained instance-level classification loss function. For example, TF loss can enhance the consistency between classification and regression while focusing on hard samples. This represents the task consistency focus classification loss. This represents the task consistency focus regression loss, and

[0107] = ;

[0108] ;

[0109] ;

[0110] in, This represents the directed candidate boxes generated by the directed RPN. and Let represent the sets of positive and negative samples, respectively. For the dense directed candidate boxes generated by the directed box generation network, Oriented R-CNN employs a multi-stage filtering mechanism to optimize the detection process. In the preprocessing stage, the initially generated candidate boxes are first coarsely screened based on classification prediction scores, retaining the top 2000 detection boxes at each level to control computational scale. Then, a non-maximum suppression (NMS) algorithm is used for fine-tuning, effectively eliminating redundant detection boxes with high spatial overlap by setting an IoU threshold of 0.8. From these directed boxes obtained across all levels, the top 2000 are selected based on classification prediction scores and sent to the positive / negative sample matching stage. In this stage, the intersection-union ratio (IoU) between the remaining 2000 directed candidate boxes and the ground truth target boxes is calculated, with an IoU threshold of 0.5. Simultaneously, the number of directed candidate boxes with an IoU greater than or equal to 0.5 is determined. If the number of positive candidate boxes is greater than or equal to 128, then 128 directed candidate boxes are randomly selected as positive samples; if the number of positive candidate boxes is less than 128, then all of these directed candidate boxes are used as positive samples. The total number of positive and negative samples is set to 512 in the configuration file. Negative samples are randomly selected from directed candidate boxes with IoU less than 0.5 (512 - the number of positive samples). CE(·) represents the cross-entropy loss function. This represents the Smooth L1 loss used for regression. Indicates focal loss. This represents the intersection-union ratio (IoU) of the predicted directed boxes and their corresponding ground truth boxes. Indicates the predicted final location of the target. The location label represents the instance-level label. The method for obtaining these instance-level labels has already been explained in Oriented R-CNN. and These represent the focus classification loss respectively. and focus regression loss The hyperparameters in the equation. The predicted directed box is obtained from equation (12), which yields... The five parameters are sufficient to determine the predicted directed box. The truth boxes are submitted to the official website for testing.

[0111] Finally, the overall loss function of the model of this invention is denoted as... As shown in the following formula:

[0112] ;

[0113] in, The loss function for a directed RPN.

[0114] Step 3: Input the image to be detected into the trained fine-grained target detection model, and obtain the final detection result after non-maximum suppression.

[0115] This invention uses two large-scale public datasets, FAIR1M-1.0, to verify the performance of fine-grained object detection [X. Sun, P. Wang, Z. Yan, F. Xu, R. Wang, W. Diao, J. Chen, J. Li, Y. Feng, T. Xu, M. Weinmann, S. Hinz, C. Wang, and K. Fu, “Fair1m: A benchmark dataset for fine-grained object recognition in high resolution remote sensing imagery,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 184, pp.116–130, 2022. [Online]. Available:

[0116] Experiments were conducted using [https: / / www.sciencedirect.com / science / article / pii / S0924271621003269] and FAIR1M-2.0, with all results tested on the official server. To our knowledge, the FAIR1M-2.0 dataset is currently the largest publicly available dataset in this field. The FAIR1M-1.0 dataset contains 24,625 remote sensing images, ranging in size from 1000×1000 to 10000×10000, of which 16,488 images were used for training, and the remainder for testing. The FAIR1M-1.0 dataset includes five coarse-grained categories: Aircraft, Ships, Cars, Stadiums, and Roads, which are further subdivided into 37 fine-grained categories: Boeing 737 (B737), Boeing 777 (B777), Boeing 747 (B747), Boeing 787 (B787), A321, A220, A330, A350, C919, ARJ21, Other Aircraft (OA), Passenger Ships (PS), Motorboats (MB), Fishing Boats (FB), Tugboats (TB), Engineering Vessels (ES), Liquid... The dataset includes the following categories: cargo ships (LCS), dry cargo ships (DCS), warships (WS), other ships (OS), small cars (SC), buses (BUS), freight trucks (CT), dump trucks (DT), vans (VAN), trailers (TRI), tractors (TRC), truck tractors (TT), excavators (EX), other vehicles (OV), baseball fields (BF), basketball courts (BC), football fields (FF), tennis courts (TC), roundabouts (RA), intersections (IS), and bridges (BR). The FAIR1M-2.0 dataset contains 42,796 remote sensing images, with 16,488, 8,287, and 18,021 images for training, validation, and testing, respectively. Both the training and validation sets are used to train the model in this invention. The target categories in the FAIR1M-2.0 dataset are identical to those in the FAIR1M-1.0 dataset. It is worth noting that the test results on the official server for both datasets do not include the subclasses OA, OS, and OV.

[0117] This invention uses a combination of Oriented-RCNN and TF loss as the baseline model, with ResNet50-FPN [T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the 11 IEEE conference on computer vision and pattern recognition, 2017, pp.2117–2125.] as the backbone network. The final model trained in this invention is named PFA-C2FI. This invention uses stochastic gradient descent for model optimization, with momentum decay and weight decay set to 0.9 and 0.0001, respectively. The batch size, learning rate, and IoU threshold in NMS are consistent with the baseline model. The model of this invention was trained for 12 epochs. Experiments were conducted on the MMrotated experimental platform using the PyTorch framework, with the source code running on four TITAN RTX GPUs (24GB × 4 memory). Hyperparameters in the focus classification loss are described below. Hyperparameters in focus regression loss Similar to the settings in TF loss. To quantitatively evaluate the overall performance of the FGOD model of this invention, the average precision (AP) for each class and the mean average precision (mAP) for all classes were used as evaluation metrics.

[0118] This invention conducted ablation experiments on the PFA-DWT module, C2FIm-FE module, and C2FIn Head to evaluate their effectiveness. As shown in Table 1, when the PFA-DWT module, C2FIm-FDE module, and C2FIn Head were each added to the baseline model, the mAP of the baseline model improved by 1.98%, 2.46%, and 1.84%, respectively, verifying the effectiveness of each module individually. When the PFA-DWT module was combined with the C2FIm-FDE module and the C2FIn detection head, respectively, the mAP of the baseline model improved by 3.45% and 3.27%, respectively. When the C2FIm-FDE module was combined with the C2FIn detection head, the mAP of the baseline model improved by 3.34%. When all three modules were added to the baseline model simultaneously, the mAP of the baseline model improved by 3.81%, verifying the effectiveness of any combination of the three modules.

[0119] Table 1. Performance of the PFA-DWT module, C2FIm-FDE module, and C2FIn detection head on the FAIR1M-1.0 dataset.

[0120]

[0121] 1) FAIRM-1.0 Dataset: The model of this invention was quantitatively compared with 14 popular FGOD models on the FAIRM-1.0 dataset. As shown in Table 2, the model of this invention achieved the highest mAP of 44.36%, which is higher than RetinaNet [T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for dense object detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318–327, 2020.], R 3 Det、S 2ANet, FRCNN [S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, 2017.], DLL [Y. Liu, X. Hu, W. Liu, X. Wang, C. Chen, P. Wan, Y. Jiang, and P. Zhong, “Discriminative latent-space learning for fine-grained object detection in remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 63, pp. 1–14, 2025], ROITrans [J. Ding, N. Xue, Y. Long, G.-S. Xia, and Q. Lu, “Learning roi transformer for oriented object detection in aerial images,” in 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2844–2853.], ORCNN [X. Xie, G. Cheng, J. Wang, X. Yao, and J. Han, “Oriented r-cnn for object detection,” in Proceedings of the IEEE / CVF international conference on computer vision, 2021, pp. 3520–3529.], PCLDet, PETDet, SFRNet, DRNet, Strip R-CNN and MSC-TF [X. Qian, Q. Jian, W. Wang, X. Yao, and G.Cheng, “Incorporating multiscale context and task-consistent focal loss into oriented object detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 63, pp. 1–11, 2025] increased the accuracy by 17.78%, 13.29%, 9.65%, 7.89%, 7.53%, 6.1%, 5.87%, 5.51%, 3.97%, 3.81%, 3.62%, 1.16%, and 0.73%, respectively. These results validate the overall effectiveness of the proposed model on the FAIRM-1.0 dataset. Furthermore, the proposed model achieved the highest AP in 19 out of 34 object categories, indicating good generalization ability.

[0122] 2) FAIRM-2.0 Dataset: The model of this invention was quantitatively compared with 12 popular FGOD models on the larger-scale FAIRM-2.0 dataset. As shown in Table 3, the model of this invention achieved the highest mAP of 48.67%, which is higher than RetinaNet and R... 3 Det, FRCNN, GLVE [Y. Xu, M. Fu, Q. Wang, Y. Wang, K. Chen, G.-S. 2ANet [J. Han, J. Ding, J. Li, and G.-S. Xia, “Align deep features for oriented object detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–11, 2022.], ROITrans, ORCNN, PCLDet, SFRNet, PETDet, DRNet, and Strip R-CNN increased efficiency by 16.16%, 10.99%, 9.30%, 8.34%, 8.16%, 4.56%, 4.38%, 4.01%, 2.99%, 2.66%, 1.63%, and 1.23%, respectively. These results verify the overall effectiveness of the model in this invention on the FAIRM-2.0 dataset and further verify the generalization ability of the model in this invention.

[0123] Table 2 shows the comparison results between the present invention and existing methods on the FAIR1M-1.0 dataset.

[0124]

[0125] Table 3. Comparison results of the present invention and existing methods on the FAIR1M-2.0 dataset.

[0126]

[0127] The visualization results of the model of this invention on the test sets of the FAIR1M-1.0 and FAIR1M-2.0 datasets are shown, such as... Figure 5 As shown, it can be seen that the model of the present invention has good detection performance for targets in five major categories under various complex background conditions, which further verifies the effectiveness of the model of the present invention.

[0128] This invention proposes a novel two-stage FGOD model, using a combination of Oriented R-CNN and TF loss as the baseline model. Its main contributions include three aspects: First, a C2FIm-FDE module is proposed and integrated into the backbone network of the baseline model to address the insufficient discriminative power of the backbone network features. Specifically, the C2FIm-FDE module, guided by generated image-level category labels, gradually enhances the discriminative power of the backbone network features for image-level categories through a coarse-to-fine progressive learning strategy. Second, a PFA-DWT module is proposed and integrated into the backbone network of the baseline model to address the insufficient ability of the backbone network features to represent directional targets. Specifically, the PFA-DWT module fully leverages the advantages of DWT's dual spatial and frequency domain localization, enabling the extraction of local target directional information and achieving precise point-by-point fusion of spatial and frequency domain features. Finally, this invention proposes a C2FIn detector head to further enhance the FGOD model's ability to discriminate similar subclasses. The C2FIn detection head guides the model to focus on differences between different subclasses within the same coarse-grained category by progressively adding coarse-grained and fine-grained instance-level supervision signals. Ablation experiments validated the effectiveness of the PFA-DWT module, the C2FIm-FDE module, and the C2FIn detection head, as well as any combination thereof. The FGOD model of this invention achieved mAPs of 44.36% and 48.67% on the FAIR1M-1.0 and FAIR1M-2.0 datasets, respectively. Quantitative comparisons with popular FGOD models demonstrate the overall effectiveness of the model of this invention.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fine-grained target detection method that integrates point-by-point frequency domain attention with image-level to instance-level supervision, employing a coarse-to-fine progressive learning approach, characterized in that... The steps are as follows: Step 1: Construct a fine-grained target detection model: The spatial features output from each layer of the feature pyramid backbone network are fed into the PFA-DWT module, where the input spatial features and the obtained frequency domain features are fused point-by-point. The fused features are then fed into the C2FIm-FDE module to enhance their discriminative power. The enhanced backbone network features are fed into the directed RPN to generate directed candidate boxes, and the pooling features of each directed candidate box are obtained through rotational alignment of the region of interest. The pooling features of the directed candidate boxes are then fed into the C2FIn detection head to predict the category and position offset of the candidate boxes. Step 2: Introduce binary cross-entropy loss and task consistency focus loss to train the constructed fine-grained object detection model, and obtain the trained fine-grained object detection model; Step 3: Input the image to be detected into the trained fine-grained target detection model, and obtain the final detection result after non-maximum suppression.

2. The fine-grained target detection method according to claim 1, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and progressive learning from coarse to fine, is characterized in that... The PFA-DWT module performs DWT transformation on the spatial features output from each layer of the feature pyramid backbone network to obtain four frequency domain components; adaptively fuses the four frequency domain components to obtain a fused feature map; processes the fused feature map through deconvolution to generate a pointwise frequency domain attention map; and then processes the pointwise frequency domain attention map... The spatial features are precisely fused point by point with the input spatial features to obtain a feature map that integrates spatial and frequency domain information.

3. The fine-grained target detection method according to claim 2, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and progressive learning from coarse to fine, is characterized in that... The C2FIm-FDE module outputs a feature map from the PFA-DWT module. As input features, coarse-grained image-level features are extracted through parallel global average pooling and global max pooling via fully connected layers. These coarse-grained features are then processed using fully connected layers and activation functions to obtain fine-grained image-level features. Feature map output by PFA-DWT module The enhanced backbone network features are obtained by fusion.

4. The fine-grained target detection method according to claim 3, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and progressive learning from coarse to fine, is characterized in that... The C2FIn detection head processes the pooling features of the directed candidate boxes through a fully connected layer to obtain coarse-grained category features, and then processes these coarse-grained category features through a fully connected layer and an activation function to obtain fine-grained image-level features. ; Utilizing fully connected layers for fine-grained image-level features Processing to obtain fine-grained class confidence scores for predicted directed candidate boxes and rotation angle Pooling features of directed candidate boxes Obtain the center coordinates, width, and height offsets of the directed candidate box, and then determine the position, center coordinates, width, and height offsets, as well as the rotation angle of the directed candidate box. Determine the final location of the predicted target.

5. The fine-grained target detection method according to claim 4, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and progressive learning from coarse to fine, is characterized in that... The feature map that integrates spatial and frequency domain information ;in, Represents the Hartmann product. This represents element-wise addition. Spatial features representing the input; The backbone network features ,in, represent Activation function This indicates element-wise multiplication; Predict the final location of the target ;in, This represents the location of the directed candidate box predicted by the directed RPN. Here are the coordinates of the center point of the directed candidate box. , These are the offsets of the width and height of the directed candidate box, respectively. The angle is the rotation angle.

6. The fine-grained target detection method according to claim 5, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and progressive learning from coarse to fine, is characterized in that... The fused feature map ;in, and Representing spatial feature maps The low-frequency components, horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components, where H, W, and C represent the height, width, and number of channels of the spatial feature, respectively. This represents a splicing operation along the channel direction. This represents a convolution operation with a kernel size of 1×1. represent Activation function; The point-by-point frequency domain attention map ;in, Represents the deconvolution operation. represent Activation function; The coarse-grained image-level features ;in, and These represent the global average pooling operation function and the global max pooling operation function, respectively. The convolution operation function representing a fully connected layer; The fine-grained image-level features ; The coarse-grained category features The fine-grained category features The center coordinates, width, and height offsets of the directed candidate box. ;in, This indicates a convolution operation with a kernel size of 3×3. This represents the convolution operation performed by the Strip module.

7. The fine-grained target detection method according to any one of claims 1-6, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and coarse-to-fine progressive learning, is characterized in that... The overall loss function ;in, The loss function for a directed RPN; The loss function of the C2FIm-FDE module is implemented using binary cross-entropy loss; It provides instance-level category and location loss functions and is implemented using coarse-grained instance-level classification loss and TF loss.

8. The fine-grained target detection method according to claim 7, characterized by graph fusion point-by-point frequency domain attention and image-level to instance-level supervision with coarse-to-fine progressive learning, is characterized in that... The loss function of the C2FIm-FDE module ； in, Represents coarse-grained image-level labels, where NC represents the number of coarse-grained categories; Represents fine-grained image-level labels, where NF represents the number of fine-grained categories; and These represent the coarse-grained and fine-grained class confidence scores of the sample images, respectively. This represents the binary cross-entropy loss function; For coarse-grained image-level features, For fine-grained image-level features, This represents the convolution operation function for a fully connected layer.

9. The fine-grained target detection method according to claim 8, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and progressive learning from coarse to fine, is characterized in that... The loss function for instance-level category and location ; And coarse-grained instance-level classification loss function ; TF loss ; in, This represents the task consistency focus classification loss. This represents the task consistency focus regression loss. This represents the intersection-union ratio (IoU) of the predicted directed boxes and their corresponding ground truth boxes. To predict the final location of the target, Represents location label, Represents the intersection-union ratio function; The coarse-grained class confidence score representing the directed candidate box is obtained through the convolution operation function of the fully connected layer. coarse-grained category features Obtained through processing; The fine-grained category confidence scores representing directed candidate boxes are obtained through the convolution operation function of the fully connected layer. Fine-grained category features Obtained through processing; and These represent the coarse-grained instance-level category label and the fine-grained instance-level category label corresponding to the directed candidate box, respectively.

10. The fine-grained target detection method according to claim 9, which integrates point-by-point frequency domain attention with image-level to instance-level supervision and progressive learning from coarse to fine, is characterized in that... The coarse-grained image-level features and fine-grained image-level features The generation of these tags yielded guidance for both coarse-grained and fine-grained image-level labels; the coarse-grained image-level labels... Fine-grained image-level labels are derived from coarse-grained instance-level labels. Inferred from fine-grained instance-level tags; The fine-grained category confidence score and the final location of the predicted target Prediction requires joint guidance from instance-level category labels and location labels; If the sample image contains an instance of a coarse-grained category, the coarse-grained image-level label is 1; otherwise, the coarse-grained image-level label is 0. If the sample image contains an instance of a fine-grained category, the fine-grained image-level label is 1; otherwise, the fine-grained image-level label is 0. The category confidence score and The convolution operation function of the fully connected layer is used to process coarse-grained image-level features respectively. and fine-grained image-level features Obtained through processing; Task consistency focus classification loss = ; Task consistency focus classification loss ; Task consistency focus regression loss ; in, This represents the directed candidate boxes generated by the directed RPN. and Let CE(·) represent the sets of positive and negative samples, respectively, and let CE(·) represent the cross-entropy loss function. This represents the Smooth L1 loss used for regression. Indicates focal loss. This represents the intersection-union ratio (IoU) of the predicted directed boxes and their corresponding ground truth boxes. Indicates the predicted final location of the target. Represents location label, and Representing the focus classification loss respectively and focus regression loss Hyperparameters in [the context].

Images