A dual-paradigm combined remote sensing image end-to-end fine-grained target detection method

Abstract

The application discloses a kind of dual paradigm combined remote sensing image end-to-end fine-grained target detection method, it is related to computer vision and pattern recognition technical field, including: extracting multi-scale feature map by feature extraction network;Based on sparse directional proposal network, using one-to-one sparse matching strategy, generate no duplicate sparse directional region proposal and its corresponding proposal feature and coarse-grained target confidence;Based on query perception refinement head, using one-to-many dense matching strategy, obtain fine-grained classification probability and refined bounding box;In the training stage, introduce double auxiliary supervision head to provide additional dense supervision signal;In inference stage, adopt two-stage consensus scoring mechanism, finally obtain fine-grained target detection result.The method realizes the end-to-end remote sensing image fine-grained target detection without NMS dependence, through the cooperation of sparse and dense dual paradigm, effectively improves the positioning accuracy and fine-grained classification performance.

Description

Technical Field

[0001] This invention relates to the field of computer vision and pattern recognition technology, and more specifically to a dual-paradigm joint end-to-end fine-grained target detection method for remote sensing images. Background Technology

[0002] Intelligent interpretation of remote sensing images is an important research direction in the fields of computer vision and pattern recognition. With the rapid development of high-resolution remote sensing satellite technology, it has become possible to perform fine-grained detection of typical targets such as ships, aircraft, and vehicles. It has important practical significance in application scenarios such as urban planning, aviation management, and disaster response.

[0003] In existing technologies, fine-grained object detection mainly adopts a two-stage detection framework. This framework decouples the localization and classification tasks through the architecture design of two stages: region proposal generation and bounding box refinement, which reduces the mutual interference between multiple tasks to a certain extent. However, the two-stage method inevitably relies on non-maximum suppression for post-processing to eliminate redundant detection boxes. This manually designed post-processing step not only introduces additional computational overhead, but the sensitivity of its threshold parameter can easily lead to the incorrect suppression of high-quality detection boxes or the incorrect retention of low-quality detection boxes, which undermines the end-to-end optimization characteristics of the model and restricts further improvement of detection accuracy.

[0004] In recent years, query-based end-to-end detection transformers have been widely used in general object detection. These methods achieve global deduplication through a one-to-one allocation strategy, eliminating the dependence on non-maximum suppression. However, when these methods are directly applied to fine-grained remote sensing object detection, their performance is often inferior to carefully tuned two-stage baseline methods. On the one hand, although sparse one-to-one matching strategies are suitable for global deduplication, they are difficult to provide sufficiently rich gradient information to learn the subtle discriminative features required for fine-grained recognition. On the other hand, the same set of queries needs to simultaneously handle multiple conflicting optimization objectives such as localization, classification, and deduplication. This highly coupled architecture design leads to serious task interference problems in fine-grained detection scenarios.

[0005] Furthermore, although existing studies have attempted to optimize fine-grained recognition through contrastive learning and prototype learning, current methods neglect the synergy between sparse supervision and dense learning in the end-to-end detection framework. Specifically, in the sparse matching paradigm, the difference in classification costs between visually similar subcategories becomes negligible, causing regression costs to dominate the matching process, which may lead to the selection of queries with precise localization but ambiguous classification features. At the same time, one-to-one sparse supervision signals are difficult to cover the subtle differences in the target features, while one-to-many dense supervision, although it can provide rich gradients, cannot achieve global deduplication. This inherent contradiction between global deduplication and fine-grained recognition has not yet been effectively resolved.

[0006] Therefore, how to design a dual-paradigm joint end-to-end fine-grained target detection method for remote sensing images that can balance positioning accuracy and fine-grained recognition capability without relying on non-maximum suppression post-processing, thereby effectively improving the detection performance of fine-grained targets in remote sensing images, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a dual-paradigm joint end-to-end fine-grained target detection method for remote sensing images, which aims to solve the problems in the prior art, such as two-stage detectors relying on nonmaximum suppression, query-based detectors having insufficient sparse supervision, contradictions between global deduplication and fine-grained recognition, and coupling interference of multi-task architecture, thereby achieving high-precision fine-grained target detection without post-processing.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A dual-paradigm joint end-to-end fine-grained target detection method for remote sensing images includes the following steps: S1. Input a remote sensing image and extract multi-scale feature maps through a feature extraction network; S2. Input the multi-scale feature map into the sparse orientation proposal network, and use a one-to-one sparse matching strategy to generate sparse orientation region proposals without repetition and their corresponding proposal features and coarse-grained target confidence. S3. Input the sparse directional region proposal, proposal features, and multi-scale feature map into the query-aware refinement head, and use a one-to-many dense matching strategy to perform fine-grained classification and bounding box refinement on each proposal to obtain the fine-grained classification probability and the refined bounding box. S4. During the training phase, a dual auxiliary supervision head is introduced to provide additional dense supervision signals to the sparse orientation proposal network and query-aware refinement head through a dense feature supervision path and an auxiliary sample injection path. S5. In the inference phase, a two-stage consensus scoring mechanism is adopted, which integrates the coarse-grained target confidence and fine-grained classification probability to calculate the final consensus score of each proposal, and obtains the fine-grained target detection result based on the final consensus score and the refined bounding box.

[0010] Preferably, in S1, the feature extraction network includes a backbone network and a feature pyramid network; The backbone network is used to extract basic features from the input image, and the feature pyramid network is used to perform multi-scale processing on the basic features to generate multi-scale feature maps.

[0011] Preferably, S2 includes: The multi-scale feature map is input into the Transformer encoder for feature enhancement to obtain the encoded features; Initialize a set of learnable queries, input the queries and encoded features into the Transformer decoder for interaction, and output the query features optimized by the decoder. The optimized query features are input into the class-independent detection head, and the directional region proposal parameters and coarse-grained target confidence scores corresponding to each query are output. Sparse directional region proposals are generated based on the directional region proposal parameters, and the optimized query features are used as the proposal features corresponding to each proposal.

[0012] Preferably, in S2, the sparse orientation proposal network adopts a classification-agnostic training strategy, maps all fine-grained subclasses to a single foreground class, and employs a one-to-one bipartite graph matching strategy, so that each real target is matched by only one predicted sparse orientation region proposal.

[0013] Preferably, in step S2, the sparse directional proposal network employs an angle-weighted regression loss function. Optimize:

[0014] in, The L1 loss represents the position and size of the bounding box. L1 loss represents the angle. To balance the hyperparameters.

[0015] Preferably, S3 includes: For each sparse directional region proposal, region of interest features are extracted from the multi-scale feature map through region of interest alignment, and semantic features are extracted from the proposal features through cross-attention operation; The region of interest features and semantic features are fused to obtain a unified feature representation; The unified feature representation is input into the decoupled gated feedforward network to generate task-adaptive classification and regression features; Based on the classification and regression features, the fine-grained classification probability and the refined bounding box are predicted respectively.

[0016] Preferably, the decoupled gated feedforward network generates classification gated signals through a gating mechanism. and regression gating signal and representation of shared features Modulation is performed to obtain classification features. and regression features :

[0017]

[0018]

[0019]

[0020]

[0021] in, To unify feature representation, σ is the activation function. , , , , For learnable projection matrices, It represents the Hadamardi (or Hadama) stack.

[0022] Preferably, in step S3, the query-aware refinement head uses query-aware recognition loss. Supervision:

[0023] in, Indicates sample weights, based on proposal quality. Dynamic adjustment; For cross-entropy loss, For the predicted probability distribution, These are the corresponding real tags.

[0024] Preferably, in step S4, introducing dual auxiliary supervision heads includes: By fusing multi-scale feature maps and encoder features through the decoupling layer attention module, fused features are obtained; A dense feature supervision path is constructed on the fused features, and prediction is performed using an anchorless dense head. Gradient feedback is provided through dense prediction loss. An auxiliary sample injection path is constructed, which converts the prediction of the dense feature supervision path into an auxiliary query group and injects it into the sparse directional proposal network and query-aware refinement head to generate additional training samples for optimization.

[0025] Preferably, S5 includes: For the i-th proposal, obtain its coarse-grained target confidence score. and fine-grained classification probability distribution C represents the number of fine-grained categories, and bg represents the background. For each fine-grained category Calculate consensus score , For balance parameters; Calculate background score ; Aggregate all scores into a vector After normalization, the final probability distribution is obtained. ; The category with the highest score is selected as the prediction result, and combined with the refined bounding box, the final fine-grained object detection result is output.

[0026] As can be seen from the above technical solution, compared with the prior art, the technical solution of the present invention has the following beneficial effects: 1. This method constructs a sparse directional proposal network as a coarse-grained stage and uses a one-to-one matching strategy to generate non-repeating directional region proposals. This eliminates the dependence on post-processing for non-maximum suppression, realizes end-to-end target detection inference, avoids the problem of false deletion or redundancy of detection boxes caused by manually designed thresholds, and improves the stability of detection results and inference efficiency.

[0027] 2. By designing a query-aware refinement head as the fine-grained stage, adopting a one-to-many dense matching strategy, and combining it with the dense supervision signals provided by dual auxiliary supervision heads, the problem of insufficient sparse supervision is effectively alleviated, the model's ability to distinguish subtle differences between fine-grained categories is enhanced, and the accuracy of fine-grained classification is improved.

[0028] 3. By introducing a decoupled gated feedforward network, the feature representations of classification and regression tasks are decoupled, and a two-stage consensus scoring mechanism is adopted to integrate coarse-grained confidence and fine-grained classification probability, which reduces the mutual interference between multiple tasks and achieves a balance between localization accuracy and fine-grained recognition accuracy, thus comprehensively improving the overall performance of fine-grained target detection. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0030] Figure 1 A flowchart of a dual-paradigm joint remote sensing image end-to-end fine-grained target detection method provided in an embodiment of the present invention; Figure 2 This is a framework diagram of a target detection model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the decoupled gated feedforward network structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the visualization effect provided for an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] like Figure 1 and Figure 2 As shown, this embodiment provides a dual-paradigm joint end-to-end fine-grained target detection method for remote sensing images, including the following steps: S1. Input a remote sensing image and extract multi-scale feature maps through a feature extraction network; S2. Input the multi-scale feature map into the sparse orientation proposal network, and use a one-to-one sparse matching strategy to generate sparse orientation region proposals without repetition and their corresponding proposal features and coarse-grained target confidence. S3. Input the sparse directional region proposal, proposal features, and multi-scale feature map into the query-aware refinement head, and use a one-to-many dense matching strategy to perform fine-grained classification and bounding box refinement on each proposal to obtain the fine-grained classification probability and the refined bounding box. S4. During the training phase, a dual auxiliary supervision head is introduced to provide additional dense supervision signals to the sparse orientation proposal network and query-aware refinement head through a dense feature supervision path and an auxiliary sample injection path. S5. In the inference phase, a two-stage consensus scoring mechanism is adopted, which integrates the coarse-grained target confidence and fine-grained classification probability to calculate the final consensus score of each proposal, and obtains the fine-grained target detection result based on the final consensus score and the refined bounding box.

[0033] It employs a dual-paradigm collaborative design of a sparse directional proposal network and a query-aware refinement head. In the coarse-grained stage, it utilizes one-to-one matching to achieve non-repeating directional region proposal generation and global deduplication. In the fine-grained stage, it uses one-to-many matching to learn discriminative features and refine bounding boxes. Simultaneously, it introduces dual auxiliary supervision heads to provide dense gradient feedback to accelerate training. Finally, it integrates the outputs of the two stages through a two-stage consensus scoring mechanism, achieving end-to-end detection without the need for non-maximum suppression post-processing. This effectively alleviates interference between multiple tasks and improves the model's fine-grained recognition capability for semantically confusing targets.

[0034] The following provides a further detailed explanation of each step in the above method; In this embodiment, S1, a remote sensing image is input, and a multi-scale feature map is extracted through a feature extraction network; The feature extraction network includes a backbone network and a feature pyramid network. The backbone network is used to extract basic features from the input image, and the feature pyramid network is used to perform multi-scale processing on the basic features to generate multi-scale feature maps.

[0035] In this embodiment S2, the multi-scale feature map is input into the sparse orientation proposal network, and a one-to-one sparse matching strategy is used to generate sparse orientation region proposals without repetition, along with their corresponding proposal features and coarse-grained target confidence scores; including: The multi-scale feature map is input into the Transformer encoder for feature enhancement to obtain the encoded features; Initialize a set of learnable queries, input the queries and encoded features into the Transformer decoder for interaction, and output the query features optimized by the decoder. The optimized query features are input into the class-independent detection head, and the directional region proposal parameters and coarse-grained target confidence scores corresponding to each query are output. Sparse directional region proposals are generated based on the directional region proposal parameters, and the optimized query features are used as the proposal features corresponding to each proposal.

[0036] The sparse orientation proposal network employs a classification-agnostic training strategy, mapping all fine-grained subclasses to a single foreground class, and uses a one-to-one bipartite graph matching strategy, ensuring that each real target is matched by only one predicted sparse orientation region proposal.

[0037] Furthermore, the sparse directional proposal network employs an angle-weighted regression loss function. Optimize:

[0038] in, The L1 loss represents the position and size of the bounding box. L1 loss represents the angle. To balance the hyperparameters, a value of 0.2 is set to adjust the weight ratio of angle loss to geometric position loss. This effectively suppresses the excessive influence of angle error on matching cost and ensures that the model achieves a balance between localization and angle regression.

[0039] In one embodiment, if the initial learnable number of queries is 300, the final output can be a sparse directed region proposal set. Each proposal Includes position parameters and target confidence ; and Indicates the coordinates of the center point of the target bounding box. and This indicates the width and height of the bounding box. This indicates the rotation angle of the bounding box, used to describe the orientation of the target in the image plane.

[0040] In this embodiment S3, the sparse directional region proposal, proposal features, and multi-scale feature map are input into the query-aware refinement head. A one-to-many dense matching strategy is used to perform fine-grained classification and bounding box refinement on each proposal to obtain the fine-grained classification probability and the refined bounding box. Unlike the sparse paradigm of sparse directional proposal networks, the Query-Aware Refinement Head (QARH) employs dense one-to-many matching supervision, assigning multiple positive samples to each proposal, thereby providing rich gradient information to learn discriminative fine-grained features. The feature extraction of the query-aware refinement head adopts a dual-path strategy: 1) extracting region of interest features from multi-scale feature maps through region of interest alignment; 2) extracting semantic features from the proposal features of the sparse directed proposal network through the query-aware RoI attention mechanism. To maintain the stability of the sparse directional proposal network training, a gradient truncation operation is applied before inputting the query features into the attention mechanism; specifically, for the ... A proposal, setting This represents the query features from the sparse directional proposal network, where The dimension for querying features; This represents the RoI feature map extracted by RoIAlign, where , and Let represent the height, width, and number of channels of the feature map, respectively. Then, the two complementary feature vectors extracted by QARH can be represented as:

[0041]

[0042] in, Implement cross-attention operation for query awareness. This represents the operation of flattening the feature map into a one-dimensional vector. As a linear projection layer, it projects the flattened spatial features onto a plane that is parallel to the plane. Same dimension, Encoding global semantic context, Used to encode local spatial features is the output dimension of the feature vector.

[0043] like Figure 3As shown, to alleviate the interference between classification and regression tasks, this embodiment proposes a decoupled gated feedforward network (FFN). The decoupled gated feedforward network dynamically adjusts the information flow of shared features through a gating mechanism to generate task-adaptive classification and regression feature representations. This module is based on a unified representation. Generate task-specific gating signals:

[0044]

[0045]

[0046] in, The SiLU activation function is used. , , For learnable projection matrices, As an intermediate feature dimension, These are the classification-gated signal, the regression-gated signal, and the shared feature representation, respectively. The decoupled features are obtained through element-level modulation.

[0047]

[0048] in, , Let be the projection matrix. Output dimensions for task-specific features. This represents the Hadamard product, also known as element-wise multiplication. and These are task-adaptive feature representations for classification and regression tasks, respectively.

[0049] Furthermore, the query-aware refinement head is supervised by Query-Aware Recognition Loss (QARL), which dynamically adjusts sample weights based on proposal quality, enabling the model to focus on high-quality samples. The first proposal is to calculate the quality item:

[0050] in, The target confidence level predicted by SOPN represents the probability that the proposal includes the prospective target; To refine the intersection-union ratio (IUU) of the proposed values ​​and their corresponding ground truth values, and to measure the accuracy of the bounding box localization; This is a hyperparameter used to balance the contribution of positioning quality to target confidence; the default value is 2. (Quality item) The sample weights combine target confidence and positioning accuracy information. The calculation is as follows:

[0051] in, For the first A genuine label for the proposal, Indicates the background category. The hyperparameter controlling the intensity of exponential scaling, used to adjust the difference in loss weights between high-quality and low-quality samples, is set to 1.5. The final QARL loss is expressed as:

[0052] in, This indicates the query perception recognition loss. Represents cross-entropy loss, This represents the predicted probability distribution.

[0053] In this embodiment S4, during the training phase, a dual auxiliary supervision head is introduced to provide additional dense supervision signals to the sparse directional proposal network and query-aware refinement head through a dense feature supervision path and an auxiliary sample injection path. This embodiment proposes a Dual Auxiliary Supervision Head (DASH) as a training module to address the common problems of sample scarcity and insufficient supervision in query-based detectors. The Dual Auxiliary Supervision Head provides additional supervision through dual supervision paths, including a dense feature supervision path and an auxiliary sample injection path. The dual-auxiliary supervision head constructs a dense supervision carrier through a decoupled layer attention module, which fuses multi-scale features from the feature pyramid network. High-level features of the Transformer encoder ,in This indicates the number of layers in the feature pyramid network. Specifically, after adjusting the scale of the encoder features to match the scale of each layer in the feature pyramid network, we can obtain... Aligned feature maps ,in , For batch size, The number of channels in the feature map. and Given the feature map height and width, these features are concatenated along the channel dimension. Apply global average pooling, through Convolutional prediction layer attention weights Based on this, through a Convolution generates fused features ; Dense feature supervision path in An anchorless, classification-agnostic dense head is attached to the top, and the focus loss and rotation IoU loss are combined for optimization. The label assignment follows an adaptive training sample selection strategy. This path provides rich gradients for the feature pyramid network and Transformer encoder by supervising the prediction of all spatial locations, thus alleviating the lack of feature-level supervision caused by sparse query matching. The auxiliary sample injection path reuses dense predictions as candidate samples and converts them into auxiliary query groups through the encoder query selection mechanism. These auxiliary queries are injected into the SOPN decoder to generate auxiliary proposals and query features, which are forward propagated to QARH during training. This path compensates for the limitation of the number of positive samples in SOPN sparse predictions and improves the robustness of fine-grained recognition. The injected samples are optimized using the same objective as the main branch, including QARL loss and bounding box regression loss. DASH introduces three auxiliary supervision signals: 1) Auxiliary encoder loss: the dense prediction loss of the DASH module itself, which directly supervises the encoder and the feature pyramid network; 2) Auxiliary decoder loss: the loss of auxiliary query in SOPN, which strengthens the coarse-grained decoder; 3) Auxiliary recognition loss: the loss of auxiliary proposal in QARH, which provides dense supervision for the fine-grained head.

[0054] In this embodiment, S5, during the inference phase, a two-stage consensus scoring mechanism is adopted, which integrates the coarse-grained target confidence and the fine-grained classification probability to calculate the final consensus score of each proposal, and obtains the fine-grained target detection result based on the final consensus score and the refined bounding box. This embodiment proposes a dual-stage consensus scoring mechanism (DSCS) to bridge the sparse and dense paradigms and achieve end-to-end inference without nonmaximum suppression. Unlike traditional methods that rely on nonmaximum suppression for deduplication, DSCS integrates the target confidence in the coarse-grained stage and the classification probability in the fine-grained stage to generate a final prediction that has both high positioning reliability and accurate semantics. set up The first prediction for SOPN The prospect probability of each proposal, i.e., the coarse-grained target confidence. ,set up Here is the fine-grained classification probability distribution predicted by QARH, where A positive integer representing the number of fine-grained categories; Indicates the background category and its corresponding index. Extract the foreground subvector from the QARH output:

[0055] in, This represents a probability subvector representing the foreground category. Indicates the first The probability of each fine-grained class, for any fine-grained class Consensus Score Calculate the geometric mean of stage-specific confidence levels:

[0056] in, As a balancing parameter, used to adjust the contribution ratio of the sparse paradigm and the dense paradigm, it is preferably set to 0.7. When the value is close to 1, the role of coarse-grained target confidence is emphasized; when... When the probability approaches 0, the role of fine-grained classification is emphasized. Since SOPN is a dedicated target with a binary classifier, its foreground probability inversion yields the background score.

[0057] The scores are aggregated into a vector. After normalization, the final probability distribution is obtained:

[0058] Among them, the denominator The sum of scores for all categories is used for probability normalization. By coordinating the predictions in both stages, DSCS ensures that the final detection result has both high localization accuracy and precise fine-grained semantics, eliminating the need for non-maximum suppression.

[0059] Furthermore, the dual-paradigm detection model constructed in this embodiment adopts an end-to-end training method, with an overall loss function. It consists of three parts, corresponding to the coarse-grained stage, the fine-grained stage, and the auxiliary supervision head:

[0060] in, The loss includes the main query, the denoised query, and the DASH auxiliary query. Focus loss is used for classification, and a combination of angle-weighted L1 loss and rotation IoU loss is used for regression. Bipartite graph matching is used for supervision. To control the fine-grained identification stage, QARL loss and regression loss are applied to the proposals generated from the main query and DASH auxiliary query. This represents the densely supervised loss specifically applied to DASH; The model was trained using the AdamW optimizer with a learning rate set to [value missing]. During training, multi-scale data augmentation strategies are employed, including random flipping and random rotation.

[0061] Detection and recognition of remote sensing aircraft targets on a fine-grained detection dataset: Performance evaluation method of remote sensing fine-grained detection using the MAR20 dataset. This dataset is currently a widely used remote sensing fine-grained aircraft detection dataset, and all objects in the dataset are annotated with directional bounding boxes according to 20 aircraft subcategories. Fine particle size test results are as follows Figure 4 And as shown in Table 1 below: Table 1

[0062] As can be seen, the dual-paradigm joint remote sensing image end-to-end fine-grained target detection method constructed in this embodiment can effectively improve the inference performance of the model in remote sensing fine-grained detection tasks. When all three core modules are used, the fine-grained detection model constructed in this embodiment improves AP by 7.13% compared with the query-based end-to-end detection baseline. Specifically, when using the sparse orientation proposal network proposed in this embodiment alone, compared to the query-based end-to-end detection baseline commonly used in existing technologies, the sparse orientation proposal network achieves global deduplication through one-to-one matching, focusing on target localization and background classification. This provides high-quality orientation region proposals for subsequent fine-grained recognition, reducing the difficulty of subsequent fine-grained recognition and resulting in a 4.68% performance improvement in AP. Building upon this, the proposed query-aware refinement head generates discriminative features for fine-grained recognition. The query-aware refinement head uses one-to-many matching to learn discriminative fine-grained features and alleviates interference between classification and regression tasks by decoupling the gated feedforward network, further improving AP by 1.81%. The dual-auxiliary supervision head enhances model training from a supervisory perspective. Through dense feature supervision and auxiliary sample injection, the dual-auxiliary supervision head addresses the problem of insufficient sparse supervision and accelerates model convergence. Based on the application of the first two strategies, the dual-auxiliary supervision head achieves a 0.64% improvement. The improved performance of AP and the visualization results also demonstrate that the dual-paradigm joint remote sensing image end-to-end fine-grained target detection method proposed in this embodiment performs better in the localization and type discrimination of fine-grained targets. Specifically, in the detection and identification of typical remote sensing targets such as aircraft, the method in this embodiment can generate more accurate bounding boxes, and the accuracy of model identification is significantly better than the benchmark method. In the detection and identification of typical small targets such as ships and vehicles, the method in this embodiment has a significantly lower false negative rate and performs better in model identification.

[0063] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0064] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images, characterized in that, Includes the following steps: S1. Input a remote sensing image and extract multi-scale feature maps through a feature extraction network; S2. Input the multi-scale feature map into the sparse orientation proposal network, and use a one-to-one sparse matching strategy to generate sparse orientation region proposals without repetition and their corresponding proposal features and coarse-grained target confidence. S3. Input the sparse directional region proposal, proposal features, and multi-scale feature map into the query-aware refinement head, and use a one-to-many dense matching strategy to perform fine-grained classification and bounding box refinement on each proposal to obtain the fine-grained classification probability and the refined bounding box. S4. During the training phase, a dual auxiliary supervision head is introduced to provide additional dense supervision signals to the sparse orientation proposal network and query-aware refinement head through a dense feature supervision path and an auxiliary sample injection path. S5. In the inference phase, a two-stage consensus scoring mechanism is adopted, which integrates the coarse-grained target confidence and fine-grained classification probability to calculate the final consensus score of each proposal, and obtains the fine-grained target detection result based on the final consensus score and the refined bounding box.

2. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, In S1, the feature extraction network includes a backbone network and a feature pyramid network; The backbone network is used to extract basic features from the input image, and the feature pyramid network is used to perform multi-scale processing on the basic features to generate multi-scale feature maps.

3. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, S2 includes: The multi-scale feature map is input into the Transformer encoder for feature enhancement to obtain the encoded features; Initialize a set of learnable queries, input the queries and encoded features into the Transformer decoder for interaction, and output the query features optimized by the decoder. The optimized query features are input into the class-independent detection head, and the directional region proposal parameters and coarse-grained target confidence scores corresponding to each query are output. Sparse directional region proposals are generated based on the directional region proposal parameters, and the optimized query features are used as the proposal features corresponding to each proposal.

4. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, In S2, the sparse orientation proposal network adopts a classification-agnostic training strategy, mapping all fine-grained subclasses to a single foreground class, and employs a one-to-one bipartite graph matching strategy, so that each real target is matched by only one predicted sparse orientation region proposal.

5. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, In S2, the sparse directional proposal network uses an angle-weighted regression loss function. Optimize: in, The L1 loss represents the position and size of the bounding box. L1 loss represents the angle. To balance the hyperparameters.

6. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, S3 includes: For each sparse directional region proposal, region of interest features are extracted from the multi-scale feature map through region of interest alignment, and semantic features are extracted from the proposal features through cross-attention operation; The region of interest features and semantic features are fused to obtain a unified feature representation; The unified feature representation is input into the decoupled gated feedforward network to generate task-adaptive classification and regression features; Based on the classification and regression features, the fine-grained classification probability and the refined bounding box are predicted respectively.

7. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 6, characterized in that, The decoupled gated feedforward network generates classification gated signals through a gating mechanism. and regression gating signal and representation of shared features Modulation is performed to obtain classification features. and regression features : in, To unify feature representation, σ is the activation function. , , , , For learnable projection matrices, It represents the Hadamardi (or Hadama) stack.

8. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, In step S3, the query-aware refinement head employs query-aware recognition loss. Supervision: in, Indicates sample weights, based on proposal quality. Dynamic adjustment; For cross-entropy loss, For the predicted probability distribution, These are the corresponding real tags.

9. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, In step S4, the introduction of dual auxiliary supervisory heads includes: By fusing multi-scale feature maps and encoder features through the decoupling layer attention module, fused features are obtained; A dense feature supervision path is constructed on the fused features, and prediction is performed using an anchorless dense head. Gradient feedback is provided through dense prediction loss. An auxiliary sample injection path is constructed, which converts the prediction of the dense feature supervision path into an auxiliary query group and injects it into the sparse directional proposal network and query-aware refinement head to generate additional training samples for optimization.

10. The dual-paradigm combined end-to-end fine-grained target detection method for remote sensing images according to claim 1, characterized in that, S5 includes: For the i-th proposal, obtain its coarse-grained target confidence score. and fine-grained classification probability distribution C represents the number of fine-grained categories, and bg represents the background. For each fine-grained category Calculate consensus score , For balance parameters; Calculate background score ; Aggregate all scores into a vector After normalization, the final probability distribution is obtained. ; The category with the highest score is selected as the prediction result, and combined with the refined bounding box, the final fine-grained object detection result is output.

Images