A Multimodal Remote Sensing Target Detection Method Based on Uncertainty-Aware Hybrid Experts

Abstract

This invention belongs to the field of intelligent remote sensing image processing and computer vision technology, and proposes a multimodal remote sensing target detection method based on uncertainty-aware hybrid experts. A modality-specific router is constructed in the shared backbone network of a unified detection model, and independent gating networks are allocated to different modalities to decouple the decision space and eliminate cross-modal gradient competition. An uncertainty-aware training strategy is designed, using normalized entropy to quantify the routing uncertainty of feature tokens, and filtering out high-uncertainty samples through dynamic thresholds. A dual-path routing mechanism is adopted, broadcasting high-uncertainty tokens to all experts during the training phase to stabilize expert training, maintaining sparse Top-k routes for deterministic tokens, and forcibly backtracking to sparse routes during the inference phase. This invention alleviates the cross-modal interference problem in multimodal joint training, improves the model's extraction capability and routing stability, and achieves high-performance detection with zero additional computational overhead during the inference phase.

Description

Technical Field

[0001] This invention belongs to the field of intelligent processing of remote sensing images and computer vision technology, and relates to a multimodal remote sensing target detection method based on uncertainty perception hybrid expert. Background Technology

[0002] Remote sensing target detection, as a core task of remote sensing image interpretation, has significant application value in fields such as all-weather maritime monitoring, intelligent traffic inspection, disaster emergency assessment, and military reconnaissance and strike. With the rapid development of remote sensing technology, data acquisition exhibits a significant multi-source heterogeneity, encompassing various imaging modalities such as optical (RGB), infrared (IFR), and synthetic aperture radar (SAR). However, the imaging mechanisms of different modalities differ greatly: RGB images emphasize spectral reflectance characteristics, infrared images reflect thermal radiation distribution, while SAR images are based on active microwave scattering imaging. This difference in physical mechanisms leads to significant gaps in texture statistics, noise distribution, and appearance features of the same target under different modalities. Consequently, when models perform multi-modal joint learning, they are prone to encountering dilemmas such as inconsistent feature coupling, divergent optimization directions, and gradient conflicts.

[0003] Existing multimodal remote sensing detection methods still have significant limitations in addressing the aforementioned challenges. Early feature-level or decision-level fusion methods typically rely on the complementarity of features from different modalities. However, these methods often implicitly assume extremely stringent data preprocessing, requiring cross-modal data to undergo rigorous spatiotemporal registration or synchronous acquisition. In practical uncontrolled applications, due to differences in sensor perspectives, sampling frequencies, or environmental occlusion, modality loss, asynchronous acquisition, and unregistered data are commonplace, resulting in low applicability of traditional fusion methods. Recently, a unified multimodal detection framework has been introduced, attempting to maintain modality specificity through conditional computation in a shared feature space by sharing a backbone network and incorporating a sparse hybrid expert (MoE) module. However, most existing MoE architectures employ a shared router mechanism, where data from all modalities undergo expert selection through a gated network with the same set of parameters. This design forces the routing gradients of different modalities to compete for the same set of router parameters during backpropagation, compelling routers to compromise between heterogeneous data distributions. This cross-modal competition directly leads to the instability of expert selection, causing serious cross-modal interference and weakening the professional capabilities of expert networks.

[0004] Remote sensing images typically feature large fields of view, complex backgrounds, and densely packed small targets. These factors result in a large number of fuzzy grid tokens in the feature space, meaning the model has extremely high uncertainty regarding their classification or localization. Under traditional Top-k hard routing constraints, these highly uncertain tokens are forcibly assigned to a few experts. This forces the selected experts to fit background noise or artifacts, deviating from their original feature domains; furthermore, other experts fail to participate in learning difficult samples, leading to an imbalance in expert utilization. Ultimately, this manifests as model training convergence oscillations, reduced effective capacity, and difficulty in achieving high-precision detection in complex scenes. Therefore, there is an urgent need for an innovative, unified multimodal remote sensing target detection method that can structurally eliminate cross-modal gradient competition caused by shared routing, adaptively and dynamically process highly uncertain samples, and improve training stability without increasing computational overhead during the inference phase. Summary of the Invention

[0005] To address the technical problems in existing unified multimodal remote sensing detection models, such as cross-modal gradient competition caused by shared routers, and routing decision oscillations and low expert utilization due to high-uncertainty samples (e.g., complex backgrounds, small targets), this invention aims to provide a multimodal remote sensing target detection method based on uncertainty-aware hybrid expert (UA-MoE). This method decouples the gating decision space from the quantified routing uncertainty, eliminating cross-modal interference while achieving adaptive reinforcement learning for difficult samples.

[0006] The technical solution adopted in this invention is as follows:

[0007] A multimodal remote sensing target detection method based on uncertainty-aware hybrid experts includes the following steps:

[0008] (1) Construct a unified multimodal detection model based on a shared feature extraction backbone network;

[0009] The shared feature extraction backbone network has a hierarchical architecture, which includes multiple feature extraction stages connected in series along the feature forward propagation direction. Each feature extraction stage is arranged in order from the input end to the output end. The resolution of the feature map decreases step by step, and the number of feature channels increases step by step. Each feature extraction stage is composed of several ConvNext Blocks connected in series.

[0010] Each ConvNext Block internally consists of a main path and residual connections; the main path sequentially includes depthwise separable convolutions, layer normalization, and 1 / 2... 1. Convolutional layer; Residual connection is used to add the input features of ConvNext Block to the output features of the main path element by element, and the result of the addition is used as the output feature of ConvNext Block;

[0011] In the last two feature extraction stages of the shared feature extraction backbone network, the 1 in ConvNextBlock of the even-numbered layers is... 1. The convolutional layer is replaced with an uncertainty-aware hybrid expert module (UA-MoE). In the UA-MoE, the output of the normalization layer in the main path of the ConvNext Block is spatially divided into grid-level tokens, which serve as the basic unit for routing selection within the UA-MoE. The replaced UA-MoE maintains the same functionality as the original 1. The convolutional layers have the same number of input and output channels, and the ConvNext Block maintains the original residual connection structure; the shared feature extraction backbone network ultimately outputs multi-scale feature maps.

[0012] (2) Each uncertainty-aware hybrid expert module (UA-MoE) integrates a mode-specific routing network group and an expert pool;

[0013] The modality-specific routing network group is constructed as follows:

[0014] In each uncertainty-aware hybrid expert module (UA-MoE), independent routing networks are constructed for synthetic aperture radar (SAR), optical (RGB), and infrared (IFR) modes, and each routing network has independent routing parameters. During forward propagation, the corresponding routing network is activated according to the modal attributes of the input data, the gating gradients of different modes are physically isolated, and the routing parameters of the currently activated mode are updated only.

[0015] Settings include The expert pool consists of several parallel expert subnetworks, each a feedforward neural network with independent weight parameters, used to perform nonlinear feature transformation on the input grid-level tokens and output expert feature representations. The modality-specific routing networks share this expert pool, and the parameters of the expert pool are jointly updated by data from SAR, RGB, and IFR modes.

[0016] (3) Forward propagation and uncertainty quantification of the uncertainty-aware hybrid expert module UA-MoE:

[0017] (3.1) For grid-level tokens with different modal inputs, activate the routing network corresponding to that modality and calculate its pair. The routing score for each expert is calculated using the following formula:

[0018]

[0019] in, Indicates spatial location as Grid-level tokens Representing modes The corresponding routing network, Represents the routing network in the 1st The scores from the experts;

[0020] (3.2) Quantify the routing uncertainty of each grid-level token based on the routing probability distribution;

[0021] a. Obtain the route probability distribution by applying temperature scaling and Softmax normalization to the route scores. ;

[0022] b. Calculate the normalized entropy of the grid-level token as a quantification of uncertainty:

[0023]

[0024] in, Let n be the routing probability distribution for the nth expert. A higher value indicates greater uncertainty in the decision-making process;

[0025] (4) Uncertainty-aware dynamic routing strategy during the training phase:

[0026] (4.1) Determine the uncertainty screening threshold based on the intramodal dynamic statistical strategy: In the current batch of data, the uncertainty screening threshold is determined by the dynamic statistical strategy within the modal. The normalized entropy values ​​of all grid-level tokens; arrange the normalized entropy values ​​in descending order, and select... The normalized entropy value of the % quantile is used as the dynamic uncertainty threshold for this mode. ;

[0027] (4.2) Grid-level Token set partitioning: based on dynamic threshold The intramodal grid-level tokens are divided into an indeterminate token set. and determine the token set :

[0028]

[0029]

[0030] (4.3) For a deterministic set of tokens, implement a sparse Top-k routing strategy: For a deterministic set of tokens For each grid-level token, only the one with the highest value in its route probability distribution is activated. A network of experts ; to be activated The output features of each expert subnetwork are weighted and summed to obtain the output features of the grid-level token.

[0031] (4.4) For an uncertain set of tokens, implement a full-expert routing broadcast strategy: For an uncertain set of tokens Each grid-level token in the pool activates all tokens in the shared expert pool. A network of experts; encompassing all The output features of each expert subnetwork are weighted and summed to obtain the output features of the grid-level token.

[0032] (5) Construct a multi-task detection head and differentiated loss supervision;

[0033] A multi-task detection head is connected to the output of the shared feature extraction backbone network to receive multi-scale feature maps and generate target detection results. The multi-task detection head selects the corresponding detection branch according to the imaging characteristics of the input modality: for SAR modality, a single-stage dense detection head is constructed; during training supervision, the weighted sum of quality focal loss (QFL), distributed focal loss (DFL), and generalized IoU loss (GIoU) ​​is used as the loss function; for optical (RGB) and infrared (IFR) modality data, a two-stage detection head including a region generation network (RPN) and a region of interest (RoI) is constructed; during training supervision, the sum of the classification cross-entropy loss of RPN and RoI and the bounding box regression loss is used as the loss function.

[0034] (6) Reasoning stage:

[0035] Remove the all-expert routing broadcast strategy and uncertainty quantization, and enforce a sparse Top-k routing strategy on all modal grid-level tokens, activating only the one with the highest probability. Each expert subnetwork performs the calculations.

[0036] The beneficial effects of this invention are:

[0037] 1. This invention effectively isolates cross-modal routing gradient competition by setting up independent routers for different remote sensing modalities and sharing an expert pool, reduces the instability of expert selection in the trade-off between heterogeneous modalities, structurally suppresses cross-modal interference and improves the generalization ability of the unified model;

[0038] 2. This invention identifies and broadcasts high-uncertainty samples based on normalized entropy, effectively solving the problems of uneven expert load and unstable training in traditional MoE when dealing with background noise and small targets. Entropy variation experiments show that this strategy reduces the overall routing entropy of the model by 67%. In the SOI-Det benchmark, the overall mAP reaches 50.66%, an improvement of 0.46% compared to the baseline SM3Det, ​​and a 2.01% improvement on the noisy SARDet-100K dataset, demonstrating the superiority of this method for difficult modes.

[0039] 3. The dual-path mechanism of this invention introduces computational overhead only during the training phase, and reverts to the efficient Top-k mode during the inference phase. Compared with the baseline SM3Det, ​​this invention reduces the number of parameters by 1.12% and FLOPs by 3.08% on the ConvNext-T backbone, and improves the inference speed (FPS) on SARDEt-100K by 15.51%, effectively solving the problem of difficult deployment of complex models;

[0040] 4. The present invention demonstrates consistent performance improvement across backbone networks of different sizes (ConvNext-T / S / B), and outperforms existing single-modal training, simple joint training and other unified detection frameworks, proving the versatility of the method in multimodal remote sensing detection tasks. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the overall framework of the present invention. Detailed Implementation

[0042] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0043] Example

[0044] In this embodiment, the SOI-Det benchmark dataset is constructed using three modality datasets. These datasets include a SAR target detection dataset, an RGB target detection dataset, and an IFR target detection dataset. Each dataset contains a certain number of image samples, divided into training and test sets.

[0045] During model training, the AdamW optimizer updates parameters with a batch size of 4 and an initial learning rate of 0.0001. During data sampling, each batch is constructed by uniformly sampling from three modality datasets at a 2:1:1 ratio. A complete loop through all datasets is ensured approximately every 20,000 iterations. All experiments are performed on a single machine. Performed on an NVIDIA RTX 3090 GPU.

[0046] A multimodal remote sensing target detection method based on uncertainty-aware hybrid experts, such as Figure 1 As shown, the steps are as follows:

[0047] Step 1: Construct a unified multimodal detection model based on a shared feature extraction backbone network;

[0048] (1.1) The lightweight and efficient ConvNext-T is used as the shared feature extraction backbone network. This shared feature extraction backbone network consists of four stages (Stage 1 to Stage 4) from shallow to deep, with the number of channels of the output feature map of each stage configured as [96, 192, 384, 768]. The input image is uniformly adjusted to 800 before entering the network. 800 pixels in size. Each stage consists of several concatenated ConvNext Blocks. Each block contains 7... 7. Depthwise Convolution, Layer Normalization, and 1 One convolutional layer (Pointwise Conv) is used, and the output is connected through residuals, which maintains the local inductive bias of convolution while ensuring semantic abstraction.

[0049] (1.2) In order to introduce sparse conditional computation and alleviate cross-modal interference in the shared feature extraction backbone network, in the deep stages of the shared feature extraction backbone network, namely the 3rd and 4th stages, the 1 in the ConvNext Block of all even-numbered layers (EvenIndex) is... One convolutional layer is replaced with an uncertainty-aware hybrid expert module. The replaced uncertainty-aware hybrid expert module maintains the same functionality as the original layer. With consistent input and output channel counts across all convolutional layers, the ConvNext Block maintains the original residual connection structure, thereby enabling the model to dynamically route without altering the backbone topology.

[0050] (1.3) The shared feature extraction backbone network ultimately outputs a multi-scale feature map, which is then divided into grid-level tokens as basic routing units. Differentiated detection loss functions are set for the imaging characteristics of different modalities: for the SAR modality, a weighted sum of quality focal loss (QFL), distributed focal loss (DFL), and generalized IoU loss (GIoU) ​​is used for supervision; for the optical RGB and infrared IFR modalities, classification cross-entropy and bounding box regression loss of the region generation network (RPN) and region of interest (RoI) are used for supervision.

[0051] Step 2: Each uncertainty-aware hybrid expert module integrates a modality-specific routing network group and an expert pool;

[0052] (2.1) In each uncertainty-aware hybrid expert module, three modality-specific routing networks are constructed in parallel, corresponding to SAR routers, RGB routers, and IFR routers, respectively. During forward propagation, the uncertainty-aware hybrid expert module reads the modality ID of the input image and automatically activates the corresponding single router, physically isolating the gradient backpropagation paths of different modalities;

[0053] (2.2) Set the number of expert sub-networks in the expert pool The expert subnetwork is a feedforward neural network with independent weight parameters, used to perform nonlinear transformations on the input grid-level tokens. All modal routing networks share this expert pool. The initial weights of the experts are directly copied from the corresponding 1 in the pre-trained ConvNext-T model. 1. Convolutional layer weights;

[0054] (2.3) For the grid-level token input to modality m, activate the routing network specific to that modality and calculate its routing scores (Logits) for the 8 experts. The formula for calculating the routing scores is as follows:

[0055]

[0056] in, Indicates spatial location as Grid-level tokens Representing modes The corresponding routing network, Represents the routing network in the 1st The scores from the experts;

[0057] Step 3: Quantify the routing uncertainty of each grid-level token based on the routing probability distribution;

[0058] (3.1) Temperature scaling and softmax normalization are applied to the route scores to obtain the route probability distribution. The formula for calculating the route probability distribution is as follows:

[0059]

[0060] Where t represents the temperature scaling factor;

[0061] (3.2) Normalized entropy is used as the quantification indicator of uncertainty for grid-level tokens. A larger normalized entropy value indicates a more uniform distribution of expert allocation of the grid-level token by the router, meaning higher decision uncertainty. The formula for calculating normalized entropy is as follows:

[0062]

[0063] (3.3) Determine the uncertainty screening threshold based on the intramodal dynamic statistical strategy: In the current batch of data, the uncertainty screening threshold is determined by the dynamic statistical strategy within the modal. The normalized entropy values ​​of all grid-level tokens are calculated; these normalized entropy values ​​are then arranged in descending order, and the normalized entropy value at the 10th percentile is selected as the dynamic uncertainty threshold for this mode. Based on this dynamic uncertainty threshold, the tokens within the modality are divided into a 10% uncertain token set and a 90% deterministic token set. The calculation formula for this set division is as follows:

[0064]

[0065]

[0066] (3.4) For a deterministic set of tokens, implement a sparse Top-k routing strategy: For a deterministic set of tokens Each grid-level token in the array, only Two expert subnetworks perform the calculations; the ones to be activated The output features of each expert subnetwork are weighted and summed to obtain the output features of the grid-level token; for the uncertain token set, a full expert route broadcast strategy is executed: for the uncertain token set Each grid-level token in the pool activates all tokens in the shared expert pool. Eight expert subnetworks are formed; the output features of all eight expert subnetworks are weighted and summed to obtain the output features of the grid-level token.

[0067] Step 4: Joint optimization and reasoning;

[0068] (4.1) An asymmetric strategy for training and inference is adopted. During the training phase, a full expert routing broadcast strategy is implemented to handle samples with high uncertainty in order to fully train the expert sub-network. For deterministic samples, Top-2 sparse routing is implemented. During the inference phase, Top-2 sparse routing is enforced for all tokens to ensure the efficiency of inference.

[0069] (4.2) During training, multi-task joint loss calculation is adopted for different modalities. The multi-task detection head selects the corresponding detection branch according to the imaging characteristics of the input modality: For SAR modality, a single-stage dense detection head is constructed; during training supervision, the weighted sum of quality focal loss (QFL), distributed focal loss (DFL), and generalized IoU loss (GIoU) ​​is used as the loss function; for optical (RGB) and infrared (IFR) modal data, a two-stage detection head including region generation network (RPN) and region of interest (RoI) is constructed; during training supervision, the sum of classification cross-entropy loss of RPN and RoI and bounding box regression loss is used as the loss function. The formula for calculating the total loss function is as follows:

[0070]

[0071] Where T = 3 is the total number of tasks. This represents the specific loss for the t-th task.

[0072] This invention validated its technical advantages in the 3-modal SOI-Det benchmark test: 1) In terms of detection accuracy, the overall average accuracy (mAP) reached 50.66%, an improvement of 0.46% compared to the baseline model SM3Det (50.20%), especially in the SAR mode with severe noise interference, the mAP reached 62.65%, an improvement of 2.01% compared to the baseline; 2) In terms of computational efficiency, thanks to the asymmetric routing strategy, the model computational cost (FLOPs) was reduced to 472G, a reduction of 3.08% compared to the baseline (487G); 3) In terms of parameter efficiency, the mode-specific router design compressed the model parameter count to 176M, a reduction of 1.12% compared to the baseline (178M); 4) In terms of inference speed, the inference speed (FPS) on the SARDet-100K dataset reached 25.54, an improvement of 15.51% compared to the baseline. Experimental results show that this invention maintains performance advantages under various model scales, providing an efficient solution for multimodal remote sensing target detection.

[0073] This invention achieves effective suppression of cross-modal interference and optimized allocation of expert resources within a unified framework through modality-specific routers and an uncertainty-aware training strategy. Based on the decoupling mechanism of modality-specific routers, this invention physically isolates the gating decision spaces of different modalities, eliminating gradient competition problems caused by differences in imaging mechanisms in shared routing schemes. Compared to shared routing methods, it exhibits stronger modality feature alignment capabilities and routing stability. Based on the uncertainty-aware mechanism of normalized entropy, this invention can accurately identify routing fuzzy tokens (such as background clutter and small targets) and broadcast these high-uncertainty samples to the full expert pool for joint training. This avoids expert overfitting and uneven load caused by Top-k sparse constraints and significantly improves the modeling ability for difficult samples through full expert collaboration. Finally, the dual-path routing strategy of this invention achieves asymmetric decoupling between training and inference. During the inference phase, it completely removes the uncertainty calculation branch and the full broadcast path, retaining only the efficient Top-k sparse activation, achieving a significant performance leap without increasing any additional computational overhead.

Claims

1. A multi-modal remote sensing target detection method based on uncertainty-aware hybrid experts, characterized in that, The method comprises the following steps: Step 1, constructing a unified multi-modal detection model based on a shared feature extraction backbone network; The shared feature extraction backbone network is a hierarchical architecture, comprising a plurality of feature extraction stages connected in series along a feature forward propagation direction, each feature extraction stage being arranged in order from an input end to an output end, the resolution of a feature map being gradually reduced, and the number of feature channels being gradually increased, each feature extraction stage being composed of a plurality of ConvNext Blocks connected in series; Step 2, a modality-specific routing network group and an expert pool are integrated in each uncertainty perception hybrid expert module; Step 3, forward propagation and uncertainty quantification of the uncertainty perception hybrid expert module; Step 4, uncertainty perception dynamic routing strategy in the training stage; Step 5, constructing a multi-task detection head and a differential loss supervision; Step 6, inference stage; Each ConvNext Block is built with a main path and a residual connection; the main path contains depthwise separable convolutions, layer normalization and 1 1convolutional layer; the residual connection is used to add the input features of the ConvNext Block and the output features of the main path element by element, and the result is used as the output features of the ConvNext Block In the last two feature extraction stages of the shared feature extraction backbone network, the 1 1convolutional layer is replaced with an uncertainty-aware hybrid expert module; in the uncertainty-aware hybrid expert module, the output of the normalization layer in the ConvNext Block main path is divided into grid-level Tokens in the spatial dimension as the basic unit of internal routing selection of the uncertainty-aware hybrid expert module; the replaced uncertainty-aware hybrid expert module maintains the same input channel number and output channel number as the original 1 1convolutional layer; the ConvNext Block maintains the original residual connection structure; the shared feature extraction backbone network finally outputs multi-scale feature maps; The modality-specific routing network group is constructed in the following manner: In each uncertainty perception hybrid expert module, independent routing networks are constructed for the SAR modality, the RGB modality and the IFR modality, and each routing network has independent routing parameters; in the forward propagation, the corresponding routing network is activated according to the modality attribute of the input data, the gradient of different modalities is physically isolated, and only the routing parameters of the current activated modality are updated; The method comprises the steps of: The expert pool comprises a plurality of parallel expert sub-networks, the expert sub-networks being feedforward neural networks with independent weight parameters, for performing nonlinear feature transformation on input grid-level tokens and outputting expert feature representations; a modal-specific routing network shares the expert pool, and parameters of the expert pool are jointly updated by data of SAR modal, RGB modal and IFR modal.

2. The multi-modal remote sensing target detection method based on uncertainty-aware hybrid experts according to claim 1, wherein, The specific implementation process of step 3 is as follows: Step 3.1, for the grid-level Token of different modal input, activate the routing network corresponding to the modal, calculate its routing score to the expert, the calculation formula of the routing score is as follows: wherein, represents a grid-level Token with spatial position , represents a modality corresponding routing network, represents the score of the routing network on the th expert; Step 3.2, quantifying the routing uncertainty of each grid-level Token based on the routing probability distribution; Step 3.1.

1. Temperature scaling and Softmax normalization of routing scores to get routing probability distribution ; Step 3.1.2, calculating the normalized entropy of the grid-level Token as an uncertainty quantification index: wherein, the routing probability distribution for the nth expert, The larger the value, the higher the decision uncertainty.

3. The multi-modal remote sensing target detection method based on uncertainty-aware hybrid experts according to claim 2, characterized in that, The specific implementation process of step 4 is as follows: Step 4.1, determining the uncertainty screening threshold based on the dynamic statistical strategy within the modal: statistically determining the normalized entropy value of all grid-level Tokens belonging to the modal in the current batch data; arranging the normalized entropy values in descending order, and selecting the normalized entropy value at the quantile point as the dynamic uncertainty threshold of the modal ;​​ Step 4.2, Grid-level Token Set Partitioning: Based on Dynamic Threshold Intra-Modal Grid-level Token Partitioning into Uncertain Token Set and Determined Token Set : Step 4.3, for the deterministic Token set, perform the sparse Top-k routing strategy: for each grid-level Token in the deterministic Token set , only expert sub-networks, ; weight-sum the output features of the activated expert sub-networks to obtain the output feature of the grid-level Token; Step 4.4, for the uncertainty Token set, perform a full expert routing broadcast strategy: for each grid-level Token in the uncertainty Token set activate all expert subnetworks in the shared expert pool; weight sum the output features of all expert subnetworks to obtain the output features of the grid-level Token.

4. The multi-modal remote sensing target detection method based on uncertainty-aware hybrid experts according to claim 3, characterized in that, The specific implementation process of step 5 is as follows: A multi-task detection head is connected to the output end of the shared feature extraction backbone network, which is used to receive multi-scale feature maps and generate target detection results; the multi-task detection head selects the corresponding detection branch according to the imaging characteristics of the input modality: for the SAR modality, a single-stage dense detection head is constructed; in the training supervision, the weighted sum of quality focal loss, distribution focal loss and generalized IoU loss is used as the loss function; for the RGB modality and the IFR modality, a two-stage detection head containing a region generation network and a region of interest is constructed; in the training supervision, the sum of the classification cross-entropy loss of RPN and RoI and the bounding box regression loss is used as the loss function.

5. The multi-modal remote sensing target detection method based on uncertainty-aware hybrid experts according to claim 4, characterized in that, The specific implementation process of step 6 is as follows: Remove full-expert routing broadcast strategy and uncertainty quantification, enforce sparse Top-k routing strategy for grid-level Tokens of all modalities, only activate the top-k expert subnetworks with the highest probabilities for computation.

Images