Multi-scale target detection method and system based on hybrid expert and density guidance

Abstract

The invention discloses a multi-scale target detection method and system based on hybrid expert and density guidance. The method comprises the following steps: extracting feature representations of a plurality of heterogeneous experts from input data, and dynamically selecting and activating the experts by adopting a self-adaptive routing mechanism; feature alignment and fusion are carried out on heterogeneous expert output, and multi-scale feature integration is realized through scale normalization, a dynamic gating mechanism and a multi-head attention mechanism; predicting a target density map by using a density-guided dynamic query (DGQ) module, and dynamically adjusting the query quantity by adopting a hierarchical query strategy according to the estimated target quantity; and inputting the dynamic query into a DETR decoder for processing, and outputting a final target detection result through a multi-layer self-attention and cross attention mechanism in cooperation with a deep supervision auxiliary prediction head. The method effectively solves the problem of scale competition in a scene where a tiny target and a large target coexist in a remote sensing image, achieves the balance detection of a multi-scale target, and remarkably improves the detection performance and the cross-domain generalization capability.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing, and specifically to a multi-scale target detection method and system based on a hybrid expert architecture and density-guided mechanism. Background Technology

[0002] With the rapid development of UAV and satellite imaging technologies, remote sensing target detection has become a key technology in civilian fields such as environmental monitoring, urban construction, disaster emergency response, and security monitoring. However, current detection algorithms mainly rely on feature extraction methods optimized for a single scale. This leads to a sharp decline in detection performance when encountering complex scenarios where multiple scale targets coexist, such as the simultaneous presence of small vehicles and large buildings, or small pedestrians and large bridges, fundamentally limiting the practical application of remote sensing detection systems.

[0003] To achieve balanced detection in multi-scale scenes, researchers have begun exploring multi-scale feature fusion methods. Specifically, small object detection, through aggressive feature enhancement and high-resolution processing, can stably capture fine-grained details, effectively compensating for the shortcomings of general-purpose detectors. Therefore, efficiently integrating complementary information from small object optimization and general-purpose object optimization can significantly enhance the robustness of the model and contribute to balanced multi-scale object detection in remote sensing scenes. However, the performance trade-off between small object detection methods and general-purpose object detection methods severely degrades the performance of traditional multi-scale detection methods. This problem is particularly challenging in scenes with dense small objects. In such cases, the excessive localization of features in small object detection methods leads to the loss of global semantic information, thereby reducing the detection accuracy of medium and large objects. Although general-purpose object detectors can provide supplementary cues, they cannot completely compensate for the feature loss or information loss in small object detection, which increases the difficulty of multi-scale feature fusion and further weakens the effectiveness of cross-scale feature fusion. Therefore, traditional multi-scale object detection methods, such as multi-scale fusion based on feature pyramids, small object detection based on context enhancement, and density-aware detection achieved through adaptive query refinement, still suffer from significant performance degradation in challenging multi-scale coexistence scenes.

[0004] To address these challenges, recent research has improved feature consistency and detection performance by introducing density estimation modules to predict and adjust query assignments. However, this heavily relies on choosing a single optimization objective (small or general) as a reference and fails to fully utilize complementary information from different scales. Some methods, such as those using context enhancement for small object detection, cross-scale interactive attention networks for multi-scale detection, and dynamic query assignment for dense scenes, attempt to achieve multi-scale fusion through weighted summation based on feature connectivity or scale-aware importance scores. However, such simple fusion strategies cannot effectively utilize the rich complementary information between different scales.

[0005] Recent methods, such as cross-scale fusion Transformers for multi-scale object detection, unified Transformer frameworks for multi-scale detection, and adaptive routing mechanisms for multi-scale object detection, employ Transformers to achieve interactive feature fusion through cross-scale attention mechanisms. However, these methods overlook several key issues in scenarios where different scales coexist. In conditions with densely packed small targets, small target detection methods suffer from severe feature over-localization, making the extraction of global semantic information nearly impossible. In cases where small and large targets are mixed, different regions exhibit inconsistent scale features; some regions may contain rich details of small targets, while others are entirely dominated by large targets, posing significant challenges to feature extraction and fusion. Even in environments with relatively uniform scale distribution, the coexistence of small targets being suppressed by large targets and large targets being weakened by small target optimization methods further complicates the problem. This scale variation leads to highly unreliable and incomplete feature extraction, severely impacting the quality of fused features generated by baseline methods. Under these complex multi-scale coexistence conditions, inaccurate feature extraction introduces significant noise and inconsistencies, making it extremely challenging to achieve effective multi-scale fusion to adapt to targets of different scales and scenarios with varying densities.

[0006] Furthermore, while existing large-scale visual foundation models have achieved significant success through large-scale self-supervised learning, these general-domain models perform poorly when generalizing to remote sensing scenarios because the target scale and density distribution are fundamentally different from those in natural images. Although large-scale detectors for remote sensing exist, these methods lack explicit mechanisms to balance detection performance in the extreme scale variations unique to aerial images. While sparse expert architectures offer computational efficiency by activating subsets of parameters based on input features, existing methods struggle to handle scale competition issues, where large targets suppress small targets. The routing strategies in current architectures do not explicitly consider spatial feature variations in remote sensing scenarios (different platforms such as satellites, aerial photography, and drones), limiting cross-domain robustness.

[0007] Despite their success in general vision tasks, there remains a critical gap in the application of large-scale models for multi-scale target detection in remote sensing. Existing methods struggle to handle scale competition and density imbalance, where large targets suppress the detection of small targets. Furthermore, existing methods lack a comprehensive approach that integrates large-scale architectures with scale-adaptive routing and density-aware mechanisms to achieve balanced detection across targets from small to large. Summary of the Invention

[0008] This invention solves the above-mentioned technical problems through the following technical means: a multi-scale target detection method based on hybrid expert and density-guided methods, comprising:

[0009] This invention solves the above-mentioned technical problems through the following technical means: a multi-scale target detection method based on hybrid expert and density-guided methods, comprising:

[0010] S1. Extract feature representations from multiple heterogeneous experts from the input data;

[0011] S2. The route-enhanced hybrid attention (REM) module is used to perform feature alignment and fusion of heterogeneous expert outputs;

[0012] S3. Use the Density Guided Dynamic Query (DGQ) module to predict the target density map and dynamically adjust the query allocation;

[0013] S4. Input the dynamic query into the DETR decoder for processing and output the final target detection result.

[0014] As a further optimized technical solution, step S1 specifically includes:

[0015] The system is configured with three heterogeneous expert backbone networks: the first type of expert (ResNet) adopts a residual convolutional neural network architecture, which includes four residual stages and outputs multi-scale feature maps with the spatial size halved layer by layer and the number of channels doubled layer by layer, making it particularly suitable for capturing the local texture of small targets; the second type of expert (ViT) adopts a visual Transformer architecture, which segments the input image into 16×16 pixel image blocks and outputs global sequence features through a 12-layer encoder, effectively modeling long-distance dependencies and suitable for semantic understanding of large targets; the third type of expert (Swin) adopts a hierarchical window attention network, which includes four stages. Each stage calculates self-attention within a 7×7 local window and expands the receptive field through downsampling, combining local detail and global semantic capabilities.

[0016] The basic backbone network adopts the ResNet-50 architecture, containing four residual stages with output channels of 256, 512, 1024, and 2048 respectively. The final output feature M has dimensions of 2048×H / 32×W / 32. The routing network consists of two fully connected layers. The first layer reduces the 2048-dimensional feature to 512 dimensions and processes it using the ReLU activation function. The second layer maps the 512-dimensional feature to E dimensions (E being the total number of configured experts), and outputs a routing score vector [s1, s2, ..., s] after Softmax normalization. E ], where Σs i =1 and s i ∈[0,1].

[0017] The adaptive routing mechanism dynamically selects the active expert by calculating the activation score of each expert. Specifically, the routing score is calculated using the following formula:

[0018] [s1,s2,…,s E ]=Softmax(FC(Flatten(GAP(M))))

[0019] Where M represents the feature representation extracted by the basic backbone network, GAP(·) represents the global average pooling operation for dimensionality reduction, Flatten(·) represents the flattening operation to convert two-dimensional features into one-dimensional vectors, FC(·) represents the fully connected layer mapped to the dimension corresponding to the number of experts E, Softmax(·) represents the normalization function, and s e ∈[0,1] represents the routing weight of the e-th expert, indicating the importance of that expert to the current input.

[0020] Activate expert collection Determined through a category-aware Top-1 selection mechanism:

[0021]

[0022] in, Indicates the set of experts that are active. This indicates the category to which expert e belongs. Indicated in each category The top-1 expert with the highest score is selected. This top-1 selection strategy by category ensures diversity of expert types while maintaining computational efficiency and preventing any expert type from being completely suppressed.

[0023] The key difference between the dynamic activation mechanism of this invention and traditional hybrid expert systems lies in the fact that traditional methods typically activate all experts or use a fixed activation pattern, leading to a waste of computational resources. In contrast, this invention dynamically determines which experts to activate based on the specific features of each input image, achieving "on-demand activation." This mechanism is particularly suitable for remote sensing image scenarios because the scale distribution varies significantly across different scenarios: scenarios with purely small targets primarily activate convolutional experts, while complex multi-scale scenarios activate multiple types of experts working collaboratively.

[0024] To further improve inference efficiency, this invention employs a parallel computing scheduling mechanism to achieve parallel execution of activated experts. It is important to note that parallel execution is limited to dynamically activated experts; inactive experts do not participate in computation, and their network parameters and intermediate activation values ​​do not need to be loaded into memory. Specifically, each activated expert is assigned to an independent computation stream, allowing forward propagation of different types of experts to occur simultaneously. A stream synchronization mechanism ensures that the features of all activated experts are available before subsequent fusion operations. This design achieves "on-demand allocation of computing resources," maximizing resource utilization efficiency while ensuring detection performance.

[0025] As a further optimized technical solution, step S2 specifically includes:

[0026] The core function of the REM module is to address the inconsistency in dimensionality of heterogeneous expert outputs and to perform effective multi-scale feature fusion through a hybrid attention mechanism. For the feature maps output by convolutional experts, a Feature Pyramid Network (FPN) is used for upsampling and alignment. For the feature sequences output by attention experts, a Simple Feature Pyramid (SFP) is used to transform them into a unified feature map representation.

[0027] After feature alignment, REM employs a hierarchical fusion mechanism for multi-scale feature integration. First, the features of each activation expert are scale-normalized:

[0028]

[0029] in, This represents the original characteristics of expert e at pyramid level l, α e It is a learnable, expert-specific calibration factor used to adjust for amplitude differences between different expert architectures, with each activated expert corresponding to an independent α. e The initial value is set to 1.0, and it is automatically learned through backpropagation during training to eliminate feature amplitude bias caused by differences in network architecture. Represents the normalized features. This represents the set of experts that are active.

[0030] Then, a dynamic gating mechanism is used to calculate scale-specific fusion weights. The gating network first aggregates expert features through global average pooling:

[0031]

[0032] Where Concat(·) represents the feature concatenation operation, g l This represents the aggregated feature vector of pyramid level l. Next, the level-specific gating weights are calculated:

[0033] w l =Softmax(W l ·g l +b l )

[0034] Among them, W l and b l It is the learnable parameter matrix and bias vector of level l, w l w represents the gating weight vector of this layer. l,e This represents the weight component of expert e at level l. The specific implementation of the dynamic gating mechanism is as follows: For each level l of the pyramid, the normalized features of all activated experts at that level are concatenated along the channel dimension to obtain a dimension of (256×|A|)×H. l ×W l The concatenated features are obtained, where |A| represents the number of activated experts. Global average pooling is performed on the concatenated features and flattened to obtain a 256×|A| dimensional vector g. g is then processed by a gated network, which contains a hidden layer (256×|A|→256 dimensions, ReLU activation) and an output layer (256→|A| dimensions). The output is normalized by Softmax to obtain the gate weights w=[w,1,...,w,|A|], satisfying Σw, e =1; The weight matrix W∈R(|A|×256) and bias vector b∈R|A| of the gated network are learned independently at each FPN layer, and are initialized using Xavier uniform distribution, without sharing parameters across layers.

[0035] At each pyramid level, the final fusion feature is achieved by combining the routing scores. e and gating weight w l,e Weighted fusion of the normalized features of activation experts:

[0036]

[0037] in, The summation symbol ∑ represents the fusion feature of level l, and it iterates through all sets of activation experts. The design leverages both the "input-expert adaptation prior" provided by routing scores and the "feature-level importance" reflected by gating weights to achieve refined fusion.

[0038] Finally, a multi-head attention mechanism is used to hierarchically integrate the fused multi-scale features:

[0039]

[0040] Where Q, K, and V represent the query, key, and value matrices, respectively, and they are the fusion features F. fused The linear projection of F, where d represents the feature dimension used to scale the attention score. final This represents the feature pyramid of the final output. The multi-head attention mechanism uses 8 attention heads with a total embedding dimension of 256 and a dimension of 32 for each head. Linear projection is performed on the fused features to obtain the query Q, key K, and value V matrix. The scaled dot product attention is obtained by calculating the dot product of the query and key, scaling by dividing by the square root of the dimension, applying Softmax normalization, and multiplying by the value matrix to obtain the attention output. This output is then processed by linear projection, residual connections, and layer normalization to generate the final feature pyramid. This feature pyramid balances local details and global context, providing high-quality multi-scale feature input for the subsequent DETR decoder.

[0041] As a further optimized technical solution, step S3 specifically includes:

[0042] The DGQ module guides query allocation by predicting the target density map. The density map is generated through three convolutional layers: the first layer is a 3×3 convolution (256 input channels, 128 output channels) with batch normalization and ReLU activation; the second layer is a 3×3 convolution (128 input channels, 64 output channels) with batch normalization and ReLU activation; and the third layer is a 1×1 convolution (64 input channels, 1 output channel) without an activation function to maintain the continuity of density values. The spatial resolution of the output single-channel density map is consistent with the input features. Figure 1 If necessary, the image can be upsampled to the original resolution using bilinear interpolation. The ground truth density map is generated using a Gaussian kernel convolution method; the formula for generating the ground truth density map is:

[0043]

[0044] Among them, D gt (p) represents the truth density value at position p, c i =(x i ,y i ) represents the center coordinates of the i-th target instance, ‖pc i ‖ represents the distance between position p and target center c. i The Euclidean distance between them, σ represents the standard deviation parameter of the Gaussian kernel (optimized to 1.5 for small targets in remote sensing images), ∑ i This represents the summation over all target instances, and exp(·) denotes the exponential function. Each pixel value in the density map reflects the degree of target clustering around that location.

[0045] The setting of the standard deviation σ of the Gaussian kernel is based on the average pixel diameter d of the target pixel One-fourth of, that is, σ = d pixel / 4. This ratio is based on the 3σ principle of the Gaussian distribution to ensure that the coverage range of the Gaussian kernel is about 1.5 times the target diameter, which can effectively represent the spatial occupancy of the target; for the AI-TOD remote sensing dataset, by statistically analyzing the bounding box sizes of the small target categories in the training set, the average diameter d pixel is about 6 pixels, so σ = 1.5 is set; for other remote sensing datasets, the average size of the target should be statistically analyzed first, and then the σ value should be calculated according to the above ratio. For example, when d pixel is about 8 pixels in the VisDrone dataset, σ = 2.0 is set, and when d pixel is about 12 pixels in the DOTA dataset, σ = 3.0 is set.

[0046] The density map classification head is used for auxiliary supervision and contains a classification branch for predicting density levels: This branch sequentially performs 3×3 convolution (1→32 channels, ReLU), 3×3 convolution (32→64 channels, ReLU), adaptive global average pooling (output 1×1 spatial size), flattening, and a fully connected layer (64→4 dimensions) on the single-channel density map, and outputs the logits of 4 density levels, corresponding to sparse scenarios (number of targets N ≤ 10), low-density scenarios (10 < N ≤ 100), medium-density scenarios (100 < N ≤ 500), and high-density scenarios (N > 500).

[0047] In the training stage, the lightweight convolution head processes the high-resolution feature map output by REM to predict the density map D pred . To further enhance the feature expression ability, the present invention introduces a counting-guided feature enhancement mechanism, which uses channel attention and spatial attention modules (CBAM) to refine the encoder features according to density predictions. Specifically, the channel attention module adaptively adjusts the weights of different feature channels to highlight the key channels related to the target density; the spatial attention module performs weighting in the spatial dimension to make the model focus on high-density regions, thereby achieving more accurate query positioning in dense regions.

[0048] Density prediction adopts a dual-supervision strategy, and the supervision loss formula is:

[0049]

[0050] Among them, represents the total density loss, D pred represents the predicted density map, D gt represents the true value density map, represents the square of the L2 norm (mean square error) to ensure the accuracy of density prediction, λcls These are the weighting coefficients of the classification loss. The binary cross-entropy loss generated by the auxiliary classification head is used to facilitate effective foreground-background discrimination and provide semantic alignment. This auxiliary classification head works in parallel with the density prediction head, enhancing the discriminative power of features through a multi-task learning mechanism.

[0051] Dynamic query allocation estimates the total number of targets based on the integral value of the predicted density map:

[0052] N=ΣD pred

[0053] Where N represents the estimated total number of target instances, and D pred This represents the predicted density map, and ∑ represents the summation of all pixel values ​​in the density map, with the result rounded to the nearest integer.

[0054] Based on the estimated target number N, a hierarchical query strategy is used to dynamically adjust the number of queries:

[0055]

[0056] Here, Queries represents the number of queries allocated. This hierarchical strategy avoids over-querying in sparse scenarios (leading to noise and slower convergence) and under-querying in dense scenarios (leading to missed detections).

[0057] The initial query location employs a density-guided weighted sampling strategy: the predicted density map is normalized to a probability distribution, such that the sampling probability of each pixel location is proportional to its density value. Then, the image spatial locations are weighted and randomly sampled according to this probability distribution to generate the required number of initial query location coordinates. This strategy results in a higher probability of sampling high-density areas (such as dense parking lots or port ship gathering areas). The DETR decoder's query focuses on potential target regions from the first layer, significantly improving the attention allocation to small, dense targets. Compared to the uniform sampling strategy, the recall rate in dense scenes is improved by approximately 8%.

[0058] As a further optimized technical solution, step S4 specifically includes:

[0059] The aligned multi-scale fused features obtained in step S2 and the dynamic query obtained in step S3 are input into the DETR decoder. The DETR decoder contains multiple decoding layers, each of which performs self-attention and cross-attention processing. Self-attention captures the relationships between queries, while cross-attention interacts the queries with the fused features.

[0060] The decoder consists of six decoding layers, with the following operation sequence: self-attention, layer normalization, cross-attention, layer normalization, feedforward network, and layer normalization. Self-attention employs a multi-head attention mechanism with eight heads and an embedding dimension of 256, with each head having a dimension of 32, used to model the relationships between queries. Cross-attention uses multi-scale deformable attention with an embedding dimension of 256, aggregating information across five feature scales. Each query learns multiple sampling point position offsets to achieve adaptive sampling of the feature map. The feedforward network contains two linear layers: the first layer expands the 256-dimensional network to 2048-dimensional and activates it using ReLU; the second layer compresses the 2048-dimensional network back to 256-dimensional, representing an 8-fold expansion.

[0061] The auxiliary prediction heads are set in the 2nd, 4th, and 6th decoding layers. The structure of each auxiliary prediction head is the same as that of the main prediction head, and each contains independent classification and regression branches. The classification branch is a 3-layer multilayer perceptron with a structure of 256-dimensional input, 256-dimensional hidden layer (ReLU activation), and num_classes-dimensional output (for the target num_classes of the dataset), outputting the logits of each query belonging to each category. The regression branch is also a 3-layer multilayer perceptron with a structure of 256-dimensional input, 256-dimensional hidden layer (ReLU activation), and 4-dimensional output, outputting normalized bounding box coordinates (cx, cy, w, h), where cx and cy represent the normalized coordinates of the center point relative to the image width and height, and w and h represent the normalization ratio of the bounding box width and height relative to the image width and height, with values ​​ranging from 0 to 1.

[0062] The loss function is configured as follows: the classification loss uses Quality Focal Loss with sigmoid activation, beta parameter set to 2.0, and loss weight is 1.0; the bounding box L1 loss is used to supervise coordinate regression, with a loss weight of 5.0; the bounding box GIoU loss is used to optimize the localization accuracy and scale prediction of the bounding box, with a loss weight of 2.0; the loss weight coefficient of the auxiliary prediction head is set to 2.0. The total loss during the training phase is twice the sum of the main prediction head loss and the auxiliary prediction head loss. This deep supervision strategy provides direct gradient signals to the intermediate decoding layer, accelerates model convergence, and improves the final detection performance.

[0063] During the inference phase, only the main prediction head output from the 6th layer decoder is used. Soft-NMS post-processing is performed on the predicted bounding boxes: the IoU threshold is set to 0.6, and a linear decay function is used to reduce the confidence of low-scoring detection boxes overlapping with high-scoring detection boxes instead of directly deleting them, thus retaining more potentially correct detections; a maximum of 1000 detection results are retained per image, with a score threshold set to 0.05 to filter out low-confidence false detections. The bounding box refinement mechanism iteratively optimizes the query representation at each decoding layer. The output bounding box of layer i serves as the input reference box for layer i+1, refining the position and size of the bounding boxes layer by layer. This mechanism makes the final output bounding box localization more accurate.

[0064] As a further optimized technical solution, the present invention adopts a three-stage progressive training strategy, which is implemented as follows:

[0065] Phase 1 is the independent pre-training phase, in which three expert backbone networks, ResNet, SwinTransformerV1, and Vision Transformer, are pre-trained on the ImageNet-1k large-scale image classification dataset. ResNet uses the stochastic gradient descent (SGD) optimizer with a learning rate of 0.1, momentum of 0.9, weight decay of 1e-4, and cosine annealing learning rate scheduling, and is trained for 90 epochs. Swin and ViT use the AdamW optimizer with a learning rate of 1e-3, weight decay of 0.05 and 0.3 respectively, and cosine learning rate scheduling combined with warmup training until convergence. The goal of this phase is to enable each expert backbone network to acquire powerful general visual feature extraction capabilities.

[0066] Phase two is the structured joint training phase, which involves training on the joint training set of the remote sensing target detection dataset. The parameters of all expert backbone networks are fixed, and only task-related modules such as the routing network, REM module, DGQ module, DETR encoder, and decoder are trained. The AdamW optimizer is used, with a base learning rate of 1e-4, weight decay of 1e-4, and a batch size of one image per GPU. Eight GPUs are used for distributed training, resulting in an equivalent batch size of 8. The learning rate is scheduled using a linear warmup followed by a step decay, multiplying by 0.1 at the 30th and 40th epochs, for a total of 50 epochs. The key to this phase is fixing the backbone parameters and rapidly training routing decisions and feature fusion capabilities to avoid overfitting on small-scale remote sensing datasets.

[0067] Phase 3 is the end-to-end fine-tuning phase. On the same training set, the parameters of the ResNet-50, Swin, and ViT backbone networks are unfrozen, but the MoE expert backbone network remains frozen to prevent catastrophic forgetting. The learning rate multiplier for the backbone network is set to 0.1, while the learning rate multiplier for other modules is 1.0, reducing the base learning rate to 1e-5. The AdamW optimizer is used, with weight decay of 1e-4, and the batch size configuration is the same as in Phase 2. The learning rate is scheduled as a linear warmup followed by a step decay, multiplying by 0.1 at the 7th and 14th epochs, for a total of 16 to 30 epochs until the validation set performance converges. Gradient clipping is enabled, with the maximum norm set to 0.1 and the norm type set to L2 norm to prevent gradient explosion. This phase allows the backbone network to adapt to the feature requirements of the object detection task with a small learning rate while maintaining training stability, ultimately achieving end-to-end system optimization.

[0068] This invention provides a multi-scale target detection system based on hybrid expert and density-guided methods, comprising:

[0069] The extraction module is used to extract feature representations from multiple heterogeneous experts from the input data;

[0070] The alignment module is used to perform feature alignment and fusion on heterogeneous expert outputs using the route-enhanced hybrid attention (REM) module;

[0071] The complementary feature generation module is used to predict the target density map and dynamically adjust the query allocation using the density-guided dynamic query (DGQ) module;

[0072] The detection module processes the dynamic query input to the DETR decoder and outputs the final target detection result.

[0073] Compared with existing technologies, the present invention has the following advantages: The present invention proposes a new multi-scale target detection framework that combines a hybrid expert architecture with a density-guided mechanism. This framework avoids the performance trade-offs of traditional homogeneous multi-branch structures when small and large targets coexist, and prevents the complete loss of global semantic information in purely small scenes.

[0074] This invention employs a dynamic expert activation mechanism, adaptively selecting and activating a subset of experts based on the features of the input remote sensing image. The REM module introduces an expert calibration factor α. e Eliminate amplitude deviation and via routing score s e With gate weight w l,eThe dual weighting enables collaborative decision-making on "who should speak" and "what is more important." The DGQ module optimizes σ by combining the characteristics of remote sensing micro-targets and accurately adapts to the remote sensing density distribution through a four-level hierarchical query strategy and density-guided sampling.

[0075] 3. Extensive experiments conducted in multiple challenging scenarios demonstrate that this invention exhibits superior performance compared to existing state-of-the-art object detection methods. This invention demonstrates excellent cross-domain generalization capabilities and shows significant advantages in migration between different remote sensing data sources. Attached Figure Description

[0076] Figure 1 This is a general structural diagram of the framework of an embodiment of the present invention;

[0077] Figure 2 This is a schematic diagram of the adaptive routing mechanism structure according to an embodiment of the present invention;

[0078] Figure 3 This is a schematic diagram of the REM module structure according to an embodiment of the present invention;

[0079] Figure 4 This is a schematic diagram of the DGQ module structure according to an embodiment of the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. The multi-scale target detection method of the present invention adopts a framework based on hybrid experts and density guidance, mainly including two key modules: a route-enhanced hybrid attention module (REM), which uses an adaptive routing mechanism to guide cross-expert feature fusion to achieve scale-adaptive feature representation; and a density-guided dynamic query module (DGQ), which guides query allocation by predicting target density to achieve adaptive processing for different density scenes.

[0081] Example 1

[0082] Please see Figure 1 The multi-scale target detection method of the present invention includes the following steps:

[0083] S1. Extract feature representations from multiple heterogeneous experts from the input remote sensing image. First, multiple different backbone networks are used to extract features from the input image, resulting in multi-scale features with different spatial resolutions. The first type of expert captures local texture features through convolution operations, the second type of expert captures global context through a self-attention mechanism, and the third type of expert captures multi-scale hierarchical features through hierarchical window attention.

[0084] Specifically, the multi-expert feature extraction process includes the following stages:

[0085] (1) Expert configuration stage: The system pre-configures multiple heterogeneous expert backbone networks, including the first type of convolutional expert, the second type of attention expert, and the third type of hierarchical attention expert. The parameters of each expert network have been optimized during the pre-training stage.

[0086] (2) Dynamic Activation Decision Stage: The input remote sensing image first passes through a basic feature extraction network to obtain feature representations, and then the routing network calculates the activation scores of each expert category. Through normalization functions and dynamic threshold filtering, the set of experts that need to be activated for the current image is determined. For example, for an image that mainly contains small targets, the first type of convolutional expert may be mainly activated; for multi-scale complex scenes, multiple types of experts may be activated simultaneously.

[0087] (3) Selective feature extraction stage: Only experts in set A perform the actual forward propagation calculation to extract image features. Unactivated experts do not participate in the calculation at all, their network weights do not need to be loaded, and intermediate activation values ​​do not need to be stored, thus greatly saving computing resources and memory space.

[0088] (4) Feature output stage: Each activation expert outputs multi-scale features with different spatial resolutions. These features carry their respective routing scores and are used for subsequent weighted fusion.

[0089] Through the above dynamic activation process, this invention achieves on-demand allocation and efficient utilization of computing resources while ensuring detection performance, making large-scale multi-expert architecture feasible in practical applications.

[0090] S2. A route-enhanced hybrid attention (REM) module is used to align and fuse features from heterogeneous expert outputs. (See also...) Figure 2 As shown, an adaptive routing mechanism is first used to calculate the activation score of each expert. The routing score is obtained through global average pooling and linear transformation, and then normalized using a normalization function. A dynamic thresholding mechanism is employed to ensure that at least one expert of each type is activated, preventing any expert type from being completely suppressed. For activated experts, their outputs are feature aligned: convolutional features are upsampled and aligned using a Feature Pyramid Network (FPN), and attention-related features are converted into a unified feature map representation using a Simple Feature Pyramid (SFP). Next, a dynamic gating mechanism is used to calculate scale-specific fusion weights. The gating network aggregates expert features through global average pooling, and then obtains the gating weights for each pyramid layer through a normalization function. At each pyramid layer, the normalized features of the activated experts are weighted and fused by combining the routing score and the gating weights. Finally, a multi-head attention mechanism is used to hierarchically integrate the fused multi-scale features to generate the final feature pyramid.

[0091] S3. Use the Density-Guided Dynamic Query (DGQ) module to predict the target density map and dynamically adjust query allocation. See also... Figure 3 As shown, a lightweight convolutional head is first used to process the high-resolution feature map output by REM to predict the density map. To enhance feature representation, a count-guided feature enhancement mechanism is introduced, utilizing channel attention and spatial attention modules (CBAM) to refine encoder features based on density prediction, enabling the model to achieve more accurate query localization in dense regions. Density prediction is trained using a dual-supervised strategy, including mean squared error loss and classification loss generated by an auxiliary classification head, enhancing feature discrimination ability through multi-task learning. Then, the total number of targets is estimated using the density map predicted by integration, and the number of queries is dynamically adjusted using a hierarchical strategy based on the estimated number of targets. Finally, a density-guided weighted sampling strategy is used to preferentially sample the initial query position from high-density regions, ensuring that the query focuses on potential target clusters from the outset.

[0092] S4. The dynamic query is input into the DETR decoder for processing. The DETR decoder contains multiple decoding layers, each performing self-attention and cross-attention processing. Self-attention captures the relationships between queries, while cross-attention interacts the query with the fused features. Auxiliary prediction heads (including classification and bounding box regression branches) are introduced into several intermediate layers of the decoder for deep supervision, accelerating training convergence. The output of the final decoder layer is processed by the main prediction head, outputting the final object detection result, including bounding box coordinates, class labels, and confidence scores. During training, the losses of the auxiliary and main prediction heads are weighted and combined for optimization; during inference, only the output of the main prediction head is used.

[0093] Example 2

[0094] This invention also provides a multi-scale target detection system based on hybrid experts and density-guided algorithms, corresponding to Embodiment 1, comprising: an extraction module for extracting feature representations from multiple heterogeneous experts from input data; an alignment module for aligning and fusing the outputs of heterogeneous experts using a route-enhanced hybrid attention (REM) module; a complementary feature generation module for predicting the target density map and dynamically adjusting the query allocation using a density-guided dynamic query (DGQ) module; and a detection module for processing the dynamic query input into a DETR decoder and outputting the final target detection result. The operations performed by each module are the same as in Embodiment 1.

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features, and such modifications or substitutions do not cause the related technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-scale target detection method based on hybrid expert and density-guided methods, characterized in that, include: S1. Extract feature representations from multiple heterogeneous experts from the input data; S2. Use the route-enhanced hybrid attention (REM) module to perform feature alignment and fusion on heterogeneous expert outputs; S3. Use the density-guided dynamic query (DGQ) module to predict the target density map and dynamically adjust the query allocation; S4. Input the dynamic query into the DETR decoder for processing and output the final target detection result.

2. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 1, characterized in that, In step S1, multiple heterogeneous expert backbone networks are used to extract features in parallel. An adaptive routing mechanism dynamically selects the activated experts, and the routing score is calculated using the following formula: [s1,s2,…,s E ]=Softmax(FC(Flatten(GAP(M)))) The activated expert set is determined through a category-aware Top-1 selection mechanism: Where M represents the basic feature, s e This represents the routing weight of expert e. Indicates the set of experts that are active. Indicates the expert category.

3. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 2, characterized in that, The adaptive routing mechanism uses dynamic thresholds to ensure that at least one expert of each type is activated, preventing any expert type from being completely suppressed.

4. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 1, characterized in that, In step S2, the REM module uses a feature pyramid network to align convolutional features and a simple feature pyramid to align attention-related features.

5. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 4, characterized in that, The REM module performs scale normalization on the features of activation experts: Scale-specific fusion weights are calculated using a dynamic gating mechanism: w l =Softmax(W l ·g l +b l ) The features are weighted and fused by combining routing scores and gating weights: And hierarchical integration is achieved through a multi-head attention mechanism: in, Represents the original feature, α e W represents the calibration factor. l and b l s represents the learnable parameter. e Indicates the routing score, w l,e Indicates the gating weight, The fusion features are represented by Q, K, and V, which represent the query, key, and value matrix, respectively, and d represents the feature dimension. final Indicates the final characteristic.

6. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 1, characterized in that, In step S3, the DGQ module uses a Gaussian kernel to generate a density map: Among them, D gt (p) represents the truth density value at position p, c i =(x i ,y i ) represents the center position of the i-th target, ‖pc i ‖ represents the Euclidean distance, and σ represents the standard deviation parameter of the Gaussian kernel (optimized to 1.5 for small targets in remote sensing images).

7. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 6, characterized in that, Density prediction employs a dual-supervision strategy, with the supervision loss being: in, D represents the total density loss. pred D represents the predicted density map. gt Represents the truth density plot. λ represents the square of the L2 norm (mean square error). cls These are the weighting coefficients of the classification loss. This represents the binary cross-entropy loss of the auxiliary classification head.

8. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 1, characterized in that, Dynamic query allocation employs a hierarchical query strategy, first estimating the total number of targets using an integral density map: N=∑D pred Then, dynamically adjust the number of queries based on the estimated target number $N$: Where N represents the estimated total number of targets, and D pred This represents the density map of the predictions, and Queries represents the number of queries allocated.

9. The multi-scale target detection method based on hybrid expert and density-guided methods as described in claim 1, characterized in that, A three-stage progressive training strategy is adopted: independent pre-training, structured joint training, and end-to-end fine-tuning.

10. A multi-scale target detection system based on hybrid expert and density-guided methods, characterized in that, include: Extraction module, alignment module, complementary feature generation module, detection module.

Images