Method for detecting dense overlapping targets based on wavelet-enhanced sparse mixture expert model

Abstract

The application provides a dense overlapping target detection method based on a wavelet enhanced sparse mixed expert model. The method first extracts multi-layer features through a backbone network to capture multi-scale spatial representations. Secondly, discrete wavelet transform is introduced on each level feature to decompose the spatial feature into the frequency domain, realize collaborative modeling of the frequency domain and spatial domain features, improve the reservation capability of detail and texture information, and introduce a lightweight dynamic hypergraph aggregation module in the deepest layer feature to adaptively learn the hyperedge structure to explicitly model the high-order correlation relationship in the local area. Thirdly, in the decoding process, the candidate queries are screened and reweighted through an IoU perception query selection mechanism, and a dynamic routing mechanism of a sparse mixed expert is introduced to realize adaptive and specialized representation learning of the query, effectively improving the target detection accuracy and reliability in complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of UAV remote sensing agriculture, and mainly relates to a method for detecting densely overlapping targets based on a wavelet-enhanced sparse hybrid expert model. Background Technology

[0002] The counting of crop seedlings and the detection of key traits during the growth stage are core components of modern intelligent breeding systems. Using UAV remote sensing imagery for corn seedling counting and wheat ear detection has become an important technical means to achieve high-throughput, non-destructive field phenotypic acquisition. High-resolution UAV imagery can not only quickly cover large-area breeding experimental fields but also accurately identify the number and spatial distribution of corn seedlings, dynamically monitor seedling growth uniformity, and provide objective data support for early assessment of seed vigor, stress resistance, and field seedling potential. Simultaneously, during the wheat heading to grain-filling stage, ear counting using UAV imagery can efficiently estimate the number of effective ears per unit area—a key factor determining yield. This technology effectively overcomes the limitations of traditional manual surveys, which are time-consuming, labor-intensive, and highly subjective, providing reliable data support for the precise screening of large-scale breeding populations, optimization of field management, and genotype-environment interaction analysis.

[0003] Detecting corn seedlings and wheat ears based on UAV remote sensing imagery is a key task in precision agriculture, essentially a fine-grained target detection problem. Such tasks typically face challenges such as small target scale, dense distribution, complex backgrounds, and varying imaging perspectives, placing higher demands on the localization accuracy, context awareness, and robustness of detection models. In recent years, with the development of deep learning, target detection methods have evolved from two-stage to single-stage, and then to end-to-end architectures. Although traditional methods such as Faster R-CNN achieve high detection accuracy through Region Proposal Networks (RPNs), the computational redundancy and inference latency introduced by its two-stage design limit its widespread application in real-time remote sensing analysis. While the YOLO series performs excellently in terms of speed and is suitable for real-time scenarios, it still suffers from high false negative rates and large localization errors when handling high-density, small-scale targets, especially in complex farmland backgrounds where it struggles to fully model the spatial relationships between targets.

[0004] In contrast, DETR (Detection Transformer) pioneered the introduction of the Transformer architecture into object detection, abandoning manually designed anchor boxes and post-processing NMS. It achieves ensemble end-to-end prediction through object queries, possessing stronger global context modeling capabilities and demonstrating good potential in handling complex scenes. However, the original DETR suffers from slow convergence and sensitivity to small targets, limiting its direct application in fine-grained remote sensing detection. Therefore, existing methods still have room for improvement in small target perception, occlusion handling, and multi-scale feature fusion in complex farmland scenes, necessitating the design of more targeted end-to-end detection frameworks tailored to the characteristics of remote sensing images. Summary of the Invention

[0005] This invention proposes a target detection method based on a wavelet-enhanced sparse hybrid expert model for densely overlapping targets in agricultural field scenarios. First, the method extracts multi-layer features through a backbone network to capture multi-scale spatial representations. Second, discrete wavelet transform is introduced at each feature level to decompose spatial features into the frequency domain, achieving collaborative modeling of frequency and spatial features and improving the preservation of details and texture information. A lightweight dynamic hypergraph aggregation module is introduced into the deepest feature layer to adaptively learn hyperedge structures, explicitly modeling high-order relationships within local regions. Third, during the decoding process, an IoU-aware query selection mechanism is used to filter and reweight candidate queries, and a dynamic routing mechanism of the sparse hybrid expert model is introduced to achieve adaptive specialized representation learning for queries, effectively improving the accuracy and reliability of target detection in complex scenarios.

[0006] The technical solution adopted in this invention is: a dense overlapping target detection method based on a wavelet-enhanced sparse hybrid expert model, comprising the following steps: Step 1: Extract multi-level pyramid features from the input image; Step 2: Wavelet frequency domain feature modulation module (WFFM) is introduced into each level of features. Spatial features are decomposed into the frequency domain through discrete wavelet transform, realizing the collaborative modeling and reconstruction of frequency domain and spatial domain features. Subsequently, dynamic hypergraph attention module (DHAM) is integrated on the deepest feature layer. Hypergraph structure is adaptively constructed to explicitly model high-order semantic associations in local regions. The multi-scale features enhanced by WFFM and DHAM are fed into a bidirectional feature fusion path composed of a feature pyramid network and a path aggregation network, realizing multi-scale feature aggregation that combines top-down and bottom-up approaches. Step 3: Construct a decoder based on a sparse hybrid expert structure to obtain higher-level features that are more discriminative and semantically consistent. These features are then input into the prediction head module to complete the accurate regression and determination of the target location box.

[0007] Furthermore, in step 1, PREsNet is used as the backbone network to extract the four-level pyramid features of the input image. The backbone network first performs preliminary downsampling and feature encoding on the input image through a 7×7 convolutional layer and a max pooling layer to obtain low-level spatial texture information. Subsequently, after four stages, each stage is composed of several residual block bottlenecks stacked together, and channel matching and spatial downsampling are completed through downsampling convolution when entering the next stage.

[0008] Furthermore, the wavelet frequency domain feature modulation module (WFFM) includes: firstly, obtaining the low-frequency approximation of the backbone features and multi-directional high-frequency details through high- and low-frequency feature decoupling; then, compressing and fusing them in the channel dimension during the cross-frequency fusion stage to form a multi-scale representation that combines global structure and local details. High- and low-frequency feature decoupling specifically includes: After decoupling the output feature layer using a single-layer two-dimensional discrete wavelet AFB2D, low-frequency components and high-frequency components in three directions are obtained. Single-layer two-dimensional discrete wavelet AFB2D applies low-pass and high-pass filters to the input features in the horizontal and vertical directions, respectively, and then performs a 2:1 downsampling operation on the results. The low-frequency components retain the main structure and overall texture information of the input features, and are a smooth approximation of the image. Provides detailed information at the horizontal, vertical, and diagonal levels; cross-frequency fusion includes concatenating the low-frequency and three high-frequency components along the channel dimension to obtain the fused features. Subsequently, the introduction Convolutional layers, batch normalization, and ReLU activation functions are used for feature compression and fusion to obtain a fused feature map. ; Furthermore, bilinear interpolation is used to upsample the fused features back to the spatial resolution of the original input feature map, thereby maintaining the scale alignment of the multi-scale feature pyramid. The specific operation is as follows:

[0009] in, The final output feature map has a spatial resolution aligned with the original input feature map. Indicates an upsampling operation. This indicates the height and width of the upsampled feature map, where The feature map height and width are defined separately.

[0010] Furthermore, the Dynamic Hypergraph Attention Module (DHAM) comprises two parts: Hypergraph-based Adaptive Relevance Enhancement HG-ACE and a Transformer encoder. The adaptive correlation enhancement HG-ACE based on hypergraphs specifically includes: First, construct the distance metric matrix: given the input feature layer as... ,in For batch size, Indicates the number of channels. The spatial dimensions of the feature are first flattened into a sequence form, and the similarity between any two nodes is calculated to form a distance matrix. :

[0011]

[0012] in, The total number of tokens. For feature dimension, This represents a one-dimensional flattening feature sequence operation. Indicates token and The cosine distance, i.e., the distance metric matrix between all position tokens; Subsequently, a neighbor weight matrix is ​​constructed: based on a threshold number of neighbors K, a differentiable feature similarity metric is used to adaptively filter neighbors, resulting in a neighbor set. Furthermore, a lightweight two-layer attention network was designed to calculate the weights, ultimately obtaining the neighbor attention weights. All neighbor weights constitute the neighbor attention matrix. The specific operation is shown in the following formula:

[0013]

[0014] in, Represents an adaptive neighbor selection function that measures the similarity of differentiable features. Represents the set of all neighbors of node i; These are the normalized attention weights, representing the neighbors. To the center Contribution For query and key projection; Let be the learnable projection matrix, and d be the attention dimension; Finally, a self-learning hypergraph structure is constructed: a locally dynamic hypergraph is built, with each central hyperedge... Connecting nodes It and its K nearest neighbors are used to capture non-Euclidean associations in local regions. Neighbor features are injected into the central node through weighted aggregation. Finally, the combined results of aggregating all tokens yield the HG-ACE output. The specific operation is shown in the following formula:

[0015]

[0016]

[0017] in, It is a shared feedforward network used for nonlinear mapping; For adaptive learning of transformable matrices, The attention weights are obtained through dynamic learning using a lightweight attention network. It is a learnable transformation matrix.

[0018] Furthermore, the specific process of the Transformer encoder is as follows: First, positional encoding is introduced into the features of the dynamic hypergraph enhancement output. get Next, the single-head attention mechanism calculates the similarity between the query matrix and the key matrix using dot product to measure the relevance between nodes. Then, it normalizes this similarity using the Softmax function to map it into an attention weight distribution, which is then applied to the value matrix to obtain the output of the single-head attention mechanism. Finally, construct multi-head attention as Finally, after being concatenated with the residuals and normalized, the output is... .

[0019] Furthermore, the decoder based on the sparse hybrid expert structure includes IoU-aware query selection and multiple sparse hybrid expert structure modules (S-MoE Blocks). The IoU-aware query selection is used to filter and reweight candidate queries before they enter S-MoE Blocks, thereby improving the quality of the decoder input. The processing steps for each sparse hybrid expert structure module are as follows: First, the input query is modeled using a self-attention mechanism to capture the dependencies between different queries; then, multi-scale feature information is fused using a cross-attention mechanism to obtain a context-enhanced query representation; next, the input is fed into a gated routing network (Router), which learns the weight distribution of each query among different experts and introduces Dropout and learnable noise into the output probability distribution to improve robustness; finally, multiple expert networks (Expert) perform high-dimensional nonlinear transformations on the weighted query features to achieve dynamic representation fusion across experts.

[0020] Furthermore, the IoU-aware query selection process is as follows: From fusion characteristics Extract the first Each query corresponds to a bounding box. The local feature response is used to obtain the local feature vector. Subsequently, based on the maximum IoU between the predicted bounding box and the ground truth annotation and its value in the data, the maximum IoU between the predicted bounding box and the ground truth annotation was determined. Semantic saliency generates importance weights Finally, the optimized query representation is output, which integrates contextual features and dynamically adjusts them according to weights, gradually suppressing low-quality queries to a mask m; the specific formula is as follows:

[0021]

[0022]

[0023] In the formula, This is a bilinear sampling feature alignment operation. It is the aggregated feature from the output of step 2. No. Each query corresponds to a bounding box; For the Sigmoid activation function, , Both represent learnable scaling factors. Represents the set of actual bounding boxes. The maximum intersection-union ratio; Denotes the L2 norm; For query The gating coefficient, For the linear transformation matrix of query-feature fusion, This indicates a feature-level concatenation operation; It is the query vector of the original input object. It is a learnable invalid query mask.

[0024] Furthermore, the expert network processing procedure is as follows: Each expert Defined as a two-layer fully connected feedforward network, it performs dimensionality mapping on the input features through a linear transformation. First, it maps the input features... The 1D feature is projected into the higher-dimensional latent space through the first fully connected layer. This expands the feature representation space and enhances nonlinear modeling capabilities, and is then mapped to the output dimension via a second fully connected layer. This is to achieve linear recombination of features and dimensional adaptation; the operation is shown in the following formula:

[0025] in, Represented as the weights and biases of the first fully connected layer. This indicates the second fully connected layer and the bias. For non-linear activation functions, use the ReLU function; This is a random deactivation mechanism to alleviate fitting. This represents the query features after the fusion of self-Attention and cross-Attention.

[0026] Furthermore, the processing procedure of the gated routing network is as follows: Query representation after self-attention weighting Then, the query features after cross-attention fusion processing are used. The gating network generates expert selection weight scores for each query feature and calculates the corresponding scores. ;

[0027]

[0028]

[0029] in, This indicates the query output optimized by the IoU-aware query selection mechanism. Indicates by Keys and values ​​obtained through linear projection; The dimensions of attention head; From Keys and values ​​generated by linear projection The parameter represents the learnable linear mapping, and E is the number of experts; Dropout introduces regularization in the gated routing phase; This means injecting Gaussian noise with a trainable scale into the gated routing output; Used to normalize scores into a probability distribution This is a general representation of a gated routing network. Next, a TopK sparse selection strategy is applied to the probability distribution, retaining only those with the highest scores. Several experts participate in the routing of the current query features, and the index of these experts is subsequently obtained. And generate expert sparse masks. The specific formula is shown below:

[0030]

[0031] in, Return the indexes of the k experts with the highest weighted scores; Finally, the scores of the retained experts are normalized so that the final gated route weights used for weighted aggregation are: Subsequently, the weighted sum of the selected expert outputs is output, that is, the weighted sum of the corresponding activated expert outputs is performed using the normalized gating scores as weights, forming a representation of cross-expert dynamic fusion. The specific formula is shown below:

[0032]

[0033] in, It is a numerically stable term that prevents division by zero. This represents the gated route weight corresponding to the j-th expert. This represents the forward inference result of the j-th expert.

[0034] The present invention also provides a dense overlapping target detection system based on a wavelet-enhanced sparse hybrid expert model, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the dense overlapping target detection method based on the wavelet-enhanced sparse hybrid expert model as described in the above technical solution.

[0035] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: 1. This invention designs a feature fusion module for joint spatial-frequency domain representation and dynamic hypergraph aggregation, which enhances the joint spatial-frequency representation capability of features at different scales and effectively models higher-order relations; 2. A sparse hybrid expert mechanism is introduced into the decoder to realize query-specific modeling and expert selection, replacing the traditional decoder layer, which effectively improves the accuracy and reliability of densely overlapping target detection; 3. The method was validated using a self-built maize seedling dataset consisting of nearly 30,000 images, and compared with current mainstream methods. Extensive experiments proved that the proposed method performs best on two different datasets, verifying the effectiveness and robustness of the proposed method in agricultural scenarios with small target scale, dense distribution, and complex background. Attached Figure Description

[0036] Figure 1 The overall framework of the MoE-DETR method of the present invention includes (a) a backbone feature extraction module, (b) an adaptive enhanced feature aggregation module, (c) an S-MoE Decoder, and (d) the structure of an S-MoE Block.

[0037] Figure 2 This is a structural diagram of the wavelet frequency domain feature modulation module (WFFM) in an embodiment of the present invention.

[0038] Figure 3 This is a structural diagram of the Dynamic Hypergraph Aggregation Module (DHAM) in an embodiment of the present invention.

[0039] Figure 4 This is a structural diagram of Expert and Router in an embodiment of the present invention. Detailed Implementation

[0040] The specific implementation process of the present invention will be described in detail below with reference to the accompanying drawings.

[0041] This invention addresses the problem of densely overlapping target detection, such as counting corn seedlings and detecting wheat ears. It proposes a densely overlapping target detection method based on a wavelet-enhanced sparse hybrid expert framework, abbreviated as MoE-DETR. Its overall network structure is as follows: Figure 1 As shown. This network mainly consists of a backbone feature extraction module ( Figure 1 (a) in the text, and the adaptive enhanced feature aggregation module ( Figure 1 (b) of the above, the decoder module based on the sparse hybrid expert structure (S-MoE) Figure 1 The S-MoE decoder module consists of three parts (c), where the S-MoE decoder module comprises IoU query selection and multiple S-MoE Blocks. The specific implementation of each S-MoE Block is as follows: Figure 1 As shown in (d) in the figure.

[0042] The specific process of the entire network is as follows: (1) In the feature extraction stage, PREsNet is used as the backbone network to extract the four-level pyramid features (i.e. S2 to S5) of the input image to capture multi-scale spatial representation.

[0043] (2) To enhance the expressive power of features, a two-stage feature enhancement module was designed: First, a wavelet frequency domain feature modulation module (WFFM) was introduced on the features at each level from S2 to S5. The spatial features were decomposed into the frequency domain through discrete wavelet transform, realizing the collaborative modeling and reconstruction of frequency domain and spatial domain features, and improving the ability to preserve details and texture information. Then, a dynamic hypergraph attention module (DHAM) was integrated on the deepest feature layer. The hypergraph structure was adaptively constructed to explicitly model the high-order semantic associations in the local area, thereby enhancing the context representation ability. The multi-scale features enhanced by WFFM and DHAM were fed into a bidirectional feature fusion path composed of a feature pyramid network (FPN) and a path aggregation network (PAN), realizing multi-scale feature aggregation that combines top-down and bottom-up approaches, further strengthening the semantic consistency and positioning accuracy of cross-level features.

[0044] (3) At the decoder end, a decoder based on a sparse mixture-of-experts (S-MoE) structure is proposed: First, an IoU-aware query selection method is introduced to filter and reweight the initial query, enhancing the query response that is highly relevant to the target and suppressing irrelevant or redundant components. Then, a sparse mixture-of-experts (S-MoE) structure is designed, which dynamically selects the Top-k expert subnetworks through a gated routing mechanism, performs adaptive nonlinear transformations for different queries, realizes fine-grained, query-specific representation learning, and improves the model's ability to model complex scenes. After processing by the S-MoE module, the target query output forms a more discriminative and semantically consistent high-level feature under the fusion of multi-expert representations, which is then input into the prediction head module to complete the accurate regression and determination of the target location box.

[0045] This method effectively improves the segmentation performance and generalization ability of the model in complex scenarios through the collaborative design of frequency domain enhancement, high-order relation modeling, and sparse expert decoding. The forward propagation formula of the entire network is shown in Equation (1).

[0046] (1) in For network input, Indicates the backbone network. This represents the feature layer extracted by the backbone network. It is a single-layer two-dimensional discrete wavelet; The deepest feature layer go through The module's processing feature layer; This indicates the output of the adaptively enhanced fused feature sequence. Indicates the initial query. This is the final output of the network.

[0047] The following is a detailed description of each module with reference to the accompanying drawings: 1. Main Feature Extraction Module In this invention, we employ PREsNet as the backbone feature extraction network. Its design inherits the residual structure of the classic ResNet-50, while introducing finer-grained grouped convolutional configurations (variant c / d structure) and frozen batch normalization (FrozenBN) in the convolutional layers and normalization processes to improve feature representation capabilities and training stability in object detection tasks. This module can effectively extract multi-scale spatial semantic information, providing a solid foundation for subsequent feature enhancement and decoding processes.

[0048] Specifically, the backbone network first performs preliminary downsampling and feature encoding on the input image using a 7×7 convolutional layer (stride=2) and a max-pooling layer (3×3, stride=2) to obtain low-level spatial texture information. Subsequently, the main body consists of four stages (S2–S5), each composed of several stacked residual bottlenecks. Upon entering a new stage, downsampling convolutions (1×1 convolutions, stride=2) are used in the shortcut branch to complete channel matching and spatial downsampling. Figure 1 As shown in (a), the network gradually increases the number of channels while progressively decreasing the spatial resolution of the feature maps, thereby achieving hierarchical feature representation. Ultimately, the backbone outputs four sets of multi-scale feature maps with spatial resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input size, corresponding to channel dimensions C2=256, C3=512, C4=1024, and C5=2048. These multi-scale features provide rich contextual information for subsequent feature fusion modules and decoders. The entire backbone feature extraction process corresponds to the backbone stage in formula (1).

[0049] 2. Adaptive Enhancement Feature Aggregation Module 2.1 Frequency Domain Feature Fusion Module Based on Discrete Wavelet Transform (WFFM) To address the issues of edge information loss and insufficient feature representation that easily occur in the detection of small and fine-grained targets, we introduce a frequency domain module based on discrete wavelet transform (WFFM). Specifically, we insert the WFFM module into the multi-scale feature layers processed by the backbone feature extraction module, effectively fusing spatial domain features with high-frequency information in the frequency domain without significantly increasing computational cost. It is worth noting that the core of WFFM consists of two parts: first, obtaining low-frequency approximations and multi-directional high-frequency details of the backbone features through high- and low-frequency feature decoupling; then, compressing and fusing these features along the channel dimension during the cross-frequency fusion stage, thereby forming a multi-scale representation that combines global structure and local details. The module structure diagram is shown below. Figure 2 As shown.

[0050] 1) Decoupling high- and low-frequency features: decoupling the output feature layer The low-frequency component is obtained after single-layer two-dimensional discrete wavelet AFB2D decoupling processing. and high-frequency components in three directions AFB2D applies low-pass and high-pass filters to the input features in the horizontal and vertical directions, respectively, and then performs a 2:1 downsampling operation on the results. It preserves the main structure and overall texture information of the input features, and is a smooth approximation of the image; Provides detailed information at fine-grained levels in the horizontal, vertical, and diagonal directions.

[0051] The core operation of this process is shown in the following formula, we will Passing through one-dimensional filters in the horizontal direction respectively and Processing and downsampling by 2 times to obtain low frequency With high frequency components Each high-frequency component is :

[0052] In the formula To output the feature map, Indicates a low-pass filter. This represents a high-pass filter, and Conv represents the convolution operation. This indicates a spatial downsampling of 2 times. These represent the number of channels, height, and width, respectively.

[0053] The high-frequency components obtained in step (1) above are then... Further pass through a one-dimensional filter in the vertical direction and The data is processed and downsampled again by a factor of 2 to finally generate three fine-grained high-frequency components LH, HL, and HH in the horizontal, vertical, and diagonal directions: (2) (3) (4) In the formula The symbol represents a two-dimensional separate convolution, which means performing 1D filtering in both the horizontal and vertical directions.

[0054] 2) Cross-frequency fusion: To fully fuse information from different frequency bands, we further concatenate the low-frequency and three high-frequency components along the channel dimension to obtain the fused features. Subsequently, in order to reduce the channel dimension and perform nonlinear fusion, a method was introduced... Convolutional layers, batch normalization, and ReLU activation functions are used for feature compression and fusion to obtain a fused feature map. The specific operation is shown in the following formula:

[0055]

[0056] in, It is the fused feature map. For splicing operations, It is a low-frequency component. This represents three high-frequency components: fine-grained in the horizontal, vertical, and diagonal directions. It is a feature map after feature compression and fusion. It is a modified linear unit activation function, used to introduce nonlinear characteristics. This indicates batch normalization. This represents a 1x1 convolutional layer used to reduce channel dimensions and perform feature fusion. These represent the number of channels, height, and width, respectively.

[0057] To maintain the alignment of the feature map with the original scale and facilitate seamless integration with the detection head, we further use bilinear interpolation to upsample the fused features back to the spatial resolution of the original input feature map, thereby maintaining the scale alignment of the multi-scale feature pyramid. The specific operation is as follows:

[0058] in, The final output feature map has a spatial resolution aligned with the original input feature map. Indicates an upsampling operation. This specifies the height and width of the upsampled feature map.

[0059] 2.2 Dynamic Hypergraph Aggregation Module DHAM To overcome the limitation of traditional graph structures that can only handle "point-to-point" relationships, and to enable the model to capture richer "many-to-many" contextual relationships in local regions, we added a feature to the deepest layer of the pyramid features output by the WFFM module. A Dynamic Hypergraph Aggregation Module (DHAM) is introduced. The structure of this module is as follows: Figure 3 As shown, this module mainly consists of two parts: Hypergraph-based Adaptive Relevance Enhancement (HG-ACE) and Transformer encoder.

[0060] 1) Hypergraph-based Adaptive Relevance Enhancement (HG-ACE) The Hypergraph-based Adaptive Relevance Enhancement Module (HG-ACE) aims to overcome the limitation of traditional graph structures, which can only model "point-to-point" relationships, by introducing hyperedges to achieve "many-to-many" higher-order relationship modeling. HG-ACE mainly consists of three parts: a distance metric matrix (Distance Matrix), a neighbor attention matrix (Neighbor Attention Matrix), and a hypergraph. The distance metric matrix generates a distance matrix by calculating the similarity between tokens at each location; the neighbor attention matrix selects K neighbors based on a distance threshold (Distance Threshold); and the hypergraph is constructed by connecting hyperedges to form a local dynamic hypergraph to model higher-order associations. HG-ACE can explicitly aggregate local multi-neighbor features while maintaining controllable computational overhead.

[0061] The specific process is as follows: First, construct the distance metric matrix: Specifically, given the input feature layer as... ,in For batch size, Indicates the number of channels. Let be the spatial size of the feature. We first flatten it into a sequence form, calculate the similarity between any two nodes, and form a distance matrix. :

[0062]

[0063] in, The total number of tokens. For feature dimension, This represents a one-dimensional flattening feature sequence operation. Indicates token and The cosine distance. That is, the distance metric matrix between all position tokens.

[0064] Subsequently, a neighbor weight matrix is ​​constructed. To control the size of the neighbor group, this paper adaptively selects neighbors based on a differentiable feature similarity metric, using a threshold neighbor number K, to obtain the neighbor set. Furthermore, to characterize the contribution of different neighbors to the central node, we designed a lightweight two-layer attention network to calculate the weights, ultimately obtaining the neighbor attention weights. All neighbor weights constitute the neighbor attention matrix. The specific operation is shown in the following formula:

[0065]

[0066] in, Represents an adaptive neighbor selection function that measures the similarity of differentiable features. Let K represent the set of all neighbors of node i. Experiments show that K=9 achieves the best balance between information content and discriminability. These are the normalized attention weights, representing the neighbors. To the center Contribution Projecting the query onto the key; is the learnable projection matrix, and d is the attention dimension.

[0067] Finally, we construct a self-learning hypergraph structure: based on the neighbor attention matrix, we further construct a local dynamic hypergraph, with each central hyperedge... Connecting nodes It and its K nearest neighbors are used to capture non-Euclidean associations in local regions. Neighbor features are injected into the central node through weighted aggregation. Finally, the combined results of aggregating all tokens yield the HG-ACE output. The specific operation is shown in the following formula:

[0068]

[0069]

[0070] in, Represents a node The neighborhood group, It is a shared feedforward network used for nonlinear mapping. For adaptive learning of transformable matrices. It is based on attention weights obtained through dynamic learning using a lightweight attention network. This is a learnable transformation matrix. This module explicitly captures the local context structure as a dynamic enhancement feature of the input token while maintaining an extremely low number of parameters.

[0071] 2) Transformer encoder To enhance the structure-aware capability of the encoder while keeping its computational cost manageable, we feed the features enhanced by the dynamic hypergraph into the Transformer encoder to further fuse local structural and global semantic information. The specific process is as follows: First, positional encoding is introduced into the features output by the dynamic hypergraph enhancement. ,get Next, the single-head attention mechanism calculates the similarity between the query matrix and the key matrix using dot product to measure the relevance between nodes. Then, it normalizes this similarity using the Softmax function, mapping it to an attention weight distribution, which is then applied to the value matrix to obtain the output of the single-head attention mechanism. Finally, a multi-head attention mechanism (h=8 in this paper) is constructed as follows: Finally, after being concatenated with the residuals and normalized, the output is... .

[0072] Its core operating formula is:

[0073]

[0074]

[0075]

[0076] in To enhance the node feature sequence of the hypergraph, It is a learnable positional encoding matrix. It is a hypergraph enhancement feature fused with location encoding. This represents the query matrix for the i-th attention head; This represents the key matrix of the i-th attention head; This represents the value matrix of the i-th attention head; This represents the scaling factor to balance the gradient; For multi-head output linear projection matrix, This indicates a splicing operation. The layer normalizes the feature dimensions of each token.

[0077] 3. Decoder based on Sparse Mixture-of-Experts (S-MoE) architecture To decouple representations between different tasks and improve the feature representation capability of the decoder, we propose a decoder based on the Sparse Mixture-of-Experts (S-MoE) architecture, such as... Figure 1 As shown in (c) above. The entire S-MoE Decoder mainly consists of two parts: IoU-aware query selection and S-MoE Blocks. The details of these two parts are as follows.

[0078] 3.1 IoU-aware query selection IoU-aware query selection is used to filter and reweight candidate queries before they enter S-MoE Blocks, improving the quality of the decoder input. This method utilizes multi-scale fusion features. Geometric alignment information with the predicted bounding box is used to perform context-aware filtering and reweighting of candidate queries. Specifically, this is achieved from fused features. Extract the first Each query corresponds to a bounding box. The local feature response is used to obtain the local feature vector. Subsequently, based on the maximum IoU between the predicted bounding box and the ground truth annotation and its value in the data, the maximum IoU between the predicted bounding box and the ground truth annotation was determined. Semantic saliency generates importance weights Finally, the optimized query representation is output, which integrates contextual features and dynamically adjusts them according to weights, gradually suppressing low-quality queries to a mask m. The specific formula is as follows:

[0079]

[0080]

[0081] In the formula, This is a bilinear sampling feature alignment operation. It comes from the adaptive enhancement feature aggregation module ( Figure 1 The multi-scale fusion features output in (b) are specifically operated as shown in the Features fusion in formula (1). No. Each query corresponds to a bounding box; For the Sigmoid activation function, , Both represent learnable scaling factors. Represents the set of actual bounding boxes. The maximum intersection-union ratio; This represents the L2 norm, used to measure the significance of the features in the region; For query The gating coefficient, For the linear transformation matrix of query-feature fusion, This indicates a feature-level concatenation operation; It is the query vector of the original input object. It is a learnable invalid query mask.

[0082] 3.2 S-MoE Blocks The S-MoE Blocks in this embodiment of the invention consist of 6 S-MoE Blocks, and the overall structure of each S-MoE Block is as follows: Figure 1 As shown in (d) in the diagram, it first models the input query through a self-attention mechanism to capture the dependencies between different queries; then, it fuses multi-scale feature information through a cross-attention mechanism to obtain a context-enhanced query representation; next, it is input into a gated routing network (Router), which learns the weight distribution of each query among different experts and introduces Dropout and learnable noise into the output probability distribution to improve robustness; finally, multiple expert networks (a lightweight and efficient two-layer feedforward neural network) perform high-dimensional nonlinear transformations on the weighted query features to achieve dynamic representation fusion across experts. The following will provide a detailed introduction to the core components, the expert network (Expert) and the gated routing network (Router), whose structures are as follows: Figure 4 As shown.

[0083] 1) Expert Network

[0084] Each expert Defined as a two-layer fully connected feedforward network, it performs dimensionality mapping on the input features through a linear transformation. First, it maps the input features... The 1D feature is projected into the higher-dimensional latent space through the first fully connected layer. This expands the feature representation space and enhances nonlinear modeling capabilities, and is then mapped to the output dimension via a second fully connected layer. This achieves linear recombination of features and dimensional adaptation. The operation is shown in the following formula:

[0085] in, This represents the weights and biases of the first fully connected layer. This indicates the second fully connected layer and the bias. For non-linear activation functions, this paper uses the ReLU function. To mitigate fitting, a random inactivation mechanism was implemented, with the inactivation rate set to p=0.1. This represents the query features after being fused by self-Attention and cross-Attention.

[0086] 2) Gated routing network To efficiently select the most suitable sub-expert to handle different query features and reduce redundant computation, we designed a lightweight, differentiable gated network and a dynamic routing method based on Top-k routing. Specifically, the query representation after self-attention weighting... Then, the query features after cross-attention fusion processing are... The gating network generates expert selection weight scores for each query feature and calculates the corresponding scores. .

[0087]

[0088]

[0089]

[0090] in, This indicates the query output optimized by the IoU-aware query selection mechanism. Indicates by Keys and values ​​obtained through linear projection. The dimensions of attention head; From Keys and values ​​generated by linear projection The parameter represents the learnable linear mapping, and E is the number of experts; Dropout introduces regularization in the gated routing phase; This means injecting Gaussian noise with a trainable scale into the gated routing output to break the fixed pattern of routing selection and improve generalization ability; It is used to normalize the score into a probability distribution, ensuring differentiability and facilitating gradient propagation. This is a general representation of a gated routing network.

[0091] Next, to improve inference efficiency and enhance the specialization of route selection, we further apply a TopK sparse selection strategy to the probability distribution, retaining only those with the highest scores. Several experts participate in the routing of the current query features, and the index of these experts is subsequently obtained. And generate expert sparse masks. The specific formula is shown below:

[0092]

[0093] in, Returns the indexes of the k experts with the highest weighted scores.

[0094] Finally, to ensure that the sum of the activated expert weights is 1, we normalize the scores of the retained experts, so that the final gated route weights used for weighted aggregation are... Subsequently, the decoder output is a weighted sum of the selected expert outputs, that is, the corresponding activated expert outputs are weighted and summed using the normalized gating scores as weights, forming a cross-expert dynamic fusion representation. The specific formula is as follows:

[0095]

[0096] in, It is a numerically stable term that prevents division by zero. This represents the gated route weight corresponding to the j-th expert. This represents the forward inference result of the j-th expert.

[0097] This invention can effectively overcome the limitations of traditional manual surveys, which are time-consuming, labor-intensive, and highly subjective, and provides reliable data support for the precise screening of large-scale breeding populations, optimization of field management, and analysis of genotype-environment interactions.

[0098] To verify the effectiveness of this invention, we conducted comparative verification on two datasets (corn seedling dataset and GWHD dataset). The methods used in the comparison included mainstream DETR models such as DINO, DAC-DETR, RT-DETR, DN-DETR, and Relation-DETR.

[0099] Table 1 summarizes the performance comparison of different object detection methods on the maize seedling dataset, with the best index highlighted in bold. The results show that the proposed MoE-DETR significantly outperforms existing mainstream DETR detection models under the ResNet-50 backbone network condition, verifying the superior detection capability of the proposed method for small and fine-grained targets in complex farmland scenarios.

[0100] Specifically, MoE-DETR achieves an overall mAP of 93.2%, surpassing methods based on larger capacity backbone networks such as DINO (Swin-L, 82.1%) and RT-DETR (ResNet-101, 91.1%), fully demonstrating the effectiveness and generalization advantages of the network structure designed in this paper.

[0101] Table 1. Performance comparison of different object detection methods on the agricultural maize seedling dataset.

[0102] All models in Table 1 primarily use ResNet-50 as the backbone network, and are compared fairly with 300 training batches. Results for large backbone networks such as Swin-L and ResNet-101 are also listed for reference.

[0103] It is worth noting that existing methods generally have significant shortcomings in small target detection (APS). When the target size is small and the spacing is close, traditional feature fusion strategies struggle to effectively preserve high-frequency details, leading to blurred boundaries or missed detections. In contrast, MoE-DETR achieves joint spatial and frequency domain modeling through WFFM, using discrete wavelet transform to separate and enhance high-frequency components, thus mitigating the feature degradation problem of small targets while preserving edge and texture details. Simultaneously, DHAM dynamically constructs local hypergraph structures to explicitly model high-order semantic relationships between seedlings, significantly enhancing contextual discrimination capabilities in densely occluded scenes. Therefore, MoE-DETR achieves an excellent performance of 92.6% in APS, outperforming DINO (Swin-L, 74.8%) by 17.8 percentage points, DN-DETR (71.6%) by 21.0 percentage points, and RT-DETR (ResNet-50, 85.6%) by 7.0 percentage points, demonstrating absolute advantage in extremely fine-grained recognition tasks.

[0104] At the decoder end, the S-MoE Decoder achieves query-level adaptive representation learning through an IoU-aware query selection mechanism and a sparse hybrid expert structure. This structure can dynamically activate dedicated expert subnetworks according to the target scale and spatial distribution, decoupling different task representations and improving the feature specialization and nonlinear fitting ability in the decoding process, which is particularly beneficial for the accurate localization of dense and morphologically similar maize seedlings. In contrast, traditional DETR variants such as DINO and DN-DETR, although introducing denoising training or anchor priors to accelerate convergence, lag behind MoE-DETR in APS by 17.8 and 20.5 percentage points respectively (comparing DINO Swin-L and DN-DETR ResNet-50), indicating that they are still limited by a fixed query-feature interaction mode and have limitations in fine-grained discrimination; while real-time models such as RT-DETR may sacrifice some contextual modeling depth in pursuit of efficiency, resulting in more false negatives (FN) and false positives (FP) in high-density seedling areas.

[0105] It should be noted that although MoE-DETR is slightly lower than RT-DETR (ResNet-101, 72.0%) by 1.6 percentage points in large target detection (APL=70.4%), its overall performance under a lightweight backbone still surpasses most heavyweight models, and improves by 5.8, 3.0, and 0.1 percentage points compared to DINO (ResNet-50), DAC-DETR, and RT-DETR (ResNet-50), respectively. In summary, MoE-DETR, through its three core designs—multi-scale frequency fusion, high-order context enhancement, and decoder representation decoupling—effectively addresses the core challenges of small target detection in agricultural scenarios, providing reliable technical support for fine-grained visual tasks such as crop phenotypic analysis.

[0106] On the other hand, embodiments of the present invention also provide a dense overlapping target detection system based on a wavelet-enhanced sparse hybrid expert model, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the dense overlapping target detection method based on the wavelet-enhanced sparse hybrid expert model as described in the above technical solution.

[0107] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A densely overlapping target detection method based on a wavelet-enhanced sparse hybrid expert model, characterized in that, Includes the following steps: Step 1: Extract multi-level pyramid features from the input image; Step 2: Wavelet frequency domain feature modulation module (WFFM) is introduced into each level of features. Spatial features are decomposed into the frequency domain through discrete wavelet transform, realizing the collaborative modeling and reconstruction of frequency domain and spatial domain features. Subsequently, dynamic hypergraph attention module (DHAM) is integrated on the deepest feature layer. Hypergraph structure is adaptively constructed to explicitly model high-order semantic associations in local regions. The multi-scale features enhanced by WFFM and DHAM are fed into a bidirectional feature fusion path composed of a feature pyramid network and a path aggregation network, realizing multi-scale feature aggregation that combines top-down and bottom-up approaches. The wavelet frequency domain feature modulation module (WFFM) includes: firstly, obtaining the low-frequency approximation of the backbone features and multi-directional high-frequency details through high- and low-frequency feature decoupling; then, compressing and fusing them in the channel dimension during the cross-frequency fusion stage to form a multi-scale representation that combines global structure and local details. The Dynamic Hypergraph Attention Module (DHAM) consists of two parts: Hypergraph-based Adaptive Relevance Enhancement (HG-ACE) and a Transformer encoder. Step 3: Construct a decoder based on a sparse hybrid expert structure to obtain higher-level features with more discriminative and semantic consistency, which are then input into the prediction head module to complete the accurate regression and determination of the target location box; The decoder based on sparse hybrid expert architecture includes IoU-aware query selection and multiple sparse hybrid expert architecture modules (S-MoE Blocks). The IoU-aware query selection is used to filter and reweight candidate queries before they enter S-MoE Blocks, thereby improving the quality of the decoder input. The process of IoU-aware query selection is as follows: From fusion characteristics Extract the first Each query corresponds to a bounding box. The local feature response is used to obtain the local feature vector. Subsequently, based on the maximum IoU between the predicted bounding box and the ground truth annotation and its value in the data, the maximum IoU between the predicted bounding box and the ground truth annotation was determined. Semantic saliency generates importance weights Finally, the optimized query representation is output, which integrates contextual features and dynamically adjusts them according to weights, gradually suppressing low-quality queries to a mask m; the specific formula is as follows: In the formula, This is a bilinear sampling feature alignment operation. It is the aggregated feature from the output of step 2. For the first Each query corresponds to a bounding box; For the Sigmoid activation function, , Both represent learnable scaling factors. Represents the set of actual bounding boxes. The maximum intersection-union ratio; Denotes the L2 norm; For query The gating coefficient, For the linear transformation matrix of query-feature fusion, This indicates a feature-level concatenation operation; It is the query vector of the original input object. It is a learnable invalid query mask. This indicates the query output after optimization by the IoU-aware query selection mechanism.

2. The dense overlapping target detection method based on wavelet-enhanced sparse hybrid expert model as described in claim 1, characterized in that: In step 1, PREsNet is used as the backbone network to extract the four-level pyramid features of the input image. The backbone network first performs preliminary downsampling and feature encoding on the input image through a 7×7 convolutional layer and a max pooling layer to obtain low-level spatial texture information. Then, after four stages, each stage is composed of several residual block bottlenecks stacked together, and channel matching and spatial downsampling are completed through downsampling convolution when entering the next stage.

3. The densely overlapping target detection method based on wavelet-enhanced sparse hybrid expert model as described in claim 1, characterized in that: High- and low-frequency feature decoupling specifically includes: After decoupling the output feature layer using a single-layer two-dimensional discrete wavelet AFB2D, low-frequency components and high-frequency components in three directions are obtained. Single-layer two-dimensional discrete wavelet AFB2D applies low-pass and high-pass filters to the input features in the horizontal and vertical directions, respectively, and then performs a 2:1 downsampling operation on the results. The low-frequency components retain the main structure and overall texture information of the input features, and are a smooth approximation of the image. Provides detailed information at the horizontal, vertical, and diagonal levels; cross-frequency fusion includes concatenating the low-frequency and three high-frequency components along the channel dimension to obtain the fused features. Subsequently, the introduction Convolutional layers, batch normalization, and ReLU activation functions are used for feature compression and fusion to obtain a fused feature map. ; Furthermore, bilinear interpolation is used to upsample the fused features back to the spatial resolution of the original input feature map, thereby maintaining the scale alignment of the multi-scale feature pyramid. The specific operation is as follows: in, The final output feature map has a spatial resolution aligned with the original input feature map. Indicates an upsampling operation. This indicates the height and width of the upsampled feature map, where These represent the feature map height and width, respectively.

4. The densely overlapping target detection method based on wavelet-enhanced sparse hybrid expert model as described in claim 1, characterized in that: The adaptive correlation enhancement HG-ACE based on hypergraphs specifically includes: First, construct the distance metric matrix: given the input feature layer as... ,in For batch size, Indicates the number of channels. The spatial dimensions of the feature are first flattened into a sequence form, and the similarity between any two nodes is calculated to form a distance matrix. : in, The total number of tokens. For feature dimension, This represents a one-dimensional flattening feature sequence operation. Indicates token and The cosine distance, i.e., the distance metric matrix between all position tokens; Subsequently, a neighbor weight matrix is ​​constructed: based on a threshold number of neighbors K, a differentiable feature similarity metric is used to adaptively filter neighbors, resulting in a neighbor set. Furthermore, a lightweight two-layer attention network was designed to calculate the weights, ultimately obtaining the neighbor attention weights. All neighbor weights constitute the neighbor attention matrix. The specific operation is shown in the following formula: in, Represents an adaptive neighbor selection function that measures the similarity of differentiable features. Represents the set of all neighbors of node i; These are the normalized attention weights, representing the neighboring nodes. For the central node Contribution For query and key projection; Let be the learnable projection matrix, and d be the attention dimension; Finally, a self-learning hypergraph structure is constructed: a locally dynamic hypergraph is built, with each central hyperedge... Connecting nodes It and its K nearest neighbors are used to capture non-Euclidean associations in local regions. Neighbor features are injected into the central node through weighted aggregation. Finally, the combined results of aggregating all tokens yield the HG-ACE output. The specific operation is shown in the following formula: in, It is a shared feedforward network used for nonlinear mapping; For adaptive learning of transformable matrices, The attention weights are obtained through dynamic learning using a lightweight attention network. It is a learnable transformation matrix.

5. The densely overlapping target detection method based on wavelet-enhanced sparse hybrid expert model as described in claim 4, characterized in that: The specific process of the Transformer encoder is as follows: First, positional encoding is introduced into the features of the dynamic hypergraph enhancement output. get Next, the single-head attention mechanism calculates the similarity between the query matrix and the key matrix using dot product to measure the relevance between nodes. Then, it normalizes this similarity using the Softmax function to map it into an attention weight distribution, which is then applied to the value matrix to obtain the output of the single-head attention mechanism. Finally, construct multi-head attention as Finally, after being concatenated with the residuals and normalized, the output is... .

6. The densely overlapping target detection method based on wavelet-enhanced sparse hybrid expert model as described in claim 1, characterized in that: The processing steps for each sparse hybrid expert structure module are as follows: First, the input query is modeled using a self-attention mechanism to capture the dependencies between different queries; then, multi-scale feature information is fused using a cross-attention mechanism to obtain a context-enhanced query representation; next, the input is fed into a gated routing network (Router), which learns the weight distribution of each query among different experts and introduces Dropout and learnable noise into the output probability distribution to improve robustness; finally, multiple expert networks (Expert) perform high-dimensional nonlinear transformations on the weighted query features to achieve dynamic representation fusion across experts.

7. The densely overlapping target detection method based on wavelet-enhanced sparse hybrid expert model as described in claim 6, characterized in that: The processing procedure for expert networks is as follows: Each expert Defined as a two-layer fully connected feedforward network, it performs dimensionality mapping on the input features through a linear transformation. First, it maps the input features... The 1D feature is projected into the higher-dimensional latent space through the first fully connected layer. This expands the feature representation space and enhances nonlinear modeling capabilities, and is then mapped to the output dimension via a second fully connected layer. This is to achieve linear recombination of features and dimensional adaptation; the operation is shown in the following formula: in, Represented as the weights and biases of the first fully connected layer. This indicates the second fully connected layer and the bias. For non-linear activation functions, use the ReLU function; To mitigate overfitting, a random deactivation mechanism is employed. This represents the query features after the fusion of self-Attention and cross-Attention.

8. The densely overlapping target detection method based on wavelet-enhanced sparse hybrid expert model as described in claim 1, characterized in that: The processing procedure for a gated routing network is as follows: Query representation after self-attention weighting Then, the query features after cross-attention fusion processing are... The gating network generates expert selection weight scores for each query feature and calculates the corresponding scores. ; in, This indicates the query output optimized by the IoU-aware query selection mechanism. Indicates by Keys and values ​​obtained through linear projection; Dimensions representing the attention head; From Keys and values ​​generated by linear projection The parameter represents the learnable linear mapping, and E is the number of experts; Dropout introduces regularization in the gated routing phase; This means injecting Gaussian noise with a trainable scale into the gated routing output; Used to normalize scores into a probability distribution This is a general representation of a gated routing network. Next, a TopK sparse selection strategy is applied to the probability distribution, retaining only those with the highest scores. Several experts participate in the routing of the current query features, and the index of these experts is subsequently obtained. And generate expert sparse masks. The specific formula is shown below: in, Return the indexes of the k experts with the highest weighted scores; Finally, the scores of the retained experts are normalized so that the final gated route weights used for weighted aggregation are: Subsequently, the weighted sum of the selected expert outputs is output, that is, the weighted sum of the corresponding activated expert outputs is performed using the normalized gating scores as weights, forming a representation of cross-expert dynamic fusion. The specific formula is shown below: in, It is a numerically stable term that prevents division by zero. This represents the gated route weight corresponding to the j-th expert. This represents the forward inference result of the j-th expert.

9. A densely overlapping target detection system based on a wavelet-enhanced sparse hybrid expert model, characterized in that: It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to call the program instructions in the memory to execute the dense overlapping target detection method based on the wavelet-enhanced sparse hybrid expert model as described in any one of claims 1-8.

Images