A method and system for UAV target detection based on visual state space model and frequency domain perception.

Abstract

This invention provides a method and system for UAV target detection based on a visual state space model and frequency domain awareness, belonging to the field of UAV target detection technology. It includes: establishing a UAV target detection framework based on a visual state space model and frequency domain awareness, consisting of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head; the backbone network uses alternating convolutional blocks and cross-stage visual state space blocks (CSVSS) to extract multi-scale features; the hybrid encoder uses a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence, and the cross-scale fusion unit uses a dual-domain representation complementary module (DRCM) with branches in both the spatial and frequency domains; the decoder with an auxiliary prediction head filters the feature sequence to obtain an initial target query, and completes target classification and localization using the initial target query; and target detection is performed using the UAV target detection framework.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target detection technology, and in particular to a UAV target detection method and system based on a visual state space model and frequency domain perception. Background Technology

[0002] With the development and advancement of computer vision technology, the use of drones equipped with cameras for target detection and tracking has been widely applied in military and civilian fields such as terrain surveying, traffic monitoring, disaster relief, and military reconnaissance. Therefore, analyzing and processing video targets captured by drones is a crucial component. Target detection, as the core task of intelligent visual perception in drones, directly determines the performance of higher-level applications such as scene understanding and path planning. Although deep learning has significantly improved the performance of general-purpose target detection, existing methods are difficult to directly adapt to Unmanned Aerial Vehicle Object Detection (UAVOD) tasks. Compared to conventional ground detection, UAVOD faces unique challenges such as a high proportion of small targets, dense target density with large scale differences, and complex and variable backgrounds, which greatly limits the application effectiveness of general-purpose detectors.

[0003] Current mainstream detectors can be broadly categorized into two types: The first type is traditional detectors, represented by Faster R-CNN and the YOLO series. These methods generally rely on hand-designed anchor boxes and non-maximum suppression (NMS) post-processing. While each method has its own advantages in terms of speed and accuracy trade-offs, they all face challenges in UAVOD scenarios, such as difficulty in setting anchor box scales and sensitivity to NMS thresholds. The second type is end-to-end detectors, represented by DETR. This type of method models the detection task as an ensemble prediction task, using a Transformer encoder to perform global contextual modeling of backbone features. It can explicitly capture long-range dependencies and eliminates the need for anchor box design and NMS post-processing, demonstrating advantages in both modeling capabilities and detection process compared to traditional detectors. However, early DETR methods suffered from slow convergence speed and high computational overhead. Subsequent works such as Deformable DETR and DINO have further optimized efficiency and accuracy through deformable attention and contrastive denoising mechanisms. Nevertheless, these methods still have high computational overhead and slow inference speed, limiting their application in real-time detection scenarios. Although the end-to-end detection paradigm effectively enhances the modeling of global dependencies between features by using self-attention mechanisms in the encoder part, the backbone networks of the two types of detectors mentioned above still generally adopt the traditional CNN architecture. While the local inductive bias of CNNs helps to capture fine-grained structures, its characteristic of layer-by-layer stacking expansion of receptive field dependencies faces significant challenges in UAVOD scenarios: targets in the high-altitude view are densely distributed and extremely small in scale, and CNNs have difficulty effectively capturing long-range dependencies across regions, resulting in small targets lacking sufficient global contextual cues, which easily leads to false detections and false negatives in complex backgrounds and dense scenes. Summary of the Invention

[0004] In view of this, this invention proposes a UAV target detection method and system based on a visual state space model and frequency domain perception. By introducing a hybrid visual state space module (HVSS) and a cross-stage visual state space block (CSVSS), global dependency modeling and local detail extraction are achieved while ensuring linear computational complexity. By fusing spatial and frequency domain information through a dual-domain representation complementarity module (DRCM), the feature representation of small targets is enhanced, thereby effectively improving the accuracy and robustness of small target detection in complex UAV scenarios and reducing false detections and false negatives.

[0005] The technical solution adopted by the embodiments of the present invention to solve its technical problem is as follows:

[0006] The first aspect of this invention provides a method for UAV target detection based on a visual state space model and frequency domain perception, comprising:

[0007] A UAV target detection framework based on a visual state space model and frequency domain perception is established, consisting of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head.

[0008] The backbone network uses alternating and continuous convolutional blocks and cross-stage visual state space blocks (CSVSS) to extract multi-scale features. The CSVSS blocks employ dual branches for representation enhancement, with global branches and convolutional local branches working in parallel and complementing each other during feature extraction.

[0009] The hybrid encoder uses a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence. The cross-scale fusion unit adopts a dual-domain representation complementary module DRCM with spatial and frequency domain branches. The spatial branch uses multi-scale convolution and channel attention mechanism to preserve position and structural information, while the frequency domain branch enhances high-frequency details and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectrum modulation. The fusion of the two features forms a complementary target representation.

[0010] A decoder with an auxiliary prediction head filters feature sequences to obtain an initial target query, and then the decoder performs target classification and localization on the initial target query.

[0011] Acquire images from drone application scenarios, establish a training set, and train the drone target detection framework.

[0012] Target detection is performed using the aforementioned UAV target detection framework.

[0013] Preferably, the training of the UAV target detection framework includes:

[0014] Step S1: The training data is fed into the backbone network and extracted by each CSVSS block to obtain global-local complementary multi-scale features {F2,F3,F4,F5}, which are then fed into the hybrid encoder.

[0015] In step S2, the hybrid encoder uses a self-attention mechanism to perform intra-scale interaction on the highest-level feature F5 to obtain F5′; F5′ is dynamically fused with {F2,F3,F4} across scales to obtain a feature sequence; wherein, the input of DRCM includes F2, F3 and the upper-level fused feature P1, P1 is the fusion result of F5′ and F4, and the output of DRCM is used as a component of the feature sequence.

[0016] Step S3: The feature sequence is used as a candidate set for the target query, and a certain number of target queries are selected as the initial target query input decoder. The decoder performs multi-layer iterative decoding processing on the target query. Through multi-layer decoding operations, the target query is updated in terms of features and corrected in terms of position. The semantic information and spatial position information of the target are optimized layer by layer to obtain the target detection box. The detection box information includes target category information and target bounding box position information, which are used to represent the category to which the target belongs and the spatial position of the target in the input image, respectively.

[0017] Step S4: Construct a loss function based on the difference between the prediction result and the actual annotation, and use the loss function to adjust the parameters of the UAV target detection framework until convergence or a stopping mechanism is triggered.

[0018] Preferably, the backbone network's CSVSS uses a dual-branch approach for representation enhancement. One branch serves as the identity mapping flow Sid, preserving the original information; the other branch serves as the representation enhancement flow Seh, which enhances representations through consecutively stacked Hybrid Visual State Space (HVSS) blocks. 1x1 convolutions are used for information exchange between channels and the number of channels is adjusted, serving as input for the next stage. Four consecutive CSVSSs sequentially extract features at different scales, resulting in global-local complementary multi-scale features {F2, F3, F4, F5}.

[0019] In CSVSS, the HVSS block performs channel expansion and segmentation on the input features to obtain the local branch X. local and global branch X global Two parallel branches model global semantics and local details respectively, and adaptive fusion is performed using learnable factors:

[0020] Local branches The local branch feature map is obtained by sequentially performing 1x1 convolution for channel reduction, 3x3 convolution for feature extraction, and 1x1 convolution for dimensionality increase followed by residual connection. ;

[0021] Global branch X global After stabilizing the distribution through layer normalization, the system proceeds in parallel to two branches: the gated branch, after layer normalization, generates a gated signal of the same shape as the main branch through a linear mapping and the SiLu activation function. The main branch sequentially executes linear projection, depthwise separable convolution, and SiLU activation function, then enters the 2D selective scanning SS2D unit to aggregate long-range context in a multi-path structure-aware manner. Finally, layer normalization is performed to obtain a preliminary global representation. The global branch output is obtained by performing a gating operation through element-wise multiplication of the two branches, followed by linear mapping and residual connection. ;

[0022] Two sets of learnable parameters a, b∈R C Generate channel-wise weights using Softmax normalization , Using normalized weights and Channel-by-channel weighted fusion is performed to obtain adaptively fused features.

[0023] Preferably, step S2, which involves dynamic cross-scale feature fusion, includes:

[0024] The upsampled feature F5′′ and feature F4 are used to obtain feature F through the Fusion module. 5'4 Feature F 5'4 The upsampled features P1, F3, and F2 are input into the DRCM to obtain y3; y3 is processed by the downsampled module to obtain y3′; y3′ and P1 are processed by the Fusion module to obtain y2; y2 is processed by the downsampled module to obtain y2′; y2′ and F5′′ are processed by the Fusion module to obtain y1; the features y3, y2, and y1 are fused and concatenated into a feature sequence.

[0025] In DRCM:

[0026] Features P1, F3, and F2 are aligned to the same resolution and pre-integrated to obtain a unified representation Z. S Based on the channel, it is divided into spatial branch and frequency domain branch, forming the characteristics of spatial branch. and frequency domain features X f ;

[0027] Spatial branching employs parallel 3x3, 3x3 (hole rate 2), and 7x7 convolutional layers to capture structural information under different receptive fields, fusing them to obtain features processed by multi-bandpass operators. ;right Global average pooling and 1x1 convolution are used to generate channel descriptors. The features are then reweighted and fused with the original features to form the output of the spatial branch. ;

[0028] Frequency domain branching transforms X using a two-dimensional fast Fourier transform. f Mapping to the frequency domain, the low-frequency mask is obtained through a Gaussian function. and high-frequency mask ;use , For frequency domain features X f Element-wise modulation is performed to obtain decoupled high-frequency components. Low-frequency components ,right , Filtering enhancement is performed to obtain , The changes in the filtering results are adaptively weighted to generate enhanced frequency domain features. , Obtained by inverse Fourier transform As the output of the frequency domain branch;

[0029] splicing in the channel dimension and Cross-domain projection and channel compression are performed through 1x1 convolution, and residual connections are added to obtain y3.

[0030] Preferably, in step S4, the loss function used during training... Loss from target classification Bounding box regression loss Bounding box overlap loss composition:

[0031] ;

[0032] ;

[0033] ; ;

[0034] ;

[0035] In the formula, This represents the model's predicted probability of the true class. The class balance factor is represented by γ, and the modulation factor is represented by γ; the predicted bounding box is represented as The true bounding box is represented as D indicates that it contains both and The minimum bounding rectangle; , , These are preset weighting coefficients.

[0036] Ideally, in DRCM, the number of frequency domain branch channels is... The number of spatial branch channels is , is the channel scaling factor, and C is the number of feature channels.

[0037] A second aspect of the present invention provides a UAV target detection system based on a visual state space model and frequency domain perception, the system comprising:

[0038] The module is used to establish a UAV target detection framework based on a visual state space model and frequency domain perception. It consists of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head.

[0039] The backbone network uses alternating and continuous convolutional blocks and cross-stage visual state space blocks (CSVSS) to extract multi-scale features. The CSVSS blocks employ dual branches for representation enhancement, with global branches and convolutional local branches working in parallel and complementing each other during feature extraction.

[0040] The hybrid encoder uses a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence. The cross-scale fusion unit adopts a dual-domain representation complementary module DRCM with spatial and frequency domain branches. The spatial branch uses multi-scale convolution and channel attention mechanism to preserve position and structural information, while the frequency domain branch enhances high-frequency details and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectrum modulation. The fusion of the two features forms a complementary target representation.

[0041] A decoder with an auxiliary prediction head filters feature sequences to obtain an initial target query, and then the decoder performs target classification and localization on the initial target query.

[0042] The training module is used to acquire images in UAV application scenarios, establish a training set, and train the UAV target detection framework.

[0043] The detection module is used to perform target detection using the UAV target detection framework.

[0044] As can be seen from the above technical solution, the UAV target detection method based on visual state space model and frequency domain perception provided by the embodiments of the present invention first establishes a UAV target detection framework based on visual state space model and frequency domain perception, which consists of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head. The backbone network uses alternating and continuous convolutional blocks and cross-stage visual state space blocks (CSVSS) to extract multi-scale features. The CSVSS block uses a dual-branch approach for representation enhancement, and the global branch and the local convolutional branch are parallel and complementary during the feature extraction process. The hybrid encoder uses a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence. The cross-scale fusion unit employs a dual-domain representation complementarity module (DRCM) with spatial and frequency domain branches. The spatial branch utilizes multi-scale convolution and channel attention mechanisms to preserve positional and structural information, while the frequency domain branch enhances high-frequency details and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectral modulation. The fusion of these two feature paths forms a complementary target representation. A decoder with an auxiliary prediction head filters feature sequences to obtain an initial target query, and the decoder performs target classification and localization on the initial target query. Images from UAV application scenarios are acquired, a training set is established, and the UAV target detection framework is trained. Target detection is then performed using the UAV target detection framework. This invention introduces a hybrid visual state space module (HVSS) and a cross-stage visual state space block (CSVSS) to achieve global dependency modeling and local detail extraction while maintaining linear computational complexity. By fusing spatial and frequency domain information through the dual-domain representation complementarity module (DRCM) to enhance the feature representation of small targets, the accuracy and robustness of small target detection in complex UAV scenarios are effectively improved, reducing false detections and false negatives. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the overall architecture of VSFPDet in this invention.

[0046] Figure 2 This is a schematic diagram of SS2D operation.

[0047] Figure 3 A schematic diagram illustrating the spatial-frequency domain representation differences in a UAVOD scene.

[0048] Figure 4 Radar charts of different AP50 types on VisDrone.

[0049] Figure 5 This is a comparison chart of the ERF of Resnet-18 and the CSVSS-based backbone network.

[0050] Figure 6 This is a schematic diagram comparing the heatmaps of the baseline method and the method of the present invention.

[0051] Figure 7This is a comparison of the detection results of the baseline method and the method of the present invention. Detailed Implementation

[0052] The technical solution and effects of the present invention will be further described in detail below with reference to the accompanying drawings.

[0053] Unmanned Aerial Vehicle Object Detection (UAVOD) has wide applications in emergency response, traffic management, and other fields. However, existing UAVOD methods are limited by the local receptive field of CNNs, making it difficult to model long-distance correlations. This results in insufficient global perception of small targets, leading to missed or false detections. Furthermore, high-frequency details such as edges and textures of small targets decay layer by layer during multiple downsampling, resulting in weak responses to deep features and further limiting detection performance. To address these challenges, this invention proposes the VSFPDet detection framework, employing a visual state space model and a frequency domain perception strategy. First, a Hybrid Visual State Space Module (HVSS) is designed as the core unit to achieve a unified modeling of long-range dependencies and extraction of local details with linear complexity. Based on this, a Cross-Stage Visual State Space Block (CSVSS) is designed to construct the backbone network, obtaining more efficient global-local complementary representations. Second, a Dual-Domain Representation Complementary Module (DRCM) is proposed to coordinate spatial and frequency domain information during the cross-semantic integration stage, improving the visibility of small targets and stabilizing position perception.

[0054] refer to Figure 1 As shown, this invention provides a method for UAV target detection based on visual state space models and frequency domain perception (VSPDet), comprising:

[0055] A UAV target detection framework based on a visual state space model and frequency domain perception is established, consisting of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head.

[0056] Acquire images from drone application scenarios, establish a training set, and train the drone target detection framework.

[0057] Target detection is performed using a drone target detection framework.

[0058] The overall structure of VSFPDet is as follows: Figure 1As shown, the model follows the paradigm of the real-time end-to-end detector RT-DETR, consisting of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head. Considering the characteristics of UAV scenarios where distant small targets require long-range semantic association and dense distribution requires fine local discrimination, this invention has made targeted designs at two levels: feature extraction and cross-scale semantic integration. In the feature extraction stage, to address the problem of insufficient global modeling capability of CNN, this invention proposes an HVSS module, which achieves global-local complementary modeling through the parallel connection of Mamba global branches and convolutional local branches; then, the HVSS is organized using the idea of ​​cross-stage partial connectivity to form a cross-stage visual state space block (CSVSS) as the basic building block of the backbone network. By stacking multiple CSVSS blocks, global-local complementary multi-scale features {F2,F3,F4,F5} are obtained; in the semantic integration stage, the hybrid encoder uses a self-attention mechanism to perform intra-scale interaction on the highest-level feature F5 to obtain F5′, and then F5′ is fused with {F2,F3,F4} across scales. To alleviate the weak response of single spatial domain modeling to small targets, this invention proposes DRCM, which projects features simultaneously into both the spatial and frequency domains for parallel processing. High-frequency components in the frequency domain explicitly enhance the edges and texture details of small targets, while low-frequency components provide a stable global semantic context, compensating for the spatial domain representation's inability to perceive weak signals. The integrated features are then fed into a decoder with an auxiliary detection head to obtain the detection results. This design enables VSFPDet to effectively balance long-range semantic modeling and local detail capture while maintaining computational efficiency, thus improving detection performance in UAV scenarios.

[0059] HVSS block:

[0060] The quality of the feature representation of the backbone network directly determines the performance limit of the detector. In view of the characteristics of small targets and large scale differences in UAVOD scene, the backbone network needs to realize global semantic modeling and local detail capture at the same time under the premise of limited on-board computing power. As mentioned in the introduction, the state space model Mamba provides a new idea for building an efficient backbone network with its linear complexity and long-range modeling capability. Existing visual Mamba modules generally use two-dimensional selective scanning [1] (2D-Selective-Scan, SS2D) as the core operation to realize the sequence modeling of two-dimensional features. Figure 2As shown (the blue blocks represent adjacent tokens in the input, which should maintain continuous local adjacency relationships. However, during the cross scan unfolding process, these adjacent tokens are scrambled and their original spatial structure information is destroyed), SS2D unfolds the two-dimensional feature map into a one-dimensional sequence in multiple directions through cross scan. After processing by the selective state space model (S6) [2], it is then cross-merged to model the global receptive field. However, in the cross scan stage, the serialization process of multi-directional scanning inevitably destroys the spatial adjacency relationship, which weakens the modeling ability of fine-grained local features. This problem is particularly prominent in the UAVOD scene: densely distributed small targets not only rely on long-range semantic association to distinguish between the target and the background, but also need fine local discrimination ability to distinguish neighboring targets with similar appearances.

[0061] To address the aforementioned problems, this invention proposes a Hybrid Visual State Space Block (HVSS). This module compensates for the loss of local information caused by serialization at the architectural level by connecting convolutional branches and Mamba branches in parallel. The specific structure of HVSS is as follows: Figure 1 As shown in (a)(ii). First, in order to obtain richer information, HVSS performs channel expansion and segmentation on the input features, resulting in two parallel branches: the local branch X. local and global branch X global This invention employs a dual-branch design to model global semantics and local details separately, and finally uses a learnable factor for adaptive fusion. Specifically, for the local branch, to effectively model local details and control overhead, this invention uses a lightweight bottleneck structure for processing. First, 1x1 convolutions are used for channel dimensionality reduction, then 3x3 convolutions are used for feature extraction, and finally 1x1 convolutions are used for dimensionality increase and residual connections are established. The mathematical expression is shown in formula (1).

[0062] (1)

[0063] In the global branch, this invention utilizes the SS2D module for long-range dependency modeling. The operation steps of the SS2D module are as follows: Figure 2 As shown. Dense small targets in UAVOD often exhibit complex spatial distribution patterns. In order to enable the global branch to better capture this spatial topological relationship, this invention adopts the Structure-Aware Scanning Strategy (SASS) [4] to replace the original cross-scan. Figure 1As shown in (b)(iii), SASS contains two parallel serpentine paths and two diagonal serpentine paths. By maintaining the continuity of scanning in multiple directions, it can more completely model the spatial layout of dense targets and provide a more stable global representation for dual-branch fusion.

[0064] Specifically, X global First, the distribution is stabilized through layer normalization, and then two branches are entered in parallel: the gated branch generates a gated signal with the same shape as the main branch through linear mapping and SiLu activation function after layer normalization; the main branch sequentially performs linear projection, depthwise separable convolution and SiLu activation function, and then enters the core SS2D to aggregate long-range context in a multi-path structure-aware manner, and then performs layer normalization at the end to obtain a preliminary global representation; the two branches perform element-wise multiplication to realize the gating operation, selectively enhance or suppress the features of the main branch, and then integrate them through linear mapping and perform residual connection to obtain the output of the main branch. The specific mathematical expression is shown in formulas (2)(3)(4)(5).

[0065] (2)

[0066] (3)

[0067] (4)

[0068] (5)

[0069] in, Indicates the output of the gated branch. and Indicates intermediate output. This indicates an element-wise multiplication operation. This represents the output of the final global branch. LN() represents the layer normalization operation, and Linear() represents the linear mapping operation.

[0070] To achieve adaptive fusion of global and local features, this invention uses two sets of learnable parameters a, b∈R C Generate channel-wise weights using Softmax normalization , :

[0071] (6)

[0072] All of the above operations are element-wise operations, where 1 represents a vector of all 1s; express, express and These represent element-wise exponentiation operations on a and b, respectively. Then, the two features are fused using normalized weights on a channel-wise weighted basis.

[0073] (7)

[0074] in, Refers to the features after adaptive fusion. This represents element-wise multiplication (the weight vector is broadcast along the spatial dimension). Through end-to-end training, this mechanism can learn the relative importance of local and global features in different channels, achieving a balance between detail representation and semantic understanding.

[0075] CSVSS block:

[0076] HVSS achieves complementary modeling of global and local data through a dual-branch design. However, its parallel structure requires all input features to be processed through two complete branches simultaneously, which leads to significant computational overhead as the number of channels increases in deep networks. In addition, dense branch connections also cause gradients to accumulate and decay during backpropagation, affecting the learning effect of deep features. To solve the above problems, this invention introduces the cross-stage partial connection idea of ​​CSPNet[3] to organize multiple HVSS blocks. The core mechanism of CSPNet is to bypass dense computation modules and reach the output directly through cross-stage splitting, thereby reducing redundant computation and improving gradient propagation while preserving representation capabilities. Based on this idea, this invention proposes the cross-stage visual state space block CSVSS and uses CSVSS as the basic building block of the backbone network.

[0077] The specific structure of the CSVSS block is as follows: Figure 1 As shown in (a)(i), specifically, given input features X∈R H×W×C First, the channels are expanded to 2C through a 1×1 convolution, and then divided into two branches in the channel dimension: one branch, denoted as Sid, serves as the identity mapping stream and retains the original information; the other branch, denoted as Seh, serves as the representation enhancement stream and is processed by m stacked HVSS blocks for representation enhancement. After processing by multiple HVSS blocks, Sid and the final outputs of the m HVSS blocks are concatenated in the channel dimension. Then, a 1×1 convolution is used to exchange information between channels and adjust the number of channels, which is finally used as the input for the next stage. The mathematical expression of CSVSS is shown in formulas (8)(9)(10).

[0078] (8)

[0079] (9)

[0080] (10)

[0081] Split(⋅) means splitting the convolutional features into two parts along the channel dimension. Y refers to the feature after processing through m HVSS blocks, and Concat(⋅) represents the channel dimension concatenation operation; Y is the final output feature map.

[0082] The hybrid encoder utilizes a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence. The cross-scale fusion unit employs a dual-domain representation complementary module (DRCM) with spatial and frequency domain branches. The spatial branch uses multi-scale convolution and channel attention mechanisms to preserve positional and structural information, while the frequency branch enhances high-frequency details and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectral modulation. The fusion of the two feature paths forms a complementary target representation. To explore potential paths to alleviate the weakening of small target response, this invention conducts a comparative analysis of typical scenes from both spatial and frequency domain perspectives (where (a) the original image; (b) the high-frequency component, highlighting details such as edges and textures; (c) the spatial domain response heatmap after multi-scale feature fusion, showing weak activation of small targets in complex backgrounds; and (d) the low-frequency component, preserving global structural and semantic information). Figure 3 (a) Shows the original scene containing a large number of small, densely packed targets at a distance; Figure 3 (c) The multi-scale fusion heatmap shows that distant small targets are almost completely inactive, while large near-field targets remain clearly visible, indicating severe attenuation of small target features in deep networks. Traditional spatial domain modeling struggles to effectively preserve the detailed information of small targets while maintaining semantic abstraction capabilities. In contrast, frequency domain analysis provides a complementary perspective. Figure 3 As shown in (b), the high-frequency components encode detailed information such as edges and textures that change rapidly, which can highlight the local features of small targets; Figure 3 (d) The low-frequency components carry the overall structure and global semantics of the scene, providing a stable background context for small target feature enhancement. This leads to two insights: (i) explicit frequency domain decoupling can highlight details that are difficult to separate in the spatial domain, providing operational space for adaptive enhancement of small target responses; (ii) high and low frequency components respectively enhance detail capture and global modeling, and their synergy with spatial features can effectively compensate for the shortcomings of single spatial domain modeling in small target representation. Based on the above analysis, this invention proposes a Dual-domain Representation Complementary Module (DRCM). In the cross-scale semantic integration stage, DRCM explicitly introduces frequency domain representation as a supplement to spatial domain features: through parallel spatial-frequency domain modeling, the former retains positional sensitivity and structural information, while the latter adaptively enhances high-frequency details related to small targets and utilizes low frequencies to provide semantic context. The two synergistically form a complementary multi-view representation, enhancing the visibility and distinguishability of small targets.

[0083] The specific structure of DRCM is as follows Figure 1As shown in (b), to fully utilize the spatial details in the shallow features, this module explicitly introduces the highest resolution F2 from the backbone network and uses it, along with F3 and the upper-layer fused feature P1 (i.e., the fusion result of F5' and F4), as three inputs. DRCM first aligns the input features to the same resolution and pre-integrates them using 1×1 convolutions to obtain a unified representation Z. S ∈R CxHxW This invention integrates multi-level semantic and spatial information. To control costs and avoid redundancy, Zs is divided into spatial and frequency domain branches based on channels: Let the number of channels in the frequency domain branch be... ,the remaining Entering the spatial branch ( =0.25).

[0084] The design of the spatial branch aims to preserve accurate location-sensitive information and structural continuity. Specifically, this invention first uses parallel 3x3, 3x3 (dilation=2) and 7x7 convolutions to construct learnable multi-bandpass operators to capture structural information under different receptive fields in a hierarchical manner: the former focuses on local texture and fine-grained details, the medium-scale convolution models the nearest neighbor context, and the large-kernel convolution supplements low-frequency contours and shape priors. The parallel convergence of multi-scale responses enables the spatial branch to have the ability to jointly represent multi-granular structures. Subsequently, SE-style channel recalibration [5] is adopted, and global average pooling and 1x1 convolution are used to generate compact channel descriptors to adaptively measure the semantic contribution of each channel, and the features are reweighted accordingly, thereby highlighting the significant channels related to the target under the guidance of global dependence and suppressing redundant or noisy responses. This spatial domain modeling path obtains structural information under different receptive fields through multi-scale convolutions, and highlights significant responses related to small targets through channel-level attention mechanisms, providing stable and fine-grained spatial support for the frequency domain branch. The specific mathematical expression of this branch is shown in formulas (11) and (12).

[0085] (11)

[0086] (12)

[0087] in, Characteristics representing spatial branching, This represents the features after processing by the multi-bandpass operator. This indicates a global average pooling operation. This represents the sigmoid function. This indicates the final output of the spatial branch; This represents a dilated convolution with a dilation rate of 2. To compensate for the limitations of spatial domain representation, this invention introduces a frequency domain branch in parallel to achieve explicit frequency decoupling of the image signal. By adaptively modulating the high and low frequency energy distribution, it enhances the high-frequency response related to small targets, while utilizing low-frequency components to provide stable semantic context. Specifically, for the input feature X... f First, the frequency domain is mapped through a two-dimensional fast Fourier transform. In order to achieve smooth and continuous high and low frequency separation, the present invention uses a Gaussian function to construct a low frequency mask, as shown in formula (13).

[0088] (13)

[0089] in, Represents frequency coordinates. Indicates the center of the spectrum. Represents a two-dimensional Gaussian low-frequency mask in frequency coordinates The value at that location, The Gaussian bandwidth parameter controls the low-frequency retention range. For a fixed proportionality coefficient, These represent the spatial dimensions of the spectrum; This indicates an exponential operation. The corresponding high-frequency mask adopts a complementary design, defined as shown in formula (14).

[0090] (14)

[0091] By using the above mask to modulate the frequency domain features element-wise, the decoupled high and low frequency components can be obtained:

[0092] (15)

[0093] in, This represents the Fourier transform operator. This represents the frequency domain characteristics of the output. and They represent the high-frequency masks respectively. and low-frequency mask The high-frequency and low-frequency features are obtained. This process achieves continuous and smooth high- and low-frequency decoupling in the spectral space, avoiding artifacts caused by hard thresholding. Subsequently, to enhance the model's adaptive modulation capability for different frequency bands, a learnable frequency domain filter is introduced to enhance the high- and low-frequency components respectively:

[0094] (16)

[0095] in, , To create a learnable frequency domain filter, this invention uses a 3x3 depthwise convolution. , These represent the enhanced high-frequency and low-frequency characteristics, respectively. Based on this, to avoid excessive modification and instability caused by directly replacing frequency domain characteristics, this invention introduces a frequency band adaptive modulation mechanism. This mechanism uses the original frequency domain characteristics as a baseline and adaptively weights the changes in the enhanced high- and low-frequency filtering results to ensure the stability and controllability of the frequency domain enhancement. The process is shown in formulas (17) and (18).

[0096] (17)

[0097] (18)

[0098] in, , For learnable frequency band modulation coefficients, This represents the enhanced frequency domain characteristics. This represents the inverse Fourier transform operator. This represents the output of the frequency domain branch. This mechanism can be viewed as an adaptive energy redistribution of the original spectrum: by balancing high-frequency detail enhancement with low-frequency semantic preservation through learnable modulation coefficients, the stability and controllability of feature enhancement are ensured.

[0099] Finally, output the spatial branch. With frequency domain branch output The data is stitched together along the channel dimension, cross-domain projection and channel compression are performed using 1x1 convolution, and residual connections are added to preserve the original information, ultimately yielding the DRCM output. The mathematical expression is shown in formula (19).

[0100] (19)

[0101] In summary, DRCM achieves multi-view feature enhancement through parallel modeling in both the spatial and frequency domains. The spatial branch utilizes multi-scale convolution and channel attention mechanisms to preserve accurate location and structural information, while the frequency branch enhances high-frequency details related to small targets and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectral modulation. The fusion of the two feature paths forms a complementary target representation, effectively alleviating the problem of weak response of small targets and providing more robust feature support for small target detection.

[0102] The decoder with an auxiliary prediction head filters feature sequences to obtain an initial target query, and then completes target classification and localization based on the initial target query.

[0103] After preprocessing and labeling the collected source data, it is divided into training, validation, and test sets for training, validation, and testing, respectively. The training framework for UAV target detection includes:

[0104] Step S1: The training data is fed into the backbone network and extracted from each CSVSS block to obtain global-local complementary multi-scale features {F2, F3, F4, F5}, which are then fed into the hybrid encoder.

[0105] In step S2, the hybrid encoder uses a self-attention mechanism to perform intra-scale interaction on the highest-level feature F5 to obtain F5′; F5′ is dynamically fused with {F2,F3,F4} across scales to obtain a feature sequence; wherein, the input of DRCM includes F2, F3 and the upper-level fused feature P1, P1 is the fusion result of F5′ and F4, and the output of DRCM is used as a component of the feature sequence.

[0106] Step S3: The feature sequence is used as a candidate set for the target query, and a certain number of target queries are selected as the initial target query input decoder. The decoder performs multi-layer iterative decoding processing on the target query. Through multi-layer decoding operations, the target query is updated in terms of features and corrected in terms of position. The semantic information and spatial position information of the target are optimized layer by layer to obtain the target detection box. The detection box information includes target category information and target bounding box position information, which are used to represent the category to which the target belongs and the spatial position of the target in the input image, respectively.

[0107] Step S4: Construct a loss function based on the difference between the prediction result and the actual annotation, and use the loss function to adjust the parameters of the UAV target detection framework until convergence or a stopping mechanism is triggered.

[0108] Step S2, dynamic cross-scale feature fusion, includes:

[0109] The upsampled feature F5'' and feature F4 are used to obtain feature F through the Fusion module. 5'4 Feature F 5'4 The upsampled features P1, F3, and F2 are input into the DRCM to obtain y3; y3 is processed by the downsampled module to obtain y3'; y3' and P1 are processed by the Fusion module to obtain y2; y2 is processed by the downsampled module to obtain y2'; y2' and F5'' are processed by the Fusion module to obtain y1; the features y3, y2, and y1 are fused and concatenated into a feature sequence;

[0110] In DRCM:

[0111] Features P1, F3, and F2 are aligned to the same resolution and pre-integrated to obtain a unified representation Z. S Based on the channel, it is divided into spatial branch and frequency domain branch, forming the characteristics of spatial branch. and frequency domain features X f ;

[0112] Spatial branching employs parallel 3x3, 3x3 (dilation=2), and 7x7 convolutional layers to capture structural information under different receptive fields, fusing them to obtain features processed by multi-bandpass operators. ;right Global average pooling and 1x1 convolution are used to generate channel descriptors. The features are then reweighted and fused with the original features to form the output of the spatial branch. ;

[0113] Frequency domain branching transforms X using a two-dimensional fast Fourier transform. f Mapping to the frequency domain, the low-frequency mask is obtained through a Gaussian function. and high-frequency mask ;use , For frequency domain features X f Element-wise modulation is performed to obtain decoupled high-frequency components. Low-frequency components ,right , Filtering enhancement is performed to obtain , The changes in the filtering results are adaptively weighted to generate enhanced frequency domain features. , Obtained by inverse Fourier transform As the output of the frequency domain branch;

[0114] splicing in the channel dimension and Cross-domain projection and channel compression are performed through 1x1 convolution, and residual connections are added to obtain y3.

[0115] In step S4, the loss function used during training. Loss from target classification Bounding box regression loss Bounding box overlap loss composition:

[0116] (20)

[0117] (twenty one)

[0118] ; (twenty two)

[0119] (twenty three)

[0120] In the formula, This represents the model's predicted probability of the true class. The class balance factor is represented by γ, and the modulation factor is represented by γ; the predicted bounding box is represented as The true bounding box is represented as D indicates that it contains both and The minimum bounding rectangle; , , These are the preset weighting coefficients. (x, y, w, h) refer to the x-coordinate and y-coordinate of the center point, the width of the bounding box, and the height of the bounding box.

[0121] This invention provides a UAV target detection method and system based on a visual state space model and frequency domain awareness. It constructs a highly efficient backbone network that combines local detail capture and global context awareness. In the feature extraction stage, it introduces a Hybrid Visual State Space Module (HVSS) to achieve a unified approach to global dependency modeling and local detail extraction with linear complexity. It also constructs a Cross-Stage Visual State Space Block (CSVSS) to reduce redundant computation and improve gradient propagation. In the encoding stage, it introduces a Dual-Domain Complementary Representation Module (DRCM) to explicitly enhance the visibility and distinguishability of small targets. This fully leverages the advantages of the end-to-end paradigm in UAVOD tasks, addressing the problem that CNN architectures struggle to effectively capture long-range dependencies across regions in UAVOD scenarios, leading to insufficient global contextual cues for small targets and a high likelihood of false positives and false negatives in complex backgrounds and dense scenes.

[0122] This invention provides a UAV target detection system based on a visual state space model and frequency domain perception, for implementation Figure 1 The UAV target detection method shown is based on a visual state space model and frequency domain perception. The system includes:

[0123] The module is used to establish a UAV target detection framework based on a visual state space model and frequency domain perception. It consists of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head.

[0124] The backbone network uses alternating and continuous convolutional blocks and cross-stage visual state space blocks (CSVSS) to extract multi-scale features. The CSVSS blocks employ dual branches for representation enhancement, with global branches and convolutional local branches working in parallel and complementing each other during feature extraction.

[0125] The hybrid encoder uses a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence. The cross-scale fusion unit adopts a dual-domain representation complementary module DRCM with spatial and frequency domain branches. The spatial branch uses multi-scale convolution and channel attention mechanism to preserve position and structural information, while the frequency domain branch enhances high-frequency details and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectrum modulation. The fusion of the two features forms a complementary target representation.

[0126] A decoder with an auxiliary prediction head filters feature sequences to obtain an initial target query, and then the decoder performs target classification and localization on the initial target query.

[0127] The training module is used to acquire images in drone application scenarios, build a training set, and train the drone target detection framework.

[0128] The detection module is used to perform target detection using a UAV target detection framework.

[0129] Experiments were conducted based on the UAV target detection method based on the visual state space model and frequency domain awareness provided in this invention to evaluate the UAV target detection framework based on the visual state space model and frequency domain awareness of this invention:

[0130] Experimental setup:

[0131] Datasets: To comprehensively evaluate the performance of the proposed method, this invention was experimentally validated on two UAV aerial target detection datasets, namely VisDrone2019[6] and UAVVaste[7]. The VisDrone dataset contains 288 video clips (261,908 frames in total) and 10,209 still images, collected from urban and suburban scenes in 14 different cities in China. The dataset was captured by various UAV platforms under different scenes, weather and lighting conditions, and contains more than 2.6 million bounding box annotations, covering common targets such as pedestrians, vehicles, bicycles and tricycles. The VisDrone-2019-DET detection task subset contains 6,471 training images, 548 validation images and 3,190 test images, labeled with ten predefined categories. The characteristics of this dataset are small target scale, severe occlusion and dense and complex scenes, which pose a great challenge to the robustness and generalization ability of the detection algorithm. In this invention, we used the VisDrone-2019-DET training and validation sets for model learning and performance evaluation, respectively. Furthermore, to test the generalization ability across datasets, this invention conducted experiments on UAVVaste, a dataset developed by the Institute of Robotics and Machine Intelligence at Poznan University of Technology, specifically designed for litter detection in low-altitude drone images. This dataset currently contains 772 images and 3,718 hand-annotated bounding boxes, covering urban and natural environments such as streets, parks, and lawns. Compared to traditional ground-based litter detection datasets, UAVVaste addresses specific challenges such as smaller target size and complex, variable backgrounds from an aerial perspective. In the experiments, its officially defined training and testing sets were used for training and evaluation, respectively.

[0132] Implementation details: All experiments were conducted on an NVIDIA GeForce RTX 3090 GPU. To ensure fair and reproducible comparative experiments, the method of this invention was implemented based on the RT-DETR[8] framework. The model was trained from the weights with random initialization and no pre-trained model was used. The input resolution was uniformly set to 640×640 pixels during the training and testing phases. The network was trained for a total of 300 epochs with a batch size of 4. The AdamW optimizer was used, with the momentum parameter set to 0.9, the weight decay coefficient set to 0.0001, and the initial learning rate set to 0.0001. To prevent overfitting, an early stopping mechanism was introduced during training, with a patience value set to 50 epochs. The model performance was evaluated using the COCO standard evaluation system. The main indicators included the average precision AP at IoU thresholds of 0.50 to 0.95 (step size 0.05) and AP50 at IoU threshold of 0.50.

[0133] Comparative experiment:

[0134] VisDrone: This invention was compared with a variety of mainstream detectors on the VisDrone-2019-DET dataset. As shown in Table 1, the comparison methods covered YOLO series single-stage detectors, Transformer-based end-to-end detectors (Deformable DETR, Sparse DETR[9], RT-DETR, etc.), and methods designed for small targets or UAV scenarios (DCFL, FBRT-YOLO, etc.).

[0135] As shown in Table 1, VSFPDet achieves excellent performance with AP of 29.8% and AP50 of 49.0%, striking a good balance between accuracy and efficiency. Compared to the YOLO series methods, the method of this invention exhibits significant advantages: for M-series models with similar model complexity, YOLOv8-M, YOLOv9-M, and YOLOv12-M (AP of 25.8%, 25.9%, and 25.4%, respectively), VSFPDet's 29.8% AP represents a significant improvement. Taking YOLOv8-M as an example, the AP of the method of this invention increases from 25.8% to 29.8%, and the AP50 increases from 42.8% to 49.0%, while reducing the number of parameters by 37.6%. Furthermore, for larger L-series models, VSFPDet maintains its lead over YOLOv8-L, YOLOv10-L, YOLOv12-L, and YOLOv13-L. In particular, compared to the best-performing YOLOv8-L (27.3% AP), the method of this invention achieves a 2.5% higher AP, while requiring only about one-third of the parameters and computational cost. This fully demonstrates the unique advantages of the end-to-end detection paradigm combined with Mamba-based global-local complementary modeling in complex aerial photography scenarios.

[0136] Compared to Transformer-type detectors, VSFPDet exhibits more significant advantages. Compared to Deformable DETR and Sparse DETR, the proposed method improves AP by 2.7% and 2.5%, respectively, and AP50 by 6.8% and 6.5%, respectively; while the computational cost is approximately 43% and 62% of theirs. For QueryDet, designed specifically for small targets, although it achieves 28.3% AP at ultra-high resolution input, its high computational cost (212 GFLOPs) limits practical deployment; in contrast, VSFPDet achieves superior performance with 29.8% AP at standard input resolution, improving efficiency by nearly 3 times. Furthermore, compared to the larger-scale RT-DETR-R50 (28.4% AP), the proposed method reduces the number of parameters and computational cost by 61.7% and 45.1%, respectively, while achieving a 1.4% higher AP, and the AP50 increases from 47.1% to 49.0%, demonstrating an excellent accuracy-efficiency tradeoff. Compared to the baseline method RT-DETR-R18 (26.5% AP, 44.2% AP50) of this invention, VSFPDet improves AP to 29.8% and AP50 to 49.0% with a 19.5% reduction in the number of parameters. Although the computational load is slightly increased, the accuracy improvement is more significant, demonstrating the superiority of the method of this invention.

[0137]

[0138] Table 1 Performance comparison on the VisDrone dataset

[0139] Compared to the latest UAV-specific detectors, VSFPDet also demonstrates superior performance. Compared to UAV-DETR-R18 (29.8% AP, 48.8% AP50), the method of this invention achieves an AP50 of 49.0% while maintaining the same AP, and reduces the number of parameters and computational cost by 19.5% and 3.1%, respectively. Compared to FBRT-YOLO-L (29.7% AP, 47.7% AP50), the AP50 is 1.3% higher, and the computational complexity is only 62.6% of its. These results fully demonstrate the superior performance of the method of this invention in UAV target detection tasks.

[0140] To analyze the performance of VSFPDet on different target categories in more detail, Figure 4This paper presents a comparison of the AP50 radar charts of the proposed method with several mainstream detectors on the VisDrone dataset across 10 categories. The charts show that VSFPDet has a significant advantage in small target categories, particularly in motor, pedestrian, and bicycle categories, demonstrating a clear performance improvement over the baseline and other methods, further validating the effectiveness of the proposed method for small target detection. For relatively larger target categories such as van and bus, the proposed method also maintains excellent detection performance, exhibiting stable detection capabilities across multiple target scales.

[0141] UAVVaste: As shown in Table 2, VSFPDet achieved excellent results on the UAVVaste dataset, with an AP of 48.1% and an AP50 of 78.4%, making it the most outstanding among all compared methods. Compared to UAV-DETR-R18 (46.5% AP, 75.1% AP50), which is specifically designed for UAV scenarios, the method of this invention improves AP by 1.6% and AP50 by 3.3%, while reducing the number of parameters by 19.5% and the computational cost by 3.1%. Compared to the baseline method RT-DETR (44.3% AP, 73.1% AP50), VSFPDet improves AP by 3.8% and AP50 by 5.3%, further validating the effectiveness of the method of this invention. Compared to the YOLO series methods, VSFPDet also has significant advantages. For YOLOv8-M (39.9% AP, 72.1% AP50), which has a similar model size, the method of this invention has an AP that is 8.2% higher and an AP50 that is 6.3% higher. Even compared to larger datasets like YOLOv10-L, YOLO11-L, and YOLO12-L (AP range 39.8%-40.2%), VSFPDet maintains a significant lead, with an AP improvement of 7.9%-8.3%, and lower parameter and computational costs. In particular, compared to FBRT-YOLO-L (41.1% AP), the proposed method achieves a 7.0% higher AP with only 62.6% of the computational complexity. In summary, VSFPDet achieves good detection performance on both the VisDrone and UAVVaste datasets, striking a reasonable balance between accuracy and efficiency, validating the effectiveness and generalization ability of the proposed method in UAV target detection tasks.

[0142]

[0143] Table 2 Performance comparison on the UAVVaste dataset

[0144] Ablation experiments were conducted using the framework of this invention:

[0145] To further verify the effectiveness of each component of the proposed method, this section conducted detailed ablation experiments on the VisDrone dataset. The ablation experiment results of the main modules are shown in Table 3.

[0146]

[0147] Table 3 Ablation experiments of the method of the present invention on the VisDrone dataset.

[0148] Independent contributions of CSVSS: Introducing CSVSS alone significantly improved AP from 26.5% to 28.2% (+1.7%), and AP50 to 46.7% (+2.5%). Notably, although CSVSS introduces local convolutional paths and Mamba global modeling paths through its internal HVSS dual-branch design, thanks to CSPNet's segmentation mechanism and Mamba's linear complexity, the number of parameters actually decreased from 20.0M to 15.9M (-20.5%), and GFLOPs decreased from 60.0 to 52.6 (-12.3%). This indicates that CSVSS not only achieves complementary global-local modeling but also improves parameter efficiency through architectural optimization. For targets of different scales, CSVSS is more effective for small targets (AP... S +1.1%), medium target (AP) M +1.9%) and the big target (AP) L Improvements of +3.1%) demonstrate the universal effectiveness of collaborative modeling of global semantics and local details for multi-scale object detection.

[0149] Independent contribution of DRCM: After introducing DRCM alone, AP increased to 27.8% (+1.3%), and AP50 increased to 46.0% (+1.8%). Of particular note is the most significant improvement DRCM had on the smaller objectives (AP). S The accuracy improved from 18.4% to 20.0%, a +1.6% increase, indicating that explicitly enhancing high-frequency detail components in the frequency domain effectively alleviated the problem of weak response of small targets in multi-scale feature fusion. The number of parameters (+0.7M) and computational cost (+7.2G) introduced by DRCM mainly come from frequency domain transformation (FFT / IFFT) and multi-scale convolution, with relatively small increases but significant improvements in accuracy.

[0150] Combining CSVSS with DRCM further improved model performance to 29.8% AP and 49.0% AP50, representing improvements of 3.3% and 4.8% respectively compared to the baseline. More importantly, the full model demonstrated even greater advantages in small object detection (AP). S (Reaching 21.8%, a 3.4% improvement over the baseline). Although the full model's FLOPs increased by 24.3% compared to the baseline, the number of parameters decreased by 19.5%, and the key metric AP in the drone scenario...S Significant improvements have been achieved, fully demonstrating the effectiveness of the present invention.

[0151]

[0152] Table 4 Ablation experiments of HVSS internal components

[0153] To further analyze the design effectiveness of HVSS, Table 4 presents the progressive ablation experiments of its internal components. First, we constructed a simplified version containing only the Mamba global branch as a starting point, achieving 27.6% AP and 45.9% AP50. Building upon this, introducing a local convolutional branch and employing simple additive fusion improved the AP to 27.9% (+0.3%), increasing the number of parameters by 1.8M and FLOPs by 1.2G, validating the effectiveness of global-local complementary modeling. Further employing a learnable fusion mechanism resulted in an AP of 28.2% (+0.6%), with only an increase of 0.6M parameters and 0.8GFLOPs, indicating that learning the optimal fusion weights for global and local features better balances their contributions.

[0154] The core design of DRCM lies in the complementary representation of spatial and frequency domains. Table 5 compares the performance of different branch combinations.

[0155]

[0156] Table 5 Ablation experiments of internal components of DRCM

[0157] The effect of spatial branching: After introducing spatial branching alone, AP increased to 27.1% (+0.6%). S Improved to 19.2% (+0.8%). The spatial branch constructs learnable multi-bandpass operators through parallel 3×3, 3×3 (dilation=2), and 7×7 multi-scale convolutions, hierarchically capturing structural information under different receptive fields. The subsequent SE-style channel recalibration mechanism further adaptively measures the semantic contribution of each channel, highlighting significant responses relevant to the target and suppressing redundant noise. This design provides stable and fine-grained spatial support for the frequency domain branch. The number of parameters increased by 0.5M, and FLOPs increased by 4.3G, mainly from the parallel computation of multi-scale convolutions.

[0158] The effect of frequency domain branching: After introducing frequency domain branching alone, AP increased to 26.9% (+0.4%). S The improvement was increased to 19.6% (+1.2%). It is worth noting that although the overall AP improvement of the frequency domain branch was slightly lower than that of the spatial branch, its gain in small target detection was more significant (AP). S(+1.2% vs +0.8%), which verifies the effectiveness of the frequency domain high-low frequency decoupling and adaptive modulation mechanism in enhancing the spatial response of small targets. The increase in parameter quantity (+0.3M) and FLOPs (+3.6G) mainly comes from frequency domain transformation and learnable filters.

[0159] Combining the two branches resulted in an AP of 27.8% (+1.3%) and an AP50 of 46.0% (+1.8%). S The improvement reached 20.0% (+1.6%). Compared to using either branch alone, the dual-branch synergy brought significant gains, validating the complementarity of the spatial-frequency domain representations. The increase in the number of parameters (+0.7M) and FLOPs (+7.2G) of the complete DRCM was manageable, and significant improvements were achieved in key small target detection metrics.

[0160] The DRCM spatial branch uses multi-scale convolution to construct multi-bandpass operators, and Table 6 verifies the effectiveness of different convolution kernel combinations.

[0161]

[0162] Table 6 Performance Comparison of Different Convolution Combinations

[0163] As shown in Table 6, when using only a single 3×3 convolution, the model achieves 26.6% AP and 44.7% AP50. At this point, the spatial branch can only capture local information within a limited receptive field, lacking the ability to represent structural features of different granularities. Introducing a 3×3 convolution with a dilation rate of 2 improves AP to 26.9% (+0.3%) and AP50 to 45.2% (+0.5%). The parallel design of the two convolutional kernels enables the spatial branch to simultaneously possess both near-range local modeling and mid-range context awareness capabilities, with only a 0.05M increase in parameters. Further introducing a 7×7 large-kernel convolution achieves 27.1% AP (+0.5%) and 45.4% AP50 (+0.7%). Large-kernel convolution can directly capture a wider range of structural information, supplementing low-frequency contours and shape priors, complementing the fine-grained texture features of small-kernel convolution. The number of parameters increased to 20.5M (+0.2M), and the number of FLOPs increased to 64.3G (+0.9G), with the relative cost manageable.

[0164] Visual analysis of the implementation effect of the framework of this invention:

[0165] To more intuitively verify the effectiveness of VSFPDet, this section uses visualization analysis to demonstrate the advantages of the proposed method from multiple dimensions. First, Figure 5This paper compares the effective receptive field (ERF) of the CSVSS-based backbone network constructed in this invention with that of the ResNet-18 backbone network in the baseline. As shown in the figure, the ERF of ResNet-18 exhibits obvious local clustering characteristics, mainly concentrated in a limited region at the center of the feature map. In contrast, the ERF of the CSVSS-based backbone network is expanded and exhibits a more uniform spatial distribution, enabling it to more effectively perceive contextual information from neighboring regions. Although the expansion of the receptive field is limited, its more balanced response pattern allows the network to capture more complete local context when processing densely distributed small targets, which is particularly crucial for distinguishing target categories with similar appearances. This improvement directly benefits from the state-space model introduced in the backbone network. Its linear complexity global modeling capability breaks the locality limitation of traditional convolution, allowing the network to obtain a wider range of context perception without affecting computational efficiency, providing sufficient feature support for small target detection from the perspective of UAVs.

[0166] After verifying the receptive field characteristics of the backbone network, this invention further analyzes the feature representation capability of VSFPDet through feature response heatmaps. Figure 6 The comparison of feature responses between Baseline and VSFPDet in typical drone aerial photography scenarios is shown. The color depth represents the degree of attention the model pays to the corresponding region, with red areas representing high response and blue areas representing low response.

[0167] Depend on Figure 6It can be seen that the high response regions of the baseline model are mainly concentrated near close-range and large-scale targets; for small vehicles in the distance, only scattered light-colored responses appear, and some areas are even completely inactive (dark blue), making it difficult for the network to perceive the existence of these targets. In contrast, the response distribution of VSFPDet is more uniform: small targets in distant and dense areas also form stable and continuous hotspots. This change proves the effectiveness of the design of this invention. First, CSVSS uses a cross-stage splitting mechanism to organize HVSS units. While ensuring global-local complementary modeling, it reduces redundant computation and improves gradient propagation through feature splitting, enabling the network to efficiently extend to deeper layers. Inside HVSS, local convolution and global branches based on state-space models are modeled in parallel. The Mamba branch explicitly establishes long-distance dependencies, allowing distant small targets to fully interact with the global context, thereby effectively alleviating the global modeling bottleneck caused by the limited receptive field of traditional CNNs. Second, DRCM performs constrained enhancement of frequency domain information in the fusion stage to compensate for the edge and detail attenuation caused by multiple downsampling, thereby improving the visibility and distinguishability of small targets. In scenarios with vegetation interference or extreme crowding, VSFPDet still maintains a stable response and can cover partially occluded targets, which is consistent with the quantitative results, indicating that the method proposed in this invention has better robustness and recall capability in UAV scenarios.

[0168] Figure 7 The comparison of detection results between the baseline method and VSFPDet is shown. For easier observation, areas of significant contrast are marked with black dashed boxes, with a magnified view of the area within the box on the left. Green boxes represent correct detections, red boxes represent missed detections, and blue boxes represent false detections. It can be seen that in high-density, highly overlapping scenes, the baseline method often exhibits numerous false detections (blue boxes) and cluttered detection frames, along with some missed detections (red boxes). In contrast, VSFPDet's results are cleaner and more stable: missed and false detections are significantly reduced, correct detections (green boxes) provide more comprehensive coverage, boundaries are closer to the target, and adjacent instances are clearly separated. This improvement further validates the effectiveness of the aforementioned design.

[0169] The proposed VSFPDet detection framework, in this invention, firstly, in the backbone network, proposes a Hybrid Visual State Space Module (HVSS) to achieve a unified global dependency modeling and local detail extraction with linear complexity, and constructs a Cross-Stage Visual State Space Block (CSVSS) to reduce redundant computation and improve gradient propagation. Secondly, in the cross-semantic integration stage, it proposes a Dual-Domain Complementary Representation Module (DRCM) to explicitly enhance the visibility and distinguishability of small targets. Experiments on the VisDrone and UAVVaste datasets demonstrate that VSFPDet, by combining state space models and frequency domain representations, achieves a good balance between detection accuracy and computational efficiency, providing an effective solution for small target detection on UAVs.

[0170] The present invention has the following advantages:

[0171] (1) A hybrid visual state space module HVSS is proposed. Through the collaborative modeling of global and local branches, a unified representation of long-range dependency capture and local spatial structure is achieved with linear complexity. Based on this, a cross-stage visual state space block (CSVSS) is designed to build the backbone network, reduce redundant computation and improve gradient propagation, and provide sufficient global-local semantic support for small target detection.

[0172] (2) Design a dual-domain representation complementary module DRCM, which introduces frequency domain representation as a complement to spatial features in the cross-semantic integration stage: the edge and texture details of small targets are explicitly enhanced by the high frequency component in the frequency domain, while the low frequency component provides a stable global semantic context. The complementary representation of spatial domain and frequency domain is realized in the fusion stage, thereby effectively improving the detection performance of small targets.

[0173] (3) An end-to-end detector VSFPDet is constructed, which combines the global-local collaboration of the backbone with the spatial-frequency complementarity of the cross-semantic integration stage. Significant performance improvement is achieved on the VisDrone and UAVVaste datasets, verifying the effectiveness and generalization ability of the proposed method in complex UAV scenarios.

[0174] The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of the invention. Those skilled in the art will understand that implementing all or part of the above-described embodiments and making equivalent changes in accordance with the claims of the present invention are still within the scope of the invention.

[0175] [1] Liu Y, Tian Y, Zhao Y, et al. Vmamba: Visual state space model[J]. Advances in neural information processing systems, 2024, 37: 103031-103063.

[0176] [2] Gu A, Dao T. Mamba: Linear-time sequence modeling with selectivestate spaces[C] / / First Conference on Language Modeling. 2024.

[0177] [3] Wang C Y, Liao H Y M, Wu Y H, et al. CSPNet: A new backbone thatcan enhance learning capability of CNN[C] / / Proceedings of the IEEE / CVFconference on computer vision and pattern recognition workshops. 2020: 390-391.

[0178] [4] Liu H, Jia C, Shi F, et al. SCSegamba: lightweight structure-aware vision mamba for crack segmentation in structures[C] / / Proceedings ofthe Computer Vision and Pattern Recognition Conference. 2025: 29406-29416.

[0179] [5]Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C] / / Proceedings of the IEEE conference on computer vision and patternrecognition. 2018: 7132-7141.

[0180] [6]Zhu P, Wen L, Du D, et al. Detection and Tracking Meet DronesChallenge[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2021, 44(11): 7380-7399.

[0181] [7] Kraft M, Piechocki M, Ptak B, et al. Autonomous, Onboard Vision-Based Trash and Litter Detection in Low Altitude Aerial Images Collected byan Unmanned Aerial Vehicle[J]. Remote Sensing, 2021, 13(5): 965.

[0182] [8] Zhao Y, Lv W, Xu S, et al. Detrs beat yolos on real-time objectdetection[C] / / Proceedings of the IEEE / CVF conference on computer vision andpattern recognition. 2024: 16965-16974.

[0183] [9] Roh B, Shin J W, Shin W, et al. Sparse DETR: Efficient End-to-EndObject Detection with Learnable Sparsity[C] / / International Conference onLearning Representations.

[0184]

[10] Wang C Y, Yeh I H, Mark Liao H Y. Yolov9: Learning what you wantto learn using programmable gradient information[C] / / European conference oncomputer vision. Cham: Springer Nature Switzerland, 2024: 1-21.

[0185]

[11] Yang C, Huang Z, Wang N. QueryDet: Cascaded sparse query foraccelerating high-resolution small object detection[C] / / Proceedings of theIEEE / CVF Conference on computer vision and pattern recognition. 2022: 13668-13677.

[0186]

[12] Wang A, Chen H, Liu L, et al. Yolov10: Real-time end-to-endobject detection[J]. Advances in Neural Information Processing Systems, 2024,37: 107984-108011.

[0187]

[13] R. Khanam and M. Hussain, “Yolov11: An overview of the keyarchitectural enhancements,”arXiv preprint arXiv:2410.17725, 2024.

[0188]

[14] Y. Tian, Q. Ye, and D. Doermann, “Yolov12: Attention-centricreal-time object detectors,” arXiv preprint arXiv:2502.12524, 2025.

[0189]

[15] Zhu X, Su W, Lu L, et al. Deformable DETR: DeformableTransformers for End-to-End Object Detection[C] / / International Conference onLearning Representations.

[0190]

[16] M. Lei, S. Li, Y. Wu, H. Hu, Y. Zhou, X. Zheng, G. Ding, S. Du,Z. Wu,and Y. Gao, “Yolov13: Real-time object detection withhypergraphenhanced adaptive visual perception,” arXiv preprint arXiv:2506.17733,2025.

[0191]

[17] Xu C, Ding J, Wang J, et al. Dynamic coarse-to-fine learning fororiented tiny object detection[C] / / Proceedings of the IEEE / CVF conference oncomputer vision and pattern recognition. 2023: 7318-7328.

[0192]

[18] Xiao Y, Xu T, Xin Y, et al. FBRT-YOLO: Faster and Better forReal-Time Aerial Image Detection[C] / / Proceedings of the AAAI Conference onArtificial Intelligence. 2025, 39(8): 8673-8681.

[0193]

[19] G. Jocher, A. Chaurasia, and J. Qiu, “Ultralytics yolov8,” 2023.

[0194]

[20] Zhang H, Liu K, Gan Z, et al. UAV-DETR: efficient end-to-endobject detection for unmanned aerial vehicle imagery[J]. arxiv preprintarxiv:2501.01855, 2025.

Claims

1. A method for UAV target detection based on a visual state-space model and frequency domain perception, characterized in that, include: A UAV target detection framework based on a visual state space model and frequency domain perception is established, consisting of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head. The backbone network uses alternating and continuous convolutional blocks and cross-stage visual state space blocks (CSVSS) to extract multi-scale features. The CSVSS blocks employ dual branches for representation enhancement, with global branches and convolutional local branches working in parallel and complementing each other during feature extraction. The hybrid encoder uses a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence. The cross-scale fusion unit adopts a dual-domain representation complementary module DRCM with spatial and frequency domain branches. The spatial branch uses multi-scale convolution and channel attention mechanism to preserve position and structural information, while the frequency domain branch enhances high-frequency details and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectrum modulation. The fusion of the two features forms a complementary target representation. A decoder with an auxiliary prediction head filters feature sequences to obtain an initial target query, and then the decoder performs target classification and localization on the initial target query. Acquire images from drone application scenarios, establish a training set, and train the drone target detection framework. Target detection is performed using the aforementioned UAV target detection framework.

2. The UAV target detection method based on visual state space model and frequency domain perception as described in claim 1, characterized in that, The training of the UAV target detection framework includes: Step S1: The training data is fed into the backbone network and extracted by each CSVSS block to obtain global-local complementary multi-scale features {F2,F3,F4,F5}, which are then fed into the hybrid encoder. In step S2, the hybrid encoder uses a self-attention mechanism to perform intra-scale interaction on the highest-level feature F5 to obtain F5′; F5′ is dynamically fused with {F2,F3,F4} across scales to obtain a feature sequence; wherein, the input of DRCM includes F2, F3 and the upper-level fused feature P1, P1 is the fusion result of F5′ and F4, and the output of DRCM is used as a component of the feature sequence. Step S3: The feature sequence is used as a candidate set for the target query, and a certain number of target queries are selected as the initial target query input decoder. The decoder performs multi-layer iterative decoding processing on the target query. Through multi-layer decoding operations, the target query is updated in terms of features and corrected in terms of position. The semantic information and spatial position information of the target are optimized layer by layer to obtain the target detection box. The detection box information includes target category information and target bounding box position information, which are used to represent the category to which the target belongs and the spatial position of the target in the input image, respectively. Step S4: Construct a loss function based on the difference between the prediction result and the actual annotation, and use the loss function to adjust the parameters of the UAV target detection framework until convergence or a stopping mechanism is triggered.

3. The UAV target detection method based on visual state space model and frequency domain perception as described in claim 2, characterized in that, The backbone network's CSVSS uses a dual-branch approach for representation enhancement. One branch serves as the identity mapping flow Sid, preserving the original information; the other branch serves as the representation enhancement flow Seh, which is processed through consecutively stacked Hybrid Vision State Space (HVSS) blocks for representation enhancement. Use 1x1 convolutions to exchange information between channels and adjust the number of channels as input for the next stage; Four consecutive CSVSS samples were used to extract features at different scales to obtain global-local complementary multi-scale features {F2, F3, F4, F5}. In CSVSS, the HVSS block performs channel expansion and segmentation on the input features to obtain the local branch X. local and global branch X global Two parallel branches model global semantics and local details respectively, and adaptive fusion is performed using learnable factors: Local branches The local branch feature map is obtained by sequentially performing 1x1 convolution for channel reduction, 3x3 convolution for feature extraction, and 1x1 convolution for dimensionality increase followed by residual connection. ; Global branch X global After stabilizing the distribution through layer normalization, the system proceeds in parallel to two branches: the gated branch, after layer normalization, generates a gated signal of the same shape as the main branch through a linear mapping and the SiLu activation function. The main branch sequentially executes linear projection, depthwise separable convolution, and SiLU activation function, then enters the 2D selective scanning SS2D unit to aggregate long-range context in a multi-path structure-aware manner. Finally, layer normalization is performed to obtain a preliminary global representation. The global branch output is obtained by performing a gating operation through element-wise multiplication of the two branches, followed by linear mapping and residual connection. ; Two sets of learnable parameters a, b∈R C Generate channel-wise weights using Softmax normalization , Using normalized weights and Channel-by-channel weighted fusion is performed to obtain adaptively fused features.

4. The UAV target detection method based on visual state space model and frequency domain perception as described in claim 2, characterized in that, Step S2, which involves dynamic cross-scale feature fusion, includes: The upsampled feature F5′′ and feature F4 are used to obtain feature F through the Fusion module. 5'4 Feature F 5'4 The upsampled features P1, F3, and F2 are input into the DRCM to obtain y3; y3 is processed by the downsampled module to obtain y3′; y3′ and P1 are processed by the Fusion module to obtain y2; y2 is processed by the downsampled module to obtain y2′; y2′ and F5′′ are processed by the Fusion module to obtain y1; the features y3, y2, and y1 are fused and concatenated into a feature sequence. In DRCM: Features P1, F3, and F2 are aligned to the same resolution and pre-integrated to obtain a unified representation Z. S Based on the channel, it is divided into spatial branch and frequency domain branch, forming the characteristics of spatial branch. and frequency domain features X f ; Spatial branching employs parallel 3x3, 3x3, and 7x7 convolutional layers to capture structural information under different receptive fields, fusing them to obtain features processed by multi-bandpass operators. ;right Global average pooling and 1x1 convolution are used to generate channel descriptors. The features are then reweighted and fused with the original features to form the output of the spatial branch. ; Frequency domain branching transforms X using a two-dimensional fast Fourier transform. f Mapping to the frequency domain, the low-frequency mask is obtained through a Gaussian function. and high-frequency mask ;use , For frequency domain features X f Element-wise modulation is performed to obtain decoupled high-frequency components. Low-frequency components ,right , Filtering enhancement is performed to obtain , The changes in the filtering results are adaptively weighted to generate enhanced frequency domain features. , Obtained by inverse Fourier transform As the output of the frequency domain branch; splicing in the channel dimension and Cross-domain projection and channel compression are performed through 1x1 convolution, and residual connections are added to obtain y3.

5. The UAV target detection method based on visual state space model and frequency domain perception as described in claim 4, characterized in that, In step S4, the loss function used during training... Loss from target classification Bounding box regression loss Bounding box overlap loss composition: ; ; ; ; ; In the formula, This represents the model's predicted probability of the true class. The class balance factor is represented by γ, and the modulation factor is represented by γ; the predicted bounding box is represented as The true bounding box is represented as D indicates that it contains both and The minimum bounding rectangle; , , These are preset weighting coefficients.

6. The UAV target detection method based on visual state space model and frequency domain perception as described in claim 5, characterized in that, In DRCM, the number of frequency domain branch channels is The number of spatial branch channels is , is the channel scaling factor, and C is the number of feature channels.

7. A UAV target detection system based on a visual state space model and frequency domain perception, characterized in that, The system for implementing the UAV target detection method based on visual state space model and frequency domain perception as described in any one of claims 1-6 includes: The module is used to establish a UAV target detection framework based on a visual state space model and frequency domain perception. It consists of a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head. The backbone network uses alternating and continuous convolutional blocks and cross-stage visual state space blocks (CSVSS) to extract multi-scale features. The CSVSS blocks employ dual branches for representation enhancement, with global branches and convolutional local branches working in parallel and complementing each other during feature extraction. The hybrid encoder uses a cross-scale fusion unit to fuse multi-scale features to obtain a feature sequence. The cross-scale fusion unit adopts a dual-domain representation complementary module DRCM with spatial and frequency domain branches. The spatial branch uses multi-scale convolution and channel attention mechanism to preserve position and structural information, while the frequency domain branch enhances high-frequency details and introduces low-frequency semantic priors through explicit decoupling of high and low frequencies and adaptive spectrum modulation. The fusion of the two features forms a complementary target representation. A decoder with an auxiliary prediction head filters feature sequences to obtain an initial target query, and then the decoder performs target classification and localization on the initial target query. The training module is used to acquire images in UAV application scenarios, establish a training set, and train the UAV target detection framework. The detection module is used to perform target detection using the UAV target detection framework.

Images