Remote sensing small target detection network and method based on frequency domain and spatial domain adaptive enhancement

Abstract

The application discloses a kind of remote sensing small target detection network and method based on frequency domain and spatial domain adaptive enhancement, belong to artificial intelligence technical field.The network is based on FCOS light weight detection framework, through the collaborative extraction of spatial domain and frequency domain features, adaptive enhancement and fusion, while significantly reducing parameter quantity and computational complexity, significantly improve small target detection precision and efficiency, finally realize the balance of light weight deployment and high-precision detection.The application can complete spatial context modeling and frequency detail recovery simultaneously in single-stage detection framework by spatial-frequency feature adaptive enhancement network, and introduces adaptive enhancement unit to dynamically adjust enhancement intensity according to local signal-to-noise ratio, to realize the significant improvement of micro target signal and background suppression.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a remote sensing small target detection network and method based on frequency domain and spatial domain adaptive enhancement. Background Technology

[0002] With the rapid development of UAVs and high-altitude remote sensing technology, high-altitude remote sensing images are increasingly widely used in urban security, maritime rescue, traffic management, power line inspection, disaster assessment, and other fields. However, due to the high shooting angle and small target size (usually less than 32×32 pixels), tiny targets often appear in images with blurred edges, lack of texture, and low signal-to-noise ratio, making them easily confused with complex backgrounds and posing a serious challenge to detection. Although general-purpose object detectors based on deep convolutional neural networks (such as Faster R-CNN and YOLO series) have made significant progress in conventional scale tasks in recent years, when directly transferred to scenes with tiny targets, the two major bottlenecks of insufficient spatial feature representation and loss of high-frequency details remain prominent, often leading to a sharp decline in detection performance.

[0003] In recent years, research on small target detection in aerial imagery has attracted widespread attention. Existing studies mainly attempt to alleviate the aforementioned problems through multi-scale feature fusion or super-resolution reconstruction. Feature Pyramid Network (FPN) and its variants combine deep semantics and shallow details through a top-down path, but introduce cross-scale feature obfuscation during the fusion process, often resulting in small targets being overwhelmed by large targets or background noise. While super-resolution-assisted methods (such as SuperYOLO and KDSMALL) can improve image resolution, they are limited by static upsampling kernels and high computational costs, making them difficult to run in real time on airborne devices. In addition, most existing methods only perform feature operations in the spatial domain. From the perspective of the frequency domain, the details and texture information of an image usually correspond to its high-frequency components, and the features of small targets are essentially high-frequency signals. Therefore, how to use frequency domain analysis as a complementary and powerful tool to highlight the features of small targets that are difficult to capture in the spatial domain is also an important research direction in the field of small target detection. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the aforementioned related technologies.

[0005] Therefore, the purpose of this invention is to provide a remote sensing small target detection network and method based on frequency domain and spatial domain adaptive enhancement. Through the spatial-frequency feature adaptive enhancement network, spatial context modeling and frequency detail recovery can be completed simultaneously within a single-stage detection framework. Furthermore, an adaptive enhancement unit is introduced to dynamically adjust the enhancement intensity according to the local signal-to-noise ratio, thereby achieving a significant improvement in the signal strength of small targets and background suppression.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows: This invention provides a remote sensing small target detection network based on adaptive enhancement in the frequency and spatial domains. The network is based on the FCOS lightweight detection framework. Through the collaborative extraction, adaptive enhancement and fusion of spatial and frequency domain features, it significantly improves the accuracy and efficiency of small target detection while greatly reducing the number of parameters and computational complexity, ultimately achieving a balance between lightweight deployment and high-precision detection.

[0007] In addition, the remote sensing small target detection network based on frequency domain and spatial domain adaptive enhancement according to the present invention may also have the following additional technical features: In some implementations, the network includes a backbone network, a feature fusion network, and a detection head; the three modules work together to achieve an end-to-end process of feature extraction, multi-scale fusion, and target detection.

[0008] In some implementations, the backbone network is a core feature extraction module, employing a four-stage pyramid structure, specifically configured as follows: Stage configuration: 4, 4, 2, and 1 FSAE blocks are deployed in 4 stages respectively, corresponding to 64, 128, 256, and 512 output channels, forming a progressive feature representation from high-resolution details to deep semantics; Design aids: Gradient flow is ensured through residual connections, and the GELU activation function is used to enhance nonlinear expressiveness, balancing lightweight design and performance.

[0009] In some of these implementations, each of the FSAE blocks includes: The multi-scale feature extraction module includes four parallel branches and channel concatenation, which concatenates the features from the four branches along the channel dimension to generate a multi-scale feature map F. ms Input the frequency domain path and the spatial domain residual path respectively; The frequency domain decomposition and enhancement module is configured to separate high-frequency components through frequency domain transformation and adaptively enhance them using a learnable filter; and, The feature adaptive fusion module is configured to dynamically balance spatial and frequency domain features through a spatial attention mechanism.

[0010] In some implementations, the four parallel branches are as follows: 1×1 depthwise convolution: small receptive field, used to capture local fine details; 3×3 depthwise convolution: medium receptive field, used to balance details and local context; 5×5 depthwise convolution: large receptive field, used to capture a wide range of background information; Global context modeling: AvgPool, used to obtain global environment information.

[0011] In some embodiments, the frequency domain decomposition and enhancement module includes: Frequency domain conversion unit: used for F ms Perform a 2D-Fast Fourier Transform on a sample-by-sample and channel-by-channel basis to map spatial features into a complex spectrum F. freq ; Spectrum decomposition unit: used to extract from F freq The amplitude spectrum characterizing energy and the phase spectrum characterizing structure are separated in the middle. Learnable enhancement unit: The learnable filter M is initialized based on a Gaussian high-pass filter, and the enhancement factor G is generated by the Softplus function to modulate the amplitude spectrum element by element; Inverse transform reconstruction unit: used to reconstruct the enhanced amplitude spectrum with the original phase spectrum, transform it back to the spatial domain via inverse FFT, and obtain the high-frequency enhancement feature F by taking the real part. enhanced .

[0012] In some of these implementations, the feature fusion network uses a top-down approach to combine multi-scale features output by the backbone network, compensating for cross-scale information differences and enhancing the feature representation of small targets at different levels. The detection head integrates deformable convolution to improve adaptability to small target deformations and outputs target classification results, confidence scores, and detection box coordinate information.

[0013] This invention also provides a method comprising the following steps: S1: Dataset acquisition and processing; S2: Construction of a remote sensing small target detection network based on frequency domain and spatial domain adaptive enhancement: The three core modules are built on the FCOS lightweight detection framework as the basic architecture, with a focus on the construction of the backbone network FSAE-TD; S3: Training parameter settings; S4: Model training and testing.

[0014] In addition, the remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement according to the present invention may also have the following additional technical features: In some of these implementations, step S2 specifically includes: S201: Building a multi-scale feature extraction module: Deploying 4 parallel convolutional branches, completing the feature channel concatenation of the branches, and generating multi-scale feature maps F. ms The four parallel branches are a 1×1 depthwise convolution branch, a 3×3 depthwise convolution branch, a 5×5 depthwise convolution branch, and a global AvgPool branch. S202: Construct a frequency domain decomposition and enhancement module, including: D-FFT transform: for F ms Perform a Fourier transform on a sample-by-sample and channel-by-channel basis to obtain the complex spectrum F. freq ; Spectral decomposition: Calculate F freq The amplitude spectrum A and the phase spectrum P; Learnable enhancement: Initialize filter M, calculate enhancement factor G, match amplitude spectrum dimension through broadcast mechanism, and perform amplitude spectrum modulation based on element-wise multiplication; Inverse transform reconstruction: The enhanced amplitude spectrum and the original phase spectrum are reconstructed, transformed back to the spatial domain by iFFT, and the real part is taken to obtain F. enhanced ; S203: Building a Feature Adaptive Fusion Module: Feature Concatenation F ms With F enhanced Generate spatial attention maps using small convolutional networks. α Adaptive feature fusion is performed in a specific manner, and the number of fused feature channels is adjusted to match the input through convolutional layers; S204: Building a feature fusion network and detection head: Deploying FPN to achieve multi-scale feature fusion, the detection head integrates deformable convolution to complete the design of classification and regression branches.

[0015] In some implementations, the training parameter settings in step S3 include: Hyperparameter configuration: initial learning rate 0.02, minimum learning rate 0.00001, training epochs 300; Evaluation metrics: AP, AP0.5, AP0.75, APvt, APt, APs, and APm were used. Step S4, model training and testing, includes: Training process: The constructed remote sensing small target detection network is iteratively trained using the preprocessed dataset. The parameters are updated using gradient descent. The loss function combines the detection loss and the regularization term. Model selection: Save the parameters of the best-performing model on the test set, with a focus on the small object detection metrics APvt and APt; Performance verification: Comparative tests were conducted on the AI-TODv2, LEVIR-Ship, and Damper_defect datasets to verify the advantages of accuracy and lightweight design.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: In this embodiment of the invention, the remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement is provided. It adopts the end-to-end single-stage small target detection framework FSENet, which significantly improves detection efficiency while maintaining detection performance. Its parameter count is only 37% of that of YOLOv6s, which makes it possible to deploy on resource-constrained platforms. In this embodiment of the invention, the remote sensing small target detection method based on frequency and spatial domain adaptive enhancement provides a spatial-frequency dual-branch feature extraction architecture. In the spatial branch, it uses separable large kernel convolution to capture long-range context, and in the frequency branch, it maps features to the frequency domain through fast Fourier transform (FFT). A frequency domain filtering module is also designed to adaptively enhance the high-frequency components related to small targets. In this embodiment of the invention, the remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement provides a spatial-frequency feature adaptive fusion module designed to effectively integrate information from the two domains. This module is based on an attention mechanism to dynamically calibrate and fuse feature maps in the spatial domain and frequency domain, so that the enhanced high-frequency information can effectively supplement and strengthen the weak target features in the spatial domain. In this embodiment of the invention, the remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement has been fully experimentally validated on multiple public aerial image datasets (including AI-TODv2, LEVIR-Ship and Damper_defect benchmark datasets). Experimental results show that FSENet outperforms or approaches existing mainstream methods in terms of small target detection accuracy, while maintaining the minimum number of parameters and FLOPs.

[0017] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] Figure 1 This is a flowchart of a remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement, as disclosed in one embodiment of the present invention. Figure 2 This is a block diagram of the FSENet network for detecting small targets in remote sensing images, as disclosed in one embodiment of the present invention. Figure 3 This is a structural diagram of a multi-scale spatial feature extraction module disclosed in one embodiment of the present invention; Figure 4 This is a schematic diagram of a frequency domain decomposition and enhancement module disclosed in one embodiment of the present invention; Figure 5 This is a structural diagram of a feature adaptive fusion module disclosed in one embodiment of the present invention; Figure 6This is a test result image on a remote sensing small target dataset disclosed in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific examples and application scenarios.

[0021] In target detection tasks, the loss of high-frequency details and insufficient contextual information often lead to the failure to correctly detect small targets. To address this issue, this invention proposes a lightweight remote sensing image small target detection network, FSENet, based on frequency and spatial domain adaptive enhancement. This network explicitly enhances high-frequency features by introducing a Frequency and Spatial Feature Adaptive Enhancement Module for Tiny Object Detection (FSAE-TD), thereby significantly improving the detection rate of small targets. First, multi-scale convolution is used to extract spatial details. Then, learnable filters are used in the frequency domain to adaptively enhance the edge and texture features of small targets. Finally, a spatial-frequency adaptive fusion mechanism optimizes the feature representation. Combining multi-level feature fusion of a Feature Pyramid Network (FPN) and a lightweight detection head integrating Deformable Convolutional Network (DCN), this architecture significantly improves the feature discrimination capability and localization accuracy of small targets while maintaining the advantages of lightweight parameters and FLOPs. It achieves a balance between accuracy and efficiency on remote sensing datasets such as AI-TODv2 and LEVIR-Ship.

[0022] In some embodiments of this invention, FSENet references the FCOS lightweight detection framework and mainly consists of three parts: a backbone network, a feature fusion network, and a detection head. (See also...) Figure 1 As shown, the backbone network is responsible for extracting features at different scales, then performing multi-scale feature fusion through the FPN network, and finally outputting the target classification and bounding box coordinates by the detection head. In FSENet, the backbone network is FSAE-TD, which is composed of a mixture of multiple FSAE blocks and FSAE downsample blocks.

[0023] In some embodiments of the present invention, the FSAE-TD backbone network is a deep feature extraction architecture specifically designed for small target detection in remote sensing images. The network adopts a four-stage pyramid structure, configuring 4, 4, 2, and 1 FSAE block modules respectively, corresponding to 64, 128, 256, and 512 output channels, forming a progressive feature representation from high-resolution details to deep semantics. Each FSAE block contains three key components: a multi-scale feature extraction module that captures spatial features of different receptive fields through parallel 1×1, 3×3, and 5×5 convolutions and global context branches, providing robust feature representations for subsequent processing; a learnable frequency domain enhancement module that maps features to the frequency domain using fast Fourier transform, adaptively enhancing high-frequency components through a learnable high-pass filter, and then reconstructing spatial features through inverse transform, thereby significantly improving the representation ability of high-frequency details (such as edges and textures); and a spatial-frequency adaptive fusion module that dynamically integrates the original spatial features with the enhanced frequency domain features through a spatial attention mechanism, enhancing semantic information while suppressing irrelevant noise, ultimately outputting more discriminative multi-scale enhanced features.

[0024] Furthermore, since small object detection relies more heavily on fine details in high-resolution feature maps, this invention allocates more computational resources (a total of 8 FSAE blocks) to the first two stages of the FSAE-TD network to ensure that key high-frequency information such as edges and textures is fully preserved in the high-resolution network layers. The entire backbone network maintains gradient flow through residual connections and uses the GELU activation function to enhance nonlinear expressive power. At an input resolution of 800×800, only about 6.8M parameters are required, achieving an organic balance between detection accuracy and computational efficiency. This provides optimized feature representations rich in multi-scale details for the subsequent feature pyramid network and detection head.

[0025] The following is an explanation of multi-scale feature extraction in the spatial domain.

[0026] The effective features of targets of different sizes exist within different scale ranges—small targets require finer, more localized details (captured by small convolutional kernels or dilated convolutions), while blurry or larger targets require broader background information (captured by large convolutional kernels or global pooling). Therefore, the Multi-scale Feature Extraction (MSFE) module utilizes parallel convolutional branches to capture spatial context information at different scales in the input feature map. Through parallel processing and feature fusion, this module generates a feature representation containing multi-scale, multi-level information, ensuring that subsequent high-frequency enhancement operations can simultaneously apply to target features of different sizes, avoiding detail loss or enhancement bias that might be caused by single-scale features.

[0027] First, this invention uses parallel convolutional layers to extract multi-scale spatial features from X to capture target context of different sizes. Specifically, branches 1, 2, and 3 use 1×1, 3×3, and 5×5 depthwise convolutions to extract features at different scales, respectively, while branch 4 models global context information to capture spatial context information of different ranges. Finally, the results from the four branches are concatenated along the channel dimension to obtain the final result. F ms , F ms The inputs will then be sent to the frequency domain path and the spatial domain residual path, respectively.

[0028] The Frequency Domain Decomposition and Enhancement (FDDE) module explicitly transforms feature maps to the frequency domain to separate and adaptively enhance high-frequency components crucial for detail perception. Based on the convolution theorem and spectral interpretability, this module uses Fourier transform to decompose spatial features into an amplitude spectrum representing energy and a structure-preserving phase spectrum. A learnable high-frequency emphasis filter dynamically modulates the amplitude spectrum, thus precisely enhancing high-frequency features. It combines prior knowledge from signal processing (frequency domain operations) with the adaptive capabilities of deep learning, enabling the network to transcend the limitations of fixed filters and autonomously decide "which frequencies to enhance" and "how much to enhance." This significantly improves the representation of details in small and blurred targets while preserving the overall image structure, serving as a crucial bridge connecting spatial features and high-frequency optimization.

[0029] Assume the feature map input to this step is F ms ∈R B×C×H×W The specific calculation steps of the FDDE module are as follows: Step 1: Fast Fourier Transform (FFT) right F ms Each sample and each channel is subjected to a 2D-FFT to transform it from the spatial domain to the frequency domain. The formula for calculating the FFT is: (1) Where x[m, n] is a signal of size m×n in the spatial domain. X [ k , l The symbol represents the complex spectrum value in the frequency domain of the input spatial domain signal x[m,n] after undergoing a two-dimensional discrete Fourier transform. k and l It is a frequency index. In the actual code, this invention uses an efficient algorithm to perform this calculation: (2) F freq The complex frequency spectrum is represented by the Fast Fourier Transform. F 2D The output of () is a complex tensor that stores the representation of the feature map in the frequency domain. Its form is... F freq =Re( F freq )+Im( F freq )* j.

[0030] Step 2: Calculate the amplitude spectrum and phase spectrum From the complex spectrum F freq The amplitude spectrum and phase spectrum are decomposed from the complex number. Its amplitude and phase are calculated as follows: Amplitude spectrum: (3) Re represents the Real Part operator, used to extract the real part of a complex number. For the complex number z = a + bj, Re(z) = a.

[0031] Im represents the Imaginary Part operator, used to extract the imaginary part of a complex number (excluding the imaginary unit j). For the complex number z = a + bj, Im(z) = b.

[0032] ; Phase spectrum: ; (4) Step 3: Apply learnable high-frequency enhancement This is the core of the process; the invention uses a learnable filter M to modulate the amplitude spectrum.

[0033] a) Calculate the enhancement factor: Use the Softplus function to activate filter M, and ensure that the base gain is 1.

[0034] (5) ; (6) G is automatically extended to the same dimension `[B,Cin,H,W]` as the amplitude spectrum A through a broadcasting mechanism.

[0035] b) Enhanced amplitude spectrum: Element-by-element multiplication of the amplitude spectrum amplifies the energy at a specific frequency.

[0036] ; (7) ⊙ represents element-wise multiplication (Hadamard Product).

[0037] Step 4: Reconstructing the complex spectrum and inverse transform The enhanced amplitude spectrum is recombined with the original phase spectrum to form a new complex spectrum, which is then transformed back to the spatial domain by inverse FFT.

[0038] a) Complex spectrum reconstruction: (8) Alternatively, a simpler complex exponent form can be used: ; (9) b) Inverse Fast Fourier Transform (iFFT): The processed frequency domain signal is transformed back to the spatial domain. iFFT is the inverse transform of FFT, and its formula is: (10) The code implementation is as follows: (11) ; This represents the two-dimensional inverse fast Fourier transform operator.

[0039] c) Take the real part: Since the initial input signal is a real signal (image pixel value), the result after transformation and inverse transformation should theoretically also be a real number. However, due to numerical calculation precision issues, the result may contain a very small imaginary part, so the real part is directly taken as the final output.

[0040] ; (12) Ultimately, F_enhanced is the output feature map after high-frequency enhancement in the frequency domain, which will be passed to the subsequent fusion and refinement steps.

[0041] The learnable high-frequency enhancement filter M starts with a priori high-pass filter, but through training, it can learn task-specific frequency enhancement patterns, making it more flexible and effective than a fixed filter. The initialization and update process of M is shown in Algorithm 1: a) Initialization based on a Gaussian high-pass filter. M is initialized as a Gaussian high-pass filter with a small center value and large edge values ​​to emphasize high-frequency components. Normalization is applied to make its mean approximately 1 to ensure that the initial state does not excessively amplify high frequencies.

[0042] Create a centralized coordinate grid and initialize the Gaussian standard deviation: (13) (14) Generate the initial Gaussian low-pass filter: (15) Convert to a high-frequency emphasis filter and normalize: (16) in, It is a normalization factor. It is a small constant to prevent division by zero.

[0043] b) Forward propagation. The feature map is decomposed into amplitude and phase spectra by Fourier transform, and the high-frequency components in the amplitude spectrum are enhanced using a learnable M-enhanced method before being reconstructed back into the spatial domain.

[0044] EnhancementFactor=1+Softplus(M) Amplitudeenhanced=Amplitude⊙EnhancementFactor The Softplus function is defined as follows: Softplus(x) = log(1 + exp(x)) c) Update mechanism. M is updated using gradient descent, with the gradient calculated via automatic differentiation. The loss function typically combines the main task loss (e.g., classification or detection loss) with possible regularization terms.

[0045] c) Update mechanism. M is updated using gradient descent, with the gradient calculated via automatic differentiation. The loss function typically combines the main task loss (e.g., classification or detection loss) with possible regularization terms.

[0046] (17) Gradient calculation is performed using the chain rule: (18) (19) This algorithm can be integrated into any deep learning architecture to enhance high-frequency information of small and ambiguous targets, thereby improving the model's ability to perceive these targets.

[0047] The Feature Adaptive Fusion (FAF) module adaptively fuses frequency-enhanced high-frequency features with the original multi-scale spatial features, further refining a superior feature representation. First, it utilizes a spatial attention map generated by a small convolutional network (…). a The graph (Figure 1) serves as the weight, dynamically balancing the contributions of features from spatial and frequency domain paths on an element-wise basis. Subsequently, a series of convolutional layers (such as 1x1 convolutional dimensionality reduction and 3x3 depthwise separable convolutions) perform nonlinear transformations and refinements on the fused features to smooth noise and integrate information. Therefore, this module does not simply add the enhanced features to the original features, but implements an intelligent, location-adaptive selection and fusion mechanism. This ensures that enhanced high-frequency features are prioritized in detailed regions, while the original features are retained in flat regions to avoid introducing noise. This maximizes the enhancement of detail while ensuring feature robustness, ultimately outputting a high-quality feature map more conducive to small target detection and recognition. Figure 4 As shown.

[0048] The FAF consists of two branches: the spatial domain branch takes the original multi-scale features Fms as input, and the frequency domain branch takes the high-frequency enhanced features Ffreq_enhanced as input. To better preserve effective feature information, an attention-based adaptive selection mechanism is used to fuse them, rather than simply adding or multiplying them.

[0049] (1) Combine the features of the two branches: (20) (2) Spatial attention map generation: A small convolutional network is used to generate a spatial attention map. a This graph is used to weigh the contributions of the original features and the enhanced features at each location.

[0050] ;(twenty one) Where DC() represents DepthwiseConv().

[0051] α ∈ R B×1×H×W Each spatial location has a weight value between [0, 1].

[0052] (3) Adaptive fusion of frequency domain and spatial domain features: ;(twenty two) In regions rich in detail (such as edges), α approaches 0, employing more of the enhanced high-frequency features; in flat regions, α approaches 1, preserving the original features to avoid introducing noise.

[0053] (4) Channel adjustment: Use a convolutional layer to fuse and adjust the channels of Fenhanced so that the number of output channels is the same as the number of input channels X.

[0054] ; (twenty three) This invention addresses the challenge of detecting small targets in aerial images due to their small size, lack of texture, and low signal-to-noise ratio. It proposes a spatial-frequency feature adaptive enhancement network (FSENet) for small target detection. This network overcomes the limitations of traditional methods that only perform feature operations in the spatial domain. By collaboratively utilizing spatial context and frequency domain details, it constructs an efficient and lightweight detection framework. The main contributions of this invention include: (1) Designing and implementing an end-to-end single-stage detection framework, FSENet, which significantly improves the detection performance of small targets while maintaining extremely high efficiency. Its parameter count is only 37% of YOLOv6s, making deployment possible on resource-constrained platforms. (2) Proposing a spatial-frequency dual-branch feature extraction architecture. The spatial branch uses separable large-kernel convolution to capture long-range context to understand the global environment of the target; the frequency branch adaptively enhances key high-frequency components representing edges and textures through fast Fourier transform and learnable frequency domain filtering modules, effectively compensating for easily lost detail information in the spatial domain. (3) A spatial-frequency feature adaptive fusion module was designed. Through an attention mechanism, features from both branches were dynamically calibrated and fused, achieving optimal information complementarity and ensuring that the enhanced features highlighted the target while suppressing background noise. (4) Extensive experimental validation was conducted on several challenging public datasets such as AI-TODv2, LEVIR-Ship, and Damper_defect. The results show that compared with mainstream methods such as Faster R-CNN, FCOS, and YOLO series, FSENet achieves or approaches the optimal level in detection accuracy (AP, especially small target metrics such as APvt, APt, and APs), and achieves a significant reduction in parameters and computational complexity (FLOPs) by orders of magnitude, demonstrating its excellent balance between accuracy and efficiency. Therefore, by using frequency domain analysis as a powerful complementary tool for spatial feature extraction, FSENet successfully alleviates the feature representation bottleneck in small target detection. Specifically, on the AI-TODv2 dataset, compared to FCOS-R50, FSENet improves the detection performance of very small (APvt), small (APt), and small targets (APs) by 6.0%, 10.4%, and 8.8%, respectively. Under the same detection architecture (such as ATSS and RetinaNet), replacing the backbone network with the FSAE-TD proposed in this invention still improves detection performance (e.g., AP50) by more than 2% while significantly reducing the number of parameters and FLOPs. These experiments fully demonstrate the significant advantages of FSENet and its core modules in enhancing the feature representation of small targets and balancing detection accuracy with computational overhead. This invention provides a new approach for remote sensing small target detection that considers both spatial context and frequency details, and is particularly suitable for resource-constrained airborne or edge computing platforms.

[0055] Example 1: like Figure 1 As shown, this embodiment discloses a remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement, including: Step 1: Acquire and process the remote sensing small target dataset, and divide the dataset. This example selects three datasets as experimental data: AI-TODv2, LEVIR-Ship, and PTL-AI_Furnas. Among them, AI-TODv2 and LEVIR-Ship are two small target detection datasets based on remote sensing images, while the PTL-AI_Furnas dataset is a public dataset for power line fault detection based on aerial images, containing small targets and various multi-scale targets.

[0056] Step 2, as follows Figure 2 As shown, a lightweight remote sensing image small target detection network, FSENet, based on frequency-spatial domain adaptive enhancement, is constructed using FCOS as the basic architecture. In FSENet, the backbone network is FSAE-TD, which consists of a mixture of multiple FSAE blocks and FSAE downsample blocks, as shown below. Figure 2 As shown, the FSAE-TD backbone network is a deep feature extraction architecture specifically designed for small target detection in remote sensing images. This network employs a four-stage pyramid structure, configuring 4, 4, 2, and 1 FSAE block modules respectively, corresponding to 64, 128, 256, and 512 output channels, forming a progressive feature representation from high-resolution details to deep semantics. Each FSAE block contains three key components: a multi-scale feature extraction module that captures spatial features of different receptive fields through parallel 1×1, 3×3, and 5×5 convolutions and global context branches, providing robust feature representations for subsequent processing; a learnable frequency domain enhancement module that maps features to the frequency domain using fast Fourier transform, adaptively enhancing high-frequency components through a learnable high-pass filter, and then reconstructing spatial features through inverse transform, thereby significantly improving the representation ability of high-frequency details (such as edges and textures); and a spatial-frequency adaptive fusion module that dynamically integrates the original spatial features with the enhanced frequency domain features through a spatial attention mechanism, enhancing semantic information while suppressing irrelevant noise, ultimately outputting more discriminative multi-scale enhanced features.

[0057] Furthermore, since small object detection relies more heavily on fine details in high-resolution feature maps, more computational resources (a total of 8 FSAE blocks) are allocated to the first two stages of the FSAE-TD network to ensure that key high-frequency information such as edges and textures are fully preserved in the high-resolution network layers. The entire backbone network maintains gradient flow through residual connections and uses the GELU activation function to enhance nonlinear expressive power. At an input resolution of 800×800, only about 6.8M parameters are required, achieving an organic balance between detection accuracy and computational efficiency. This provides optimized feature representations rich in multi-scale details for the subsequent feature pyramid network and detection head.

[0058] The backbone network FSAE-TD specifically includes: Step 201, Multi-scale Spatial Feature Extraction: First, we use parallel convolutional layers to extract multi-scale spatial features from the input image to capture target context of different sizes. Branches 1, 2, and 3 use 1×1, 3×3, and 5×5 depthwise convolutions to extract features at different scales, respectively. Branch 4 models global context information to capture spatial context information of different ranges. Finally, the results from the four branches are concatenated along the channel dimension to obtain the desired result. F ms , F ms Next, the frequency domain path and spatial domain residual path will be input respectively. For example... Figure 3 As shown.

[0059] Step 202, Frequency Domain Decomposition and Enhancement: The spatial features are decomposed into an amplitude spectrum representing energy and a phase spectrum preserving structure using Fourier transform. The amplitude spectrum is then dynamically modulated using a learnable high-frequency emphasis filter, thereby precisely enhancing the high-frequency features. For example... Figure 4 As shown.

[0060] Specifically, it includes: Step 2021, Fast Fourier Transform. For F ms Each sample and each channel is subjected to a 2D-FFT to transform it from the spatial domain to the frequency domain.

[0061] Step 2022: Calculate the amplitude spectrum and phase spectrum. From the complex spectrum... F freq The amplitude spectrum and phase spectrum are decomposed from it.

[0062] The formulas for amplitude spectrum (A) and phase spectrum (P) are given in equations (3) and (4) above.

[0063] Step 2023: Apply learnable high-frequency enhancement. Use a learnable filter M to modulate the amplitude spectrum. The learnable high-frequency enhancement filter M starts from the prior of the high-pass filter, but can learn task-specific frequency enhancement patterns through training, making it more flexible and effective than a fixed filter.

[0064] First, calculate the enhancement factor G, and then calculate the enhancement amplitude spectrum. The specific calculation method is shown in equations (5)-(7).

[0065] The specific calculation steps for the learnable filter M are as follows: Step 20231: Initialization based on a Gaussian high-pass filter. M is initialized as a Gaussian high-pass filter with a small center value and large edge values ​​to emphasize high-frequency components. By normalization, its mean is made approximately 1 to ensure that the initial state does not excessively amplify high frequencies. The calculation method is shown in equations (13)-(16).

[0066] Step 20232: Forward Propagation. The feature map is decomposed into amplitude and phase spectra using Fourier transform. The high-frequency components in the amplitude spectrum are enhanced using a learnable M-vector, and then the spatial domain is reconstructed.

[0067] Step 20233: Update mechanism. The update method is shown in equations (17)-(19).

[0068] Step 2024: Reconstruct the complex spectrum and perform inverse transform. The enhanced amplitude spectrum is recombined with the original phase spectrum to form a new complex spectrum, which is then transformed back to the spatial domain using an inverse FFT. The calculation method is shown in equations (9), (11), and (12).

[0069] Step 203, Feature Adaptive Fusion: Adaptively fuse the frequency-enhanced high-frequency features with the original multi-scale spatial features, and further refine a better feature representation. For example... Figure 5 As shown.

[0070] Specifically, this module includes two branches, with the spatial domain branch receiving the original multi-scale features as input. F ms The input to the frequency domain branch is the high-frequency enhanced feature. F freq_enhanced To better preserve effective feature information, an attention-based adaptive selection mechanism is used to fuse them, rather than simply adding or multiplying them.

[0071] Step 2031: Combine the features of the two branches, and combine them in the following way (20).

[0072] Step 2032: Spatial attention map generation, the specific method is shown in equation (21).

[0073] Step 2033: Adaptive fusion of frequency domain and spatial domain features, the specific method is shown in equation (22).

[0074] Step 2034: Channel adjustment, see equation (23) for details.

[0075] Step 3: Training Parameter Settings. Experiments were conducted within the PyTorch version 2.3.0 deep learning framework. All experiments were performed using a single RTX 4090 graphics card with 24GB of memory. The initial learning rate was 0.02, the minimum learning rate was set to 0.00001, and the number of training epochs was set to 300. Depending on the IoU threshold, AP, AP0.5, AP0.75, APvt, APt, APs, and APm were used as evaluation metrics. AP, AP0.5, and AP0.75 correspond to the average mAP and AP with IoU thresholds of 0.5 and 0.75, respectively, while APvt, APt, APs, and APm represent very small, small, small, and medium-scale evaluation metrics, respectively. For the LEVIR-Ship dataset, only AP0.5 was used to evaluate detection accuracy.

[0076] Step 4: Use the dataset from Step 1 to train and test the model in segments, and save the parameters with the best test results. Test results are shown in Tables 1, 2, and 3. Figure 6 As shown, experimental results demonstrate that, compared to state-of-the-art methods, the proposed method maintains excellent detection performance while significantly reducing the number of parameters and computational cost. In particular, compared to methods such as SSD, ORFENet, Faster R-CNN, YOLOv6 / 8 / 12, and FCOS, the proposed method achieves the lowest parameter count and the best detection performance.

[0077] Table 1 shows the performance of the proposed FSENet and other mainstream methods on the AI-TOD-V2 test set.

[0078] Table 2 shows the performance of the proposed FSENet and other mainstream methods on the LEVIR-SHIP test set.

[0079] Table 3 shows the performance of the proposed FSENet and other mainstream methods on the Damper_defect test set.

[0080] Step 5: Use the trained model to detect small targets in real-world remote sensing images to obtain the category, confidence score, and corresponding bounding box coordinates of each detected target.

[0081] For the parts of this invention not described in detail, please refer to the prior art or the art known to those skilled in the art. This embodiment does not limit these aspects and will not describe them in detail here.

[0082] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement, characterized in that, Includes the following steps: S1. Acquire and process the remote sensing small target dataset, and divide the dataset; S2. Construction of a remote sensing small target detection network based on frequency domain and spatial domain adaptive enhancement: The three core modules are built on the FCOS lightweight detection framework as the basic architecture, with a focus on the construction of the backbone network FSAE-TD. The network includes a backbone network, a feature fusion network, and a detection head; the three modules work together to realize an end-to-end process of feature extraction, multi-scale fusion, and target detection. The backbone network is the core feature extraction module, which adopts a four-stage pyramid structure, and its specific configuration includes: Stage configuration: 4, 4, 2, and 1 FSAE blocks are deployed in 4 stages respectively, corresponding to 64, 128, 256, and 512 output channels, forming a progressive feature representation from high-resolution details to deep semantics; Auxiliary design: Gradient flow is ensured through residual connections, and the GELU activation function is used to enhance nonlinear expressiveness, balancing lightweight design and performance; Each of the FSAE blocks includes: The multi-scale feature extraction module includes four parallel branches and channel concatenation, which concatenates the features from the four branches along the channel dimension to generate a multi-scale feature map F. ms Input the frequency domain path and the spatial domain residual path respectively; The frequency domain decomposition and enhancement module is configured to separate high-frequency components through frequency domain transformation and adaptively enhance them using a learnable filter; and, The feature adaptive fusion module is configured to dynamically balance spatial and frequency domain features through a spatial attention mechanism; S3, Training parameter settings; S4. Model training and testing; S5. The trained model is used for small target detection in real-world remote sensing images to obtain the category, confidence score, and corresponding bounding box coordinates of each detected target.

2. The remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement according to claim 1, characterized in that, The four parallel branches are as follows: 1×1 depthwise convolution: small receptive field, used to capture local fine details; 3×3 depthwise convolution: medium receptive field, used to balance details and local context; 5×5 depthwise convolution: large receptive field, used to capture a wide range of background information; Global context modeling: AvgPool, used to obtain global environment information.

3. The remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement according to claim 1, characterized in that, The frequency domain decomposition and enhancement module includes: Frequency domain conversion unit: used for F ms Perform a 2D-Fast Fourier Transform on a sample-by-sample and channel-by-channel basis to map spatial features into a complex spectrum F. freq ; Spectrum decomposition unit: used to extract from F freq The amplitude spectrum characterizing energy and the phase spectrum characterizing structure are separated in the middle. Learnable enhancement unit: The learnable filter M is initialized based on a Gaussian high-pass filter, and the enhancement factor G is generated by the Softplus function to modulate the amplitude spectrum element by element; Inverse transform reconstruction unit: used to reconstruct the enhanced amplitude spectrum with the original phase spectrum, transform it back to the spatial domain via inverse FFT, and obtain the high-frequency enhancement feature F by taking the real part. enhanced .

4. The remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement according to claim 1, characterized in that, The feature fusion network adopts a top-down path to combine multi-scale features output by the backbone network, which compensates for cross-scale information differences and strengthens the feature representation of small targets at different levels. The detection head integrates deformable convolution to improve adaptability to small target deformations and outputs target classification results, confidence scores, and detection box coordinate information.

5. The remote sensing small target detection method based on frequency domain and spatial domain adaptive enhancement according to claim 1, characterized in that, The training parameter settings in step S3 include: Hyperparameter configuration: initial learning rate 0.02, minimum learning rate 0.00001, training epochs 300; Evaluation metrics: AP, AP0.5, AP0.75, APvt, APt, APs, and APm were used. Step S4, model training and testing, includes: Training process: The constructed remote sensing small target detection network is iteratively trained using the preprocessed dataset. The parameters are updated using gradient descent. The loss function combines the detection loss and the regularization term. Model selection: Save the parameters of the best-performing model on the test set, with a focus on the small object detection metrics APvt and APt; Performance verification: Comparative tests were conducted on the AI-TODv2, LEVIR-Ship, and Damper_defect datasets to verify the advantages of accuracy and lightweight design.

Images