A multimodal remote sensing small target detection method across space-frequency domain

Abstract

The present invention relates to the field of remote sensing image detection and recognition, and discloses a multimodal remote sensing small target detection method across space-frequency domains. It designs a dual-modal feature-level fusion target detection architecture, distributes the extraction of visible light and infrared channel features, and proposes a cross-domain gated self-attention fusion module to interact and fuse dual-modal features. Finally, the multi-scale features are fused and input into the detection head to obtain multimodal target detection results. Among them, the cross-domain gated self-attention fusion module explores the global context information of the image from the frequency domain perspective, and introduces a polarization self-attention mechanism to establish long-range dependencies on space-frequency domain differential features, thereby filtering redundant information and enhancing the adaptive fusion of multimodal complementary features. The model of the present invention improves the accuracy and robustness of remote sensing small target detection, while having extremely low computational complexity, is suitable for mobile deployment on satellite and airborne terminals, and improves image processing accuracy and response speed.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image detection and recognition, and in particular to a multi-modal remote sensing small target detection method across space-frequency domains. Background Art

[0002] With the increasing popularity of high-resolution satellites and drones, object detection in remote sensing images has garnered widespread attention across various fields. Compared to natural image detection, remote sensing images are much larger, and the objects of interest occupy only a very small number of pixels. This places high demands on the model's ability to detect small objects and requires the model to be able to rapidly reason about large images. Furthermore, objects come in a variety of categories and sizes, requiring the model to be able to detect objects of varying sizes. Furthermore, remote sensing images often contain complex backgrounds, such as vegetation and buildings, which can interfere with the object detection task and make it difficult for the algorithm to accurately identify and locate the target.

[0003] With the remarkable success of deep learning technology in natural image processing, many studies have successfully applied deep learning algorithms to remote sensing. Multimodal fusion detection has been proven to be an effective means of improving the perception of small targets in remote sensing. However, existing multimodal fusion detection methods still face the following challenges: 1) they fail to fully utilize multimodal complementary information, resulting in poor detection of complex small targets in remote sensing imagery; 2) the high recognition performance of multimodal models often comes at the cost of high computational complexity, making large-scale intelligent models difficult to deploy on small unmanned aerial vehicles (UAVs) and satellite platforms.

[0004] In order to solve the above problems, the present invention proposes an extremely lightweight multimodal remote sensing small target detection method across space-frequency domains. Summary of the Invention

[0005] To address the above issues, this application proposes a multimodal remote sensing small target detection method across the space-frequency domain. First, this application designs a two-stream backbone network based on YOLOv8n as the baseline model of this application. Then, this application designs a cross-domain gated self-attention fusion module (CGSA). Through cross-space-frequency domain differential feature extraction, self-attention feature fusion, and gating units, this application realizes the adaptive fusion of multimodal complementary features across the space-frequency domain. The specific contents are as follows:

[0006] A multimodal remote sensing small target detection method across space-frequency domains includes the following steps:

[0007] S1, acquiring a remote sensing image, and preprocessing the remote sensing image to obtain an initial image;

[0008] S2. Using the improved multimodal remote sensing small target detection model to process the initial image, the target detection result of the remote sensing image is obtained;

[0009] The improved multimodal remote sensing small target detection model includes a YOLOv8n dual-stream backbone network, an improved cross-domain gated self-attention fusion module, a lightweight self-attention mechanism and a YOLOv8n detection head.

[0010] Preferably, the specific content of the target detection result of the remote sensing image obtained by processing the initial image using the improved multimodal remote sensing small target detection model in S2 is:

[0011] S201, dividing the initial image into a visible light image and an infrared image;

[0012] S202, using the YOLOv8n dual-stream backbone network to perform multi-scale feature fusion on the visible light image and the infrared image to obtain image features, the image features include visible light features and infrared characteristics ;

[0013] S203, using an improved cross-domain gated self-attention fusion module to integrate frequency domain information and spatial domain information of the feature image to obtain information to be detected;

[0014] S204: Use the YOLOv8n detection head to detect the information to be detected to obtain a target detection result, where the target detection result includes a target position and a target category.

[0015] Preferably, the specific contents of improving the cross-domain gated self-attention fusion module CGSA in S203 include:

[0016] Cross-domain differential feature extraction (CFE) is used to promote the learning of complementary features in the frequency and spatial domains during the integration of frequency and spatial domain information.

[0017] Self-attention feature fusion (SAFF) is used to guide the fusion of visible light and infrared features in feature images and capture long-range dependencies;

[0018] The adaptive gated AG unit is used to dynamically allocate the fusion weights of visible light and infrared features to achieve more sensitive and adaptive feature cross-fusion.

[0019] Preferably, the operating steps of the cross-domain differential feature extraction CFE are:

[0020] Visible light characteristics and infrared characteristics Visible light frequency characteristic diagram can be obtained through FFT and infrared frequency characteristics , which can be decomposed into the amplitude components of visible light and phase components , infrared amplitude component and phase components , respectively expressed as:

[0021] ;

[0022] ;

[0023] The amplitude and phase components of the visible light mode and the infrared mode are connected in series and subjected to feature enhancement operation to generate enhanced global frequency features. The amplitude component is , the phase component is , and The expressions are:

[0024] ;

[0025] ;

[0026] Among them, Cat[.] represents channel dimension cascade, and The bottleneck layer contains a 1×1 convolution for feature dimensionality reduction, a RELU activation function, and a 1×1 convolution for feature dimension expansion;

[0027] Apply inverse FFT to convert the enhanced global frequency features back to the spatial domain to obtain the reconstructed feature map affected by the frequency domain ;

[0028] ;

[0029] stands for inverse FFT, Represents the reconstruction features;

[0030] Use Reconstruction Features Subtract the original spatial domain features to obtain the visible light features and infrared characteristics , obtain the cross-domain differential features of each modality, the cross-domain differential features of each modality include visible light differential features and infrared differential features , respectively expressed as:

[0031] ;

[0032] ;

[0033] in is the visible light differential feature, is the infrared differential feature, and It is a 1×1 convolution operation.

[0034] Preferably, the operating steps of the self-attention feature fusion SAFF are:

[0035] Cross-domain differential features are generated in the Q and V branches consisting of 1×1 convolution and Reshape operations respectively. and , the features on the Q branch are completely folded, and the features on the V branch maintain high resolution;

[0036] Combine the Q-branch results of the visible light modality and the infrared modality, perform a Softmax operation to obtain the fused weight distribution, and multiply the fused weight distribution with the information-rich V-branch features of the visible light modality and the infrared modality;

[0037] The dual-channel attention weights are obtained by sequentially passing through the 1×1 convolution layer, the Layer Normalization layer and the Sigmoid function and ;

[0038] Note that channel-level multiplication is performed between the weights and the input features to produce feature representations with less noise. The visible light output feature is And the infrared output characteristics are , to suppress redundant non-critical information and enhance the key information that affects detection.

[0039] After filtering the feature representation, the final feature is obtained by cross-adding the two modalities.

[0040] After channel concatenation and 1×1 channel mixing convolution, the output of the CGSA module is obtained.

[0041] Preferably, the process of re-enhancing by cross-adding the two modalities to obtain the final feature is a cross-fusion process, and the expression of the cross-fusion process is:

[0042] ;

[0043] ;

[0044] in, and Represents the cross-fusion result of the two modalities, Represents the adaptive gating unit AG.

[0045] Preferably, the output of the CGSA module is the fusion feature of visible light and infrared modalities, expressed as:

[0046] ;

[0047] in Represents a 1×1 convolution operation.

[0048] Preferably, the AG module introduces a multiplication gating mechanism and nonlinear transformation to adaptively determine the contribution of each modal feature to the final fusion result according to the characteristics of the input data and effectively filter out noise information.

[0049] Preferably, the expression of the AG module is:

[0050] ;

[0051] ;

[0052] in, represents a 1×1 convolution operation, and tanh represents a nonlinear transformation activation function. represents the Sigmoid function, Represents dynamic learnable weights.

[0053] In summary, the present invention provides a multimodal remote sensing small target detection method across space-frequency domains. Compared with traditional technologies, the present invention designs a dual-modal feature-level fusion target detection architecture, distributes the extraction of visible light and infrared channel features, and proposes a cross-domain gated self-attention fusion module for the interaction and fusion of dual-modal features, which improves the accuracy and robustness of remote sensing small target detection. At the same time, it has extremely low computational complexity, is suitable for satellite-borne and airborne mobile deployment, and improves image processing accuracy and response speed.

[0054] The technical method of the present invention is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a step diagram of a multi-modal remote sensing small target detection method across space-frequency domains of the present invention;

[0056] Figure 2 Improved multimodal remote sensing small target detection model diagram for the present invention;

[0057] Figure 3 This is the schematic diagram of the cross-domain gated self-attention fusion module;

[0058] Figure 4 This is a comparison chart of target detection results of different models under the DroneVehicle dataset. Figure 4 (a) is a set of visible light-infrared image pairs, where there are dense targets and targets in the dark. Figure 4 (b) in the figure is a set of visible light-infrared image pairs, where there are dense targets and targets under strong light. DETAILED DESCRIPTION

[0059] The technical method of the present invention is further described below through the accompanying drawings and embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and values described in these embodiments do not limit the scope of this application.

[0060] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0061] Technologies, systems, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0062] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0063] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0064] The present invention provides a multi-modal remote sensing small target detection method across space-frequency domains, such as Figure 1 As shown in Figure 1, this application redesigns the feature extraction network of YOLOv8n to form a two-stream backbone that extracts deep features from both visible light and infrared images. Secondly, this application designs a cross-domain gated self-attention fusion module (CGSA) that integrates frequency and spatial domain information and introduces a lightweight self-attention mechanism to achieve long-range modeling without increasing complexity. Finally, this application retains the detection head structure of the original network.

[0065] The specific steps include:

[0066] S1. Acquire a remote sensing image and preprocess the remote sensing image to obtain an initial image.

[0067] S2. Use the improved multimodal remote sensing small target detection model to process the initial image to obtain the target detection result of the remote sensing image.

[0068] like Figure 2 As shown, the improved multimodal remote sensing small target detection model includes a YOLOv8n dual-stream backbone network, an improved cross-domain gated self-attention fusion module, a lightweight self-attention mechanism and a YOLOv8n detection head.

[0069] Preferably, the specific content of the target detection result of the remote sensing image obtained by processing the initial image using the improved multimodal remote sensing small target detection model in S2 is:

[0070] S201 , dividing the initial image into a visible light image and an infrared image.

[0071] S202, using the YOLOv8n dual-stream backbone network to perform multi-scale feature fusion on the visible light image and the infrared image to obtain image features, the image features include visible light features and infrared characteristics .

[0072] S203: Using an improved cross-domain gated self-attention fusion module to integrate frequency domain information and spatial domain information of the feature image to obtain information to be detected.

[0073] S204: Use the YOLOv8n detection head to detect the information to be detected to obtain a target detection result, where the target detection result includes a target position and a target category.

[0074] Preferably, current multimodal feature fusion methods typically design an encoder module to strengthen the reliance on global information, or combine convolution with the encoder to achieve comprehensive learning of local and global information. Inevitably, these methods come at the cost of computational effort. This application designs a lightweight CGSA cross-domain gated self-attention module for multimodal feature fusion.

[0075] like Figure 3 As shown, the specific contents of the improved cross-domain gated self-attention fusion module CGSA in S203 include:

[0076] Cross-domain differential feature extraction (CFE) is used to promote the learning of complementary features in the frequency domain and spatial domain in the process of integrating frequency domain information and spatial domain information.

[0077] The spectral convolution theorem in Fourier theory shows that processing information in Fourier space can capture the global frequency representation, which greatly reduces the amount of calculation compared to the encoder structure. Assume a visible light feature map , frequency characteristic diagram after fast Fourier transform (FFT) for .

[0078] in yes The position coordinates in . and Respectively represent along and The frequency variables in the direction. In the case of multi-channel images, the FFT transform is applied to each channel separately. For simplicity, this application omits the channel symbols in the formula. The amplitude spectrum component is , the frequency spectrum components are , the formula is:

[0079] .

[0080] .

[0081] and They are The real and imaginary parts of .

[0082] Therefore, CFE affects the spatial domain features by extracting and reconstructing the global frequency information of the two modalities, and performs differential enhancement with the spatial local information, thereby promoting the information flow between the frequency domain and the spatial domain to effectively filter and mix the features of different modalities.

[0083] Preferably, the operating steps of the cross-domain differential feature extraction CFE are:

[0084] Visible light characteristics and infrared characteristics Visible light frequency characteristic diagram can be obtained through FFT and infrared frequency characteristics , which can be decomposed into the amplitude components of visible light and phase components , infrared amplitude component and phase components , respectively expressed as:

[0085] .

[0086] .

[0087] The amplitude and phase components of the visible light mode and the infrared mode are connected in series and subjected to feature enhancement operation to generate enhanced global frequency features. The amplitude component is , the phase component is , and The expressions are:

[0088] .

[0089] .

[0090] Among them, Cat[.] represents channel dimension cascade, and The bottleneck layer contains a 1×1 convolution for feature dimension reduction, a RELU activation function, and a 1×1 convolution for feature dimension expansion.

[0091] Apply inverse FFT to convert the enhanced global frequency features back to the spatial domain to obtain the reconstructed feature map affected by the frequency domain .

[0092] .

[0093] stands for inverse FFT, Represents the reconstruction feature.

[0094] Use Reconstruction Features Subtract the original spatial domain features to obtain the visible light features and infrared characteristics , obtain the cross-domain differential features of each modality, the cross-domain differential features of each modality include visible light differential features and infrared differential features , respectively expressed as:

[0095] .

[0096] .

[0097] in is the visible light differential feature, is the infrared differential feature, and It is a 1×1 convolution operation.

[0098] Given the advantages of the self-attention mechanism in long-term modeling and its ability to capture the internal correlations of features, this application redesigns the polarized self-attention mechanism to guide the fusion of visible light and infrared features. Since visible light features and infrared features are largely spatially aligned, this application focuses on the channel-dimensional features of different modalities for feature fusion.

[0099] Self-attention feature fusion (SAFF) is used to guide the fusion of visible light and infrared features in feature images and capture long-range dependencies.

[0100] Preferably, the operating steps of the self-attention feature fusion SAFF are:

[0101] Cross-domain differential features are generated in the Q and V branches consisting of 1×1 convolution and Reshape operations respectively. and , the features on the Q branch are completely folded, and the features on the V branch maintain high resolution.

[0102] The Q-branch results of the visible light modality and the infrared modality are combined, and a Softmax operation is performed to obtain the fused weight distribution. Since the features of the Q-branch are compressed features, in order to achieve information enhancement, the fused weight distribution is multiplied with the information-rich V-branch features of the visible light modality and the infrared modality.

[0103] The dual-channel attention weights are obtained by sequentially passing through the 1×1 convolution layer, the Layer Normalization layer and the Sigmoid function and .

[0104] Note that channel-level multiplication is performed between the weights and the input features to produce feature representations with less noise. The visible light output feature is And the infrared output characteristics are , to suppress redundant non-critical information and enhance the key information that affects detection.

[0105] After filtering the feature representation, it is enhanced by cross-adding the two modalities to obtain the final feature.

[0106] After channel concatenation and 1×1 channel mixing convolution, the output of the CGSA module is obtained.

[0107] Preferably, the process of re-enhancing by cross-adding the two modalities to obtain the final feature is a cross-fusion process, and the expression of the cross-fusion process is:

[0108] .

[0109] .

[0110] in, and Represents the cross-fusion result of the two modalities, Represents the adaptive gating unit AG.

[0111] The adaptive gated AG unit is used to dynamically allocate the weights of visible light and infrared feature fusion to achieve more sensitive and adaptive feature cross-fusion.

[0112] It is understandable that considering that treating dual-modal features equally for fusion may lead to suboptimal results, this application designs an adaptive and learnable gating unit for cross-fusion of multimodal features. Specifically, the AG module introduces a multiplication gating mechanism and nonlinear transformation to adaptively determine the contribution of each modal feature to the final fusion result based on the characteristics of the input data, and effectively filter out noise information. This can enhance the model's ability to flexibly and dynamically adjust the importance of different modal features, thereby improving the model's fusion performance and robustness in multimodal tasks.

[0113] Preferably, the output of the CGSA module is the fusion feature of visible light and infrared modalities, expressed as:

[0114] .

[0115] in Represents a 1×1 convolution operation.

[0116] Preferably, the AG module introduces a multiplication gating mechanism and nonlinear transformation to adaptively determine the contribution of each modal feature to the final fusion result according to the characteristics of the input data and effectively filter out noise information.

[0117] Preferably, the expression of the AG module is:

[0118] .

[0119] .

[0120] in, represents a 1×1 convolution operation, and tanh represents a nonlinear transformation activation function. represents the Sigmoid function, Represents dynamic learnable weights. 1. Bimodal feature-level fusion object detection architecture.

[0121] To demonstrate the effectiveness and lightweightness of the algorithm of the present application, the present invention conducts target detection tests on the DroneVehicle dataset.

[0122] DroneVehicle is a recently released large-scale RGB-IR remote sensing vehicle detection dataset captured by drones. The dataset contains 28,439 visible-light-infrared image pairs, encompassing urban roads, residential areas, and parking lots, capturing a variety of complex scenes from day to night. It includes five vehicle types: car, truck, bus, van, and freight car.

[0123] In this experiment, 17,990 image pairs were selected as the training set, 1,469 pairs as the validation set, and 8,980 pairs as the test set.

[0124] Experiments were conducted using the PyCharm IDE on a Windows 11 system. This environment includes Python 3.8, PyTorch 1.8, and CUDA 11.1. Computing resources included an Intel Core i7-13700KF CPU and an NVIDIA RTX 4090 GPU. Training was performed for 300 epochs with a batch size of 16. The SGD optimizer used an initial learning rate of 0.01, a momentum of 0.937, and a weight decay of 0.0005.

[0125] This application conducted ablation experiments on the cross-domain gated self-attention fusion module (CGSA) designed in this application to determine its impact on network performance, as shown in Table 1. This application's base model uses a bimodal YOLOv8n intermediate layer concat fusion approach (YOLO8n+MidFusionCAT). Replacing the base model's concat fusion method with the self-attention feature fusion (SAFF) module improved mAP50 by 1.5% with only a slight increase in computational overhead. Furthermore, this application added the cross-domain differential feature extraction (CFE) module and adaptive gating (AG) module designed in this application to SAFF, improving recognition accuracy by 1.7% and 1.8%, respectively. This demonstrates the effectiveness of introducing frequency-domain global feature extraction, cross-domain differential features, and adaptive gating units for adaptively learning feature fusion weights. Finally, the introduction of the full CGSA module improved mAP50 by 2.0% compared to the baseline model, demonstrating its ability to enhance the deep fusion of features from different modalities.

[0126] Table 1 Ablation experiment of cross-domain gated self-attention fusion module on DroneVehicle dataset

[0127] ;

[0128] As shown in Table 2, it can be seen that in the DroneVehicle remote sensing small target dataset, compared with the single-mode or basic multi-mode fusion models of YOLOv8n and YOLOv11n, the present invention fully utilizes cross-modal and cross-space-frequency domain information and demonstrates higher recognition accuracy. The recognition accuracy mAP50 reaches 84.9%, the number of parameters is only 6.31M, and the computational load is 13.9GFLOPs. Compared with the YOLOv8n visible light single-mode model, the recognition accuracy mAP50 of the present invention is improved by 12.7%, and compared with the YOLOv8n infrared single-mode model, the recognition accuracy mAP50 of the present invention is improved by 5.3%.

[0129] Table 2 Comparative experiment results of target detection on the DroneVehicle dataset

[0130] ;

[0131] In order to more intuitively demonstrate the detection performance of this method, this application selects different methods and the model of this application for qualitative comparison. Figure 4 In (a), the freight car target is located in the dark area above the visible light image at night. Only the YOLOv8n infrared single-mode and the model of this application can achieve accurate detection. Both the visible light single-mode and the baseline model have missed detections and misclassifications. Although infrared single-mode detection is not affected by light at night, due to the lack of texture information in infrared images, it is easy to cause confusion for targets with similar outlines. Figure 4 In (b), YOLOv8n has misclassification issues in infrared detection mode. This comparative experiment proves that the model of this application is robust to complex remote sensing small target detection.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical method of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical method to deviate from the spirit and scope of the technical method of the present invention.

Claims

1. A multimodal remote sensing small target detection method across space-frequency domain, characterized in that: The following steps are involved: S1, acquiring a remote sensing image, and preprocessing the remote sensing image to obtain an initial image; S2. Using the improved multimodal remote sensing small target detection model to process the initial image, the target detection result of the remote sensing image is obtained; The improved multimodal remote sensing small target detection model includes a YOLOv8n two-stream backbone network, a cross-domain gated self-attention fusion module and a YOLOv8n detection head; In S2, the target detection results of the remote sensing image obtained by processing the initial image using the improved multimodal remote sensing small target detection model are as follows: S201, dividing the initial image into a visible light image and an infrared image; S202, using the YOLOv8n dual-stream backbone network to perform multi-scale feature fusion on the visible light image and the infrared image to obtain image features, the image features include visible light features and infrared characteristics ; S203. A cross-domain gated self-attention fusion module is designed to integrate frequency domain information and spatial domain information of visible light and infrared feature images to generate multi-scale fused feature maps; S204, using a YOLOv8n detection head to detect the fused feature images of different scales to obtain target detection results, where the target detection results include target position, target category, and prediction confidence; The specific contents of the improved cross-domain gated self-attention fusion module CGSA in S203 include: Cross-domain differential feature extraction (CFE) is used to promote the learning of complementary features in the frequency and spatial domains during the integration of frequency and spatial domain information. Self-attention feature fusion (SAFF) is used to guide the fusion of visible light and infrared features in feature images and capture long-range dependencies; Adaptive gated AG unit, used to dynamically assign weights for visible light and infrared feature fusion to achieve more sensitive and adaptive feature cross-fusion; The operating steps of the cross-domain differential feature extraction CFE are as follows: Visible light characteristics and infrared characteristics Visible light frequency characteristic diagram can be obtained through FFT and infrared frequency characteristics , which can be decomposed into the amplitude components of visible light and phase components , infrared amplitude component and phase components , respectively expressed as: ； ； The amplitude and phase components of the visible light mode and the infrared mode are connected in series and subjected to feature enhancement operation to generate enhanced global frequency features. The amplitude component is , the phase component is , and The expressions are: ； ； Among them, Cat[.] represents channel dimension cascade, and The bottleneck layer contains a 1×1 convolution for feature dimensionality reduction, a RELU activation function, and a 1×1 convolution for feature dimensionality increase; Apply inverse FFT to transform the enhanced global frequency features back to the spatial domain to obtain the reconstructed features affected by the frequency domain: ； stands for inverse FFT, Represents the reconstruction features; Use Reconstruction Features Subtract the original spatial domain features to obtain the visible light features and infrared characteristics , obtain the cross-domain differential features of each modality, the cross-domain differential features of each modality include visible light differential features and infrared differential features , respectively expressed as: ； ； in is the visible light differential feature, is the infrared differential feature, and It is a 1×1 convolution operation.

2. The method for detecting small targets in multimodal remote sensing across space and frequency domains according to claim 1, wherein: The operating steps of the self-attention feature fusion SAFF are: Cross-domain differential features are generated in the Q and V branches consisting of 1×1 convolution and Reshape operations respectively. and , the features on the Q branch are completely folded, and the features on the V branch maintain high resolution; Combine the Q-branch results of the visible light modality and the infrared modality, perform a Softmax operation to obtain the fused weight distribution, and multiply the fused weight distribution with the information-rich V-branch features of the visible light modality and the infrared modality; The dual-channel attention weights are obtained by sequentially passing through the 1×1 convolution layer, the Layer Normalization layer and the Sigmoid function and ; Channel-level multiplication is performed between the attention weight and the input feature to generate feature representation. The visible light output feature is And the infrared output characteristics are , to suppress redundant non-critical information and enhance the key information that affects detection; After filtering the feature representation, the final feature is obtained by recalibrating and enhancing it through cross-addition of the two modalities; After channel concatenation and 1×1 channel mixing convolution, the output of the CGSA module is obtained.

3. The method for detecting small targets in multi-modal remote sensing across space and frequency domains according to claim 2, wherein: The process of recalibrating and enhancing the final features by cross-adding the two modalities is called the cross-fusion process. The expression of the cross-fusion process is: ； ； in, is the visible light output characteristic, is the infrared output characteristic, and Represents the cross-fusion result of the two modalities, Represents the adaptive gating unit AG.

4. The method for detecting small targets in multimodal remote sensing across space and frequency domains according to claim 3, wherein: The output of the CGSA module is the fusion feature of visible light and infrared modalities, expressed as: ； in Represents a 1×1 convolution operation.

5. The method for detecting small targets in multi-modal remote sensing across space and frequency domains according to claim 1, wherein: The AG module introduces a multiplication gating mechanism and nonlinear transformation to adaptively determine the contribution of each modal feature to the final fusion result based on the characteristics of the input data and effectively filter out noise information.

6. The method for detecting small targets in multi-modal remote sensing across space and frequency domains according to claim 5, characterized in that: The expression of the AG module is: ； ； in, and are any two features to be fused, represents a 1×1 convolution operation, and tanh represents a nonlinear transformation activation function. represents the Sigmoid function, Represents dynamic learnable weights.

Images