A dual-mode end-to-end target detection method for unmanned aerial vehicles and complex scenes

Abstract

The application provides a dual-mode end-to-end target detection method for unmanned aerial vehicles and complex scenes, and relates to the technical field of unmanned aerial vehicle target detection. In the input and feature extraction stage, a channel splitting operation is used to physically isolate the visible light and infrared data, and the data is transmitted to a parallel dual-flow convolutional network for independent multi-scale feature extraction. In the deep feature fusion stage, a cross-modal difference attention module is added. The spatial cosine similarity is calculated to generate a difference saliency mask, which helps the model to give high weights to the infrared features in areas with inconsistent feature structures. A semantic-guided spatial weighted filtering module is embedded in the neck network. The deep high-level semantic features are used to perform pixel-by-pixel weighted filtering on the shallow features, which can retain the edge details of small targets while greatly suppressing background clutter. At the same time, a scale adaptive weight mechanism is introduced during model optimization to increase the regression penalty for small targets.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target detection technology, and in particular to a dual-modal end-to-end target detection method for UAVs and complex scenarios. Background Technology

[0002] Object detection is a fundamental task in computer vision with wide applications in real life. In the field of drones, such as military surveillance, intelligent security, and disaster assessment, object detection is a crucial prerequisite for various complex applications. Therefore, a reliable object detection algorithm is essential for the drone industry. Current drone detection algorithms are mainly based on visible light, performing well under sufficient lighting conditions. However, drones often face harsh scenarios in actual operations, such as drastic changes in lighting, darkness at night, and complex backgrounds. In these situations, visible light detection algorithms often suffer from severe false negatives and missed detections, making stable operation impossible.

[0003] In recent years, an increasing number of drones have begun carrying infrared payloads. Infrared images are insensitive to changes in lighting and can clearly reflect the thermal radiation contours of objects, thus maintaining excellent target acquisition and anti-interference capabilities even in complex low-light conditions. However, infrared images have relatively low resolution and lack detail and texture information. In contrast, visible light images provide rich detail information. Therefore, introducing infrared modalities to form dual-modal detection (visible and infrared) is a key approach to overcome the shortcomings of single-modal applications. By fusing features from these two modalities, the detection challenges in complex scenes can be effectively addressed.

[0004] With the diversification of UAV missions, practical applications place higher demands on bimodal target detection algorithms. These algorithms not only require strong robustness under extreme lighting conditions such as nighttime, but also need to maintain high-precision and stable detection when faced with complex and variable target scales and background interference. Existing bimodal detection methods still have significant limitations in the feature fusion stage. Most current methods focus on simple feature channel concatenation or element-wise addition. This fusion approach introduces a large amount of irrelevant background redundancy while merging target features, failing to fully utilize the potential of bimodal data and neglecting the differences and complementarities between modal features. Some studies employ traditional global self-attention mechanisms for cross-modal fusion. These methods rely on massive full matrix multiplication operations, with computational complexity exponentially increasing with image resolution, significantly consuming the limited computing power of UAVs. Furthermore, conventional feature concatenation methods fail to convert similarity into a saliency mask, making them highly susceptible to spatial feature inaccuracies and fusion failures during severe UAV vibrations and sudden changes in lighting.

[0005] Meanwhile, as target detection technology is crucial for many downstream tasks of UAVs, the simplicity of the model architecture and its environmental adaptability are equally important. Currently, most mainstream bimodal detection networks use traditional architectures, generally relying on complex anchor box assignment mechanisms and cumbersome post-processing components such as non-maximum suppression. In the complex and ever-changing real-world scenarios faced by UAVs, these manually designed components require frequent hyperparameter adjustments for different environments, which not only increases the difficulty of algorithm deployment and debugging but also easily leads to fluctuations in detection accuracy. Existing bimodal target detection networks, due to structural redundancy and the failure to accurately extract modal differences to remove background interference during feature fusion, still have significant room for improvement in detection accuracy and robustness when facing extreme scenarios such as low-light conditions and complex background interference for UAVs at night. Summary of the Invention

[0006] To address the shortcomings of existing technologies and meet the requirements of high accuracy and robustness for real-world UAVs in complex scenarios, this invention provides a dual-modal end-to-end target detection method for UAVs and complex scenarios. Based on the RT-DETR model architecture, a multimodal extension and reconstruction is performed to propose a novel end-to-end dual-modal target detection framework, CDA-DETR, which can achieve better performance in complex lighting and background interference scenarios. While completing end-to-end inference, it greatly improves the model's adaptability to complex interference environments, and completes the UAV dual-modal target detection task with high accuracy and strong robustness.

[0007] On the one hand, the present invention provides a dual-modal end-to-end target detection method for UAVs and complex scenarios, comprising the following steps:

[0008] Step 1: Acquire and preprocess UAV dual-modal image data;

[0009] Acquire paired visible light and infrared images collected by the UAV, ensuring absolute alignment of the bimodal images in pixel-level space; then, concatenate the paired visible light and infrared images in the channel dimension to construct a bimodal input tensor.

[0010] Step 2: Construct the dual-modal target detection framework CDA-DETR;

[0011] The dual-modal target detection framework CDA-DETR includes a two-stream convolutional network, a feature fusion layer, a multi-scale neck network, and a detection head;

[0012] The dual-stream convolutional network is used to achieve bimodal input separation and parallel bi-branch feature extraction. First, paired visible light and infrared images are acquired, and then concatenated along the channel dimension to construct a bimodal input tensor. Where B represents the batch size, 6 is the total number of channels after splicing, the indices of the 6 channels are defined as 0-5 respectively, and H and W represent the spatial height and width of the bimodal input tensor respectively.

[0013] After the bimodal input tensor X is input into a two-stream convolutional network, it is passed through an explicit channel separation function f split The bimodal input tensor is rigidly cut along the channel dimension, separating it into two independent data streams; the channel separation function f split As shown in formula (1):

[0014] (1);

[0015] Among them, channel indices 0 to 2 are divided into visible light tensors X. vis Channel indices 3 to 5 are divided into infrared tensors X. ir Subsequently, the visible light tensor X vis and infrared tensor X ir The data are fed into parallel two-stream convolutional networks, where independent and parallel layer-by-layer downsampling feature extraction is performed simultaneously. The two-stream convolutional network consists of a series of convolutional layers connected in series with C2f modules.

[0016] Two-stream convolutional networks output feature pyramids at four key scales: visible light features. With infrared features In this context, visible light features and infrared features at the same level together constitute a bimodal feature pair at that level, and the first-level feature pair... Let this be the shallow feature P1, and the third, fourth, and fifth level feature pairs... , , These are denoted as deep features P3, P4, and P5, respectively. Finally, the extracted deep features P3, P4, and P5 will be fed into the cross-modal differential attention module (CDA) as the main features containing high-level semantics. The visible light features in the shallow feature P1 will be... With infrared features Perform element-wise addition to generate a fused shallow feature layer. Then it is fed into a multi-scale neck network for background noise suppression and edge detail integration;

[0017] The feature fusion layer embeds a cross-modal differential attention module (CDA) for deep feature complementation; let the input visible light feature tensor be t1, and the aligned infrared feature tensor be t2. The cross-modal differential attention module (CDA) utilizes independent... The convolutional layer combines batch normalization and ReLU activation function to perform feature space alignment and linear mapping on the input visible light feature tensor t1 and infrared feature tensor t2, generating query feature Q and key feature K respectively, as shown in formula (2) and formula (3):

[0018] (2);

[0019] (3);

[0020] Calculate the element-wise absolute difference matrix of the visible light characteristic tensor t1 and the infrared characteristic tensor t2. ; it is fed into the squeeze-incentive attention module SE for channel dimension weight redistribution, and the difference value feature V of the focused target ontology is constructed, as shown in formula (4):

[0021] (4);

[0022] The cross-modal difference attention module CDA introduces spatial cosine similarity. By introducing the L2 norm, the query feature Q and the key feature K are normalized. The summation operation Sum is performed on the channel dimension to obtain the initial similarity distribution Attn, which measures the consistency of the bimodal spatial structure, as shown in formula (5):

[0023] (5);

[0024] The cross-modal difference attention module CDA uses a constant 1 minus the initial similarity distribution Attn; it decays the weights of high similarity regions to near 0, while simultaneously reversing the activation of the weights of low similarity regions to approach 1, thereby generating a targeted difference significance attention mask matrix M', as shown in formula (6):

[0025] (6);

[0026] The Cross-Modal Differential Attention Module (CDA) introduces a residual connection structure. It performs a pixel-wise multiplication (⊙) of the differential saliency attention mask matrix M' with the differential value feature V, then performs a weighted summation, and finally adds it element-wise to the differential value feature V itself, outputting the cross-modal fusion feature tensor at that scale. As shown in formula (7):

[0027] (7);

[0028] After computation by three parallel cross-modal differential attention modules (CDA), the feature fusion layer finally outputs a deep multi-scale enhanced fusion feature that is deeply complementary and aligned. This deep multi-scale enhanced fusion feature is composed of cross-modal fusion feature tensors at three different depth scales. A set that is formed together;

[0029] The multi-scale neck network introduces a semantically guided spatial weighted filtering (SSWF) mechanism to perform top-down information transfer and multi-scale integration of the feature pyramid. The multi-scale neck network is connected to the output of the feature fusion layer and first receives... and the highest layer features After adjusting the spatial resolution through the upsampling layer, and The features are concatenated along the channel dimension and then fed into the C2f module for processing; the processed features are then upsampled again and compared with... The same concatenation and C2f processing are performed to complete the top-down integration of deep features, outputting an integrated deep feature tensor containing the global receptive field. .

[0030] Then, the integrated deep feature tensor is utilized. shallow features Perform spatial weighted filtering; integrate deep feature tensors The input is fed into a separate 1×1 convolutional layer and processed by the Sigmoid activation function. The processed feature values ​​are then projected onto the (0,1) interval to generate a spatial weight matrix. As shown in formula (8):

[0031] (8);

[0032] Obtain the spatial weight matrix Then, it is compared with shallow features. Perform pixel-by-pixel multiplication along the spatial dimension to generate shallow enhancement features after noise reduction. As shown in formula (9):

[0033] (9);

[0034] Finally, the shallow enhancement features With integration of deep feature tensors The components are concatenated along the channel dimension to output the final integrated noise-free multi-scale enhanced feature tensor.

[0035] The detection head is used to perform end-to-end prediction; the detection head specifically adopts the decoder structure of RT-DETR and is connected to the output of the multi-scale neck network, including the forward inference stage and the backpropagation stage;

[0036] In the forward inference phase, the detection head receives a noiseless multi-scale enhanced feature tensor output from the multi-scale neck network, performs self-attention and cross-attention decoding matrix operations, and then outputs a set of fixed-number prediction vectors containing the target bounding box coordinates and corresponding class confidence through a linear mapping layer, thereby achieving complete end-to-end feature parsing.

[0037] In the backpropagation phase, a loss function is constructed to calculate the error between the detector head's predicted output and the labeled ground truth, thereby guiding the weight update of the CDA-DETR model. Specifically, the loss function is composed of classification loss... Regression loss with bounding box The weighted mixed loss function L;

[0038] The hybrid loss function L is shown in equation (10):

[0039] (10);

[0040] in, and These are the weighting coefficients for the classification loss and the bounding box regression loss, respectively.

[0041] For classification loss The Focal Loss function is used, as shown in Equation (11):

[0042] (11);

[0043] in, This represents the model's predicted probability for the target category. As a class balance factor, As a focusing factor;

[0044] For bounding box regression loss The combination of CIoU loss and smoothing L1 loss is adopted, as shown in formulas (12), (13), and (14):

[0045] (12);

[0046] (13);

[0047] (14);

[0048] Where IoU is the intersection-over-union ratio of the predicted bounding box and the ground truth bounding box, and b is the coordinate of the center point of the predicted bounding box. The coordinates of the center point of the true bounding box. ρ is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, c is the diagonal length of the minimum bounding box, and v is the aspect ratio consistency factor. To balance the aspect ratio penalty term, x is the difference between the predicted coordinates and the true coordinates; The loss function is responsible for constraining the overlap between the predicted bounding box and the ground truth bounding box, the Euclidean distance between the center points, and the aspect ratio at the global feature scale; the smoothing L1 loss function is responsible for accurately narrowing down the absolute distance of the coordinates at the micro-pixel scale.

[0049] Calculate the bounding box regression loss At that time, a scale-adaptive weighting mechanism based on the area of ​​the ground truth bounding box was introduced; the size of the area of ​​the ground truth bounding box was used as a dynamic evaluation benchmark for the inverse penalty coefficient: the smaller the target area, the larger the weight multiplier assigned to the target in the global regression loss;

[0050] The scale-adaptive hybrid loss value is obtained by using a hybrid loss function, and the backpropagation of the entire CDA-DETR detection model and the update and iteration of the network weight parameters are completed.

[0051] Step 3: Perform forward inference and target detection based on the dual-modal target detection framework CDA-DETR;

[0052] The bimodal input tensor to be detected is fed into the trained bimodal object detection framework CDA-DETR for forward inference. The bimodal tensor sequentially passes through a dual-stream convolutional network for feature extraction, through the cross-modal differential attention module (CDA) in the feature fusion layer for deep complementary fusion, and through the semantically guided spatial weighted filtering (SSWF) mechanism in the multi-scale neck network for background noise suppression and multi-scale integration. Finally, the detection head receives a noise-free multi-scale enhanced feature pyramid, performs self-attention and cross-attention decoding matrix operations, and directly outputs a fixed number of prediction vectors containing the target bounding box coordinates and corresponding class confidence scores. This model achieves end-to-end feature parsing, using only a confidence threshold to filter and parse the prediction vectors, ultimately outputting accurate object detection results.

[0053] On the other hand, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the described dual-modal end-to-end target detection method for UAVs and complex scenarios.

[0054] Thirdly, this application proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned dual-modal end-to-end target detection method for UAVs and complex scenarios.

[0055] The beneficial effects of adopting the above technical solution are as follows:

[0056] This invention provides a dual-modal end-to-end target detection method for UAVs and complex scenarios, with the following main contributions:

[0057] We considered the characteristics of UAVs, such as the variable target scale under complex perspectives and the susceptibility of single-modal failure in extreme lighting scenarios. We proposed a new dual-modal end-to-end target detection framework, CDA-DETR. In the initial stage of model feature extraction, the visible light and infrared dual-modal input data are physically isolated by channel splitting. The data is then fed into a parallel two-stream convolutional network to extract multi-scale features independently. During feature fusion, a cross-scale hierarchical fusion strategy is executed to avoid the mixing of low-level physical information and feature contamination in the initial stage.

[0058] In the deep feature fusion stage, we embedded multiple cross-modal difference attention modules (CDA) based on spatial cosine similarity to interact with and refine deep features of different modalities. By using difference saliency masks, we reverse-weight infrared features in regions with inconsistent structures, accurately focus on and extract complementary information between the two modalities, and improve the overall anti-interference performance of the model under the failure mode.

[0059] We propose incorporating a semantically guided spatially weighted filtering (SSWF) mechanism into the neck network (feature pyramid). This mechanism utilizes deep high-level semantic features to generate a spatial weight matrix, which performs pixel-by-pixel weighted filtering on shallow features. This effectively preserves the edge contours of small targets and suppresses background clutter in complex scenes. Furthermore, we introduce a scale-adaptive weighting mechanism based on the ground truth bounding box area into the hybrid loss function optimized in the model. This dynamically increases the regression penalty for small targets, further improving the localization accuracy for difficult-to-identify small targets.

[0060] Compared with existing technologies, this invention achieves end-to-end feature parsing, using only confidence thresholds to complete result filtering and parsing, significantly improving the model's adaptability to complex interference environments and detection performance. Extensive experimental results on the DroneVehicle bimodal dataset demonstrate that our proposed CDA-DETR method achieves significant advantages over classical target detection methods, improving mAP50-95 to 59.4% and mAP50 to 79.4%. Furthermore, compared to the baseline model RT-DETR-Add, mAP50-95 is improved by 6.2% and mAP50 by 5.9%, indicating that the CDA-DETR method exhibits excellent performance and environmental adaptability in the variable environment and complex target detection scenarios of UAVs. Attached Figure Description

[0061] Figure 1 Flowchart of the overall architecture of the dual-modal target detection framework CDA-DETR according to an embodiment of the present invention;

[0062] Figure 2 Comparison of detection results for complex main road scenarios in embodiments of the present invention;

[0063] Among them, (a) - YOLOv10-Add model detection results, (b) - RT-DETR-Add model detection results, and (c) - CDA-DETR model detection results of this scheme;

[0064] Figure 3 Comparison of local artifacts and small target detection results in an embodiment of the present invention;

[0065] Among them, (a) - YOLOv10-Add model detection results, (b) - RT-DETR-Add model detection results, and (c) - CDA-DETR model detection results of this scheme. Detailed Implementation

[0066] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0067] Example 1:

[0068] On the one hand, the present invention provides a dual-modal end-to-end target detection method for UAVs and complex scenarios, comprising the following steps:

[0069] Step 1: Acquire and preprocess UAV dual-modal image data;

[0070] Acquire paired visible light and infrared images collected by the UAV in complex scenes, and ensure absolute alignment of the bimodal images in pixel-level space; then, concatenate the paired visible light and infrared images in the channel dimension to construct a bimodal input tensor, which serves as the unified input for the subsequent detection framework.

[0071] Step 2: Construct the dual-modal target detection framework CDA-DETR; in this embodiment, the overall algorithm data flow and network architecture of the dual-modal target detection framework CDA-DETR are as follows: Figure 1 As shown.

[0072] The dual-modal target detection framework CDA-DETR includes a two-stream convolutional network, a feature fusion layer, a multi-scale neck network, and a detection head;

[0073] The dual-stream convolutional network is used to achieve bimodal input separation and parallel bi-branch feature extraction. First, paired visible light and infrared images are acquired. Considering that absolute alignment of the bimodal pixel-level space must be guaranteed when performing geometric data augmentation operators such as random cropping and scaling during the training phase, the system concatenates the two images along the channel dimension to construct a bimodal input tensor. Where B represents the batch size, 6 is the total number of channels after splicing, the indices of the 6 channels are defined as 0-5 respectively, and H and W represent the spatial height and width of the bimodal input tensor respectively; this operation ensures spatial pixel geometric constraints while reusing and unifying the data loading interface of the infrastructure.

[0074] If a dual-modal input tensor X is fed into a two-stream convolutional network and then directly used for blending and extraction using conventional single-branch convolution, severe semantic aliasing and feature contamination will occur in the initial stage of the low-level features. Therefore, the model uses an explicit channel separation function f split The bimodal input tensor is rigidly cut along the channel dimension, separating it into two independent data streams; the channel separation function f split As shown in formula (1):

[0075] (1);

[0076] Among them, channel indices 0 to 2 are divided into visible light tensors X. vis Channel indices 3 to 5 are divided into infrared tensors X. ir Subsequently, the visible light tensor X vis and infrared tensor X ir The data are fed into parallel two-stream convolutional networks, where independent and parallel layer-by-layer downsampling feature extraction is performed simultaneously. The two-stream convolutional network consists of a series of convolutional layers connected in series with C2f modules.

[0077] To balance the capture of low-level edges of small targets on drones with the overall lightweight inference speed of the model, the two-stream convolutional network outputs feature pyramids at four key scales: visible light features. With infrared features In this context, visible light features and infrared features at the same level together constitute a bimodal feature pair at that level, and the first-level feature pair... Let this be the shallow feature P1, and the third, fourth, and fifth level feature pairs... , , These are denoted as deep features P3, P4, and P5, respectively. In this architecture, the second-level feature P2 serves only as a downsampling space transition within the backbone network and does not participate in subsequent cross-modal fusion and detection prediction stages. Finally, the extracted deep features P3, P4, and P5 will be fed into the cross-modal differential attention module (CDA) as the main features containing high-level semantics. The visible light features in the shallow feature P1... With infrared features Perform element-wise addition to generate a fused shallow feature layer. Then it is fed into a multi-scale neck network for background noise suppression and edge detail integration;

[0078] The feature fusion layer embeds a cross-modal differential attention module (CDA) for deep feature complementation. After acquiring the aforementioned multi-scale independent modal features, the feature fusion layer adopts differentiated cross-modal fusion strategies for different semantic abstraction levels at different network depths. For shallow feature P1 containing rich low-level texture and high-resolution edge information, the visible light features in shallow feature P1 are directly processed. With infrared features Perform element-wise addition and output the first fused feature tensor. However, for the deep features P3, P4, and P5 responsible for high-level semantic representation, traditional detection frameworks typically employ simple channel concatenation or addition operations. When drones face extreme lighting or complex background interference, especially the visible light modality, which is prone to failure, the amplitude of its deep features will drastically decrease. If the aforementioned homogeneous fusion method is directly applied at this time, the noise floor of the failed modality will severely contaminate the robust features of the infrared modality, leading to distortion of the multi-scale semantics after fusion. To address this issue, independent cross-modal differential attention modules (CDA) are configured at the deep feature P3, P4, and P5 levels respectively. This module abandons the conventional approach of relying on absolute feature values ​​and instead explicitly purifies complementary information by measuring the correlation between the two modal features in spatial structure. Taking any deep-scale CDA module as an example, let the input visible light feature tensor be t1 and the aligned infrared feature tensor be t2. Considering the huge differences in the numerical distribution of the two modalities under extreme scenarios, the cross-modal differential attention module CDA utilizes independent The convolutional layer combines batch normalization and ReLU activation function to perform feature space alignment and linear mapping on the input visible light feature tensor t1 and infrared feature tensor t2, generating query feature Q and key feature K respectively, as shown in formula (2) and formula (3):

[0079] (2);

[0080] (3);

[0081] Simultaneously, to explicitly capture the difference information between modes, the element-wise absolute difference matrix of the visible light feature tensor t1 and the infrared feature tensor t2 is calculated. Considering that direct subtraction would introduce significant background noise, the signal is fed into the existing squeeze-excited attention module SE for channel dimension weight redistribution, constructing the difference feature V that focuses on the target ontology, as shown in formula (4):

[0082] (4);

[0083] In the feature correlation calculation stage, if the traditional scaled dot product is used directly, the dot product result will directly fail when the visible light mode failure causes a sudden change in feature amplitude, and it cannot accurately represent the consistency of the spatial structure. Therefore, the cross-modal difference attention module CDA introduces spatial cosine similarity, and normalizes the query feature Q and key feature K by introducing the L2 norm, which strictly eliminates the interference of feature absolute brightness and amplitude changes. The summation operation Sum is performed on the channel dimension to obtain the initial similarity distribution Attn that measures the consistency of the bimodal spatial structure, as shown in formula (5):

[0084] (5);

[0085] In order to dynamically assign higher response weights to effective modalities with more significant feature representations in any region where a single modality structure fails or information is missing, the cross-modal differential attention module CDA uses a constant 1 minus the initial similarity distribution Attn; this inversion operation constitutes the core of the CDA module: the weights of high similarity regions are decayed to close to 0, while the weights of low similarity regions are reverse-activated to approach 1, thereby generating a targeted differential significance attention mask matrix M', as shown in formula (6):

[0086] (6);

[0087] Finally, to prevent gradient decay of small targets during backpropagation in deep networks, the Cross-Modal Differential Attention (CDA) module introduces a residual connection structure. It performs a pixel-wise multiplication operation (⊙) between the differential saliency attention mask matrix M' and the differential value feature V, then performs a weighted summation, and finally adds it element-wise to the differential value feature V itself, outputting the cross-modal fusion feature tensor at that scale. As shown in formula (7):

[0088] (7);

[0089] After computation by three parallel cross-modal differential attention modules (CDA), the feature fusion layer finally outputs a deep multi-scale enhanced fusion feature that is deeply complementary and aligned. This deep multi-scale enhanced fusion feature is composed of cross-modal fusion feature tensors at three different depth scales. A set that is formed together;

[0090] The multi-scale neck network introduces a semantically guided spatial weighted filtering (SSWF) mechanism for top-down information transfer and multi-scale integration of the feature pyramid. In traditional feature pyramid networks, shallow and deep features are typically directly upsampled and then concatenated. However, from the UAV's top-down perspective, the first fused feature extracted by the shallow network... While preserving high-resolution edge details of tiny targets, it also contains extremely severe background clutter and noise. If it is directly and forcibly merged with deep features, the shallow noise will inevitably contaminate the deep semantics, causing the weak response of the drone's tiny targets to be completely drowned out by the noise.

[0091] To block the upward propagation path of shallow noise, this model innovatively introduces a semantically guided spatial weighted filtering mechanism (SSWF) into the multi-scale neck network. It utilizes the high-level semantic localization information of deep features to guide the spatial noise reduction filtering of shallow features.

[0092] The multi-scale neck network is connected at the output of the feature fusion layer and first receives... and the highest layer features After adjusting the spatial resolution through the upsampling layer, and The features are concatenated along the channel dimension and then fed into the C2f module for processing; the processed features are then upsampled again and compared with... The same concatenation and C2f processing are performed to complete the top-down integration of deep features, outputting an integrated deep feature tensor containing the global receptive field. .

[0093] Then, the integrated deep feature tensor is utilized. shallow features Perform spatial weighted filtering; integrate deep feature tensors The input is fed into a separate 1×1 convolutional layer and processed by the Sigmoid activation function. The processed feature values ​​are then projected onto the (0,1) interval to generate a spatial weight matrix. As shown in formula (8):

[0094] (8);

[0095] Obtain the spatial weight matrix Then, it is compared with shallow features. Perform pixel-by-pixel multiplication along the spatial dimension to generate shallow enhancement features after noise reduction. As shown in formula (9):

[0096] (9);

[0097] The physical significance of this mathematical operation lies in: leveraging the advanced semantic target localization capabilities of deep features, forcibly assigning a high weight close to 1 to pixels belonging to the microscopic target body in shallow features, while suppressing the response of background clutter pixels to near 0. Finally, the shallow enhancement features are then... With integration of deep feature tensors The components are concatenated along the channel dimension to output the final integrated noise-free multi-scale enhanced feature tensor.

[0098] The detection head is used to perform end-to-end prediction; the detection head specifically adopts the decoder structure of RT-DETR and is connected to the output of the multi-scale neck network, including the forward inference stage and the backpropagation stage;

[0099] In the forward inference phase, the detection head receives a noiseless multi-scale enhanced feature tensor output from the multi-scale neck network, performs self-attention and cross-attention decoding matrix operations, and then outputs a set of fixed-number prediction vectors containing the target bounding box coordinates and corresponding class confidence through a linear mapping layer, thereby achieving complete end-to-end feature parsing.

[0100] In the backpropagation phase, a loss function is constructed to calculate the error between the detector head's predicted output and the labeled ground truth, thereby guiding the weight update of the CDA-DETR model. Specifically, the loss function is composed of classification loss... Regression loss with bounding box The weighted mixed loss function L;

[0101] The hybrid loss function L is shown in equation (10):

[0102] (10);

[0103] in, and These are the weighting coefficients for the classification loss and the bounding box regression loss, respectively.

[0104] For classification loss The Focal Loss function is used, as shown in Equation (11):

[0105] (11);

[0106] in, This represents the model's predicted probability for the target category. As a class balance factor, This is the focus factor; the function introduces a dynamic scaling factor. Actively reduce the gradient contribution weight of a large number of easily classified background samples in the total loss, forcing the model to focus its optimization on small target categories that are difficult to identify during backpropagation;

[0107] For bounding box regression loss The combination of CIoU loss and smoothing L1 loss is adopted, as shown in formulas (12), (13), and (14):

[0108] (12);

[0109] (13);

[0110] (14);

[0111] Where IoU is the intersection-over-union ratio of the predicted bounding box and the ground truth bounding box, and b is the coordinate of the center point of the predicted bounding box. The coordinates of the center point of the true bounding box. ρ is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, c is the diagonal length of the minimum bounding box, and v is the aspect ratio consistency factor. To balance the aspect ratio penalty term, x is the difference between the predicted coordinates and the true coordinates; The loss function is responsible for constraining the overlap between the predicted bounding box and the ground truth bounding box, the Euclidean distance between the center points, and the aspect ratio at the global feature scale; the smoothing L1 loss function is responsible for accurately narrowing down the absolute distance of the coordinates at the micro-pixel scale.

[0112] For classification loss The model employs the Focal Loss function. This function, by introducing a dynamic scaling factor, actively reduces the gradient contribution weight of easily categorized background samples in the total loss, forcing the model to focus its optimization efforts on difficult-to-identify small target categories during backpropagation. For bounding box regression loss... Because tiny targets in drone scenarios are extremely sensitive to pixel-level positional shifts, conventional single IoU loss cannot provide precise gradient guidance. Therefore, the model employs a combination of CIoU loss and smoothed L1 loss. CIoU loss constrains the overlap, Euclidean distance, and aspect ratio between the predicted and ground truth bounding boxes at the global feature scale; while smoothed L1 loss precisely narrows down the absolute distance of the coordinates at the micro-pixel scale.

[0113] More importantly, to further overcome the bottleneck of regression accuracy for small targets, the calculation of bounding box regression loss needs to be improved. At that time, a scale-adaptive weighting mechanism based on the area of ​​the ground truth box was introduced. This mechanism breaks through the traditional network's penalty strategy of treating all targets at the same scale. It uses the size of the area of ​​the ground truth box as a dynamic evaluation benchmark for the inverse penalty coefficient: the smaller the target area, the larger the weight multiplier given to the target in the global regression loss.

[0114] The scale-adaptive hybrid loss value is obtained by using a hybrid loss function, and the backpropagation of the entire CDA-DETR detection model and the update and iteration of the network weight parameters are completed.

[0115] Step 3: Perform forward inference and target detection based on the dual-modal target detection framework CDA-DETR;

[0116] The bimodal input tensor to be detected is fed into the trained bimodal object detection framework CDA-DETR for forward inference. The bimodal tensor sequentially passes through a dual-stream convolutional network for feature extraction, through the cross-modal differential attention module (CDA) in the feature fusion layer for deep complementary fusion, and through the semantically guided spatial weighted filtering (SSWF) mechanism in the multi-scale neck network for background noise suppression and multi-scale integration. Finally, the detection head receives a noise-free multi-scale enhanced feature pyramid, performs self-attention and cross-attention decoding matrix operations, and directly outputs a fixed number of prediction vectors containing the target bounding box coordinates and corresponding class confidence scores. This model achieves end-to-end feature parsing, using only a confidence threshold to filter and parse the prediction vectors, ultimately outputting accurate object detection results.

[0117] Example 2:

[0118] To verify the effectiveness of the proposed bimodal target detection model under complex top-down perspectives, this study uses the authoritative bimodal dataset DroneVehicle, which focuses on UAV vision, for training and evaluation.

[0119] The DroneVehicle dataset, released by a research team from Tianjin University, is a large-scale benchmark library specifically built for dual-modal target detection on unmanned aerial vehicles (UAVs). The dataset contains 56,878 high-resolution aerial images, precisely constructing 28,439 spatially aligned visible light (RGB) and infrared (IR) image pairs. In terms of annotation, the dataset provides up to 953,087 oriented bounding box instances, finely divided into five core vehicle categories: cars, trucks, buses, vans, and freight cars.

[0120] The unique advantage of this dataset lies in its extremely high scene complexity and stringent physical challenges. Data acquisition involved flight altitudes ranging from 80 to 120 meters, encompassing both vertical overhead views and various tilted perspectives, resulting in extreme miniaturization and scale variability of the targets to be detected. More importantly, the data samples comprehensively cover all-weather lighting conditions, including daytime, dusk, late night, and extreme darkness. In nighttime scenes, the visible light mode suffers severe information attenuation or even complete failure; while in daytime, intense sunlight scenarios, the infrared mode is prone to thermal cross-effects caused by the mixing of target and background thermal radiation. This bidirectional extreme modal asymmetry, coupled with the complex terrain clutter of urban roads and densely populated residential areas, perfectly addresses the core pain points addressed in this research, making it an ideal testing ground for verifying the dynamic complementary capabilities of cross-modal differential attention modules and neck noise reduction filtering mechanisms.

[0121] Regarding the specific details of network training and implementation, all experiments were conducted in a unified environment based on the PyTorch deep learning framework, with the hardware platform equipped with two NVIDIA RTX 4090 GPUs for distributed data parallel acceleration. To ensure absolute fairness in the comparative experiments and avoid evaluation bias caused by underlying pre-trained features, all models were trained without any pre-trained weights, following a complete training strategy from scratch. The spatial resolution of the network input bimodal images was uniformly adjusted to 640×640. The models underwent 200 epochs of iterative training, with a global batch size of 8. The AdamW optimizer was used for optimization, with an initial learning rate of 1×10⁻⁴, momentum of 0.9, and weight decay coefficient of 1×10⁻⁴. To reduce memory usage and improve training efficiency, automatic mixed precision technology was enabled during training. In addition, to prevent overfitting, an early stopping mechanism with a tolerance parameter of 20 was introduced, while other training parameters remained at the framework's default configuration.

[0122] To objectively evaluate the performance advantages of CDA-DETR, this embodiment conducts a comprehensive quantitative comparison with the most representative classic single-modal object detection models on the DroneVehicle dataset. Considering that the original YOLOv8, YOLOv10, DETR, and RT-DETR are all single-modal networks, to adapt to the dual-modal input attributes of this study and ensure the fairness of the comparative experiments, this paper uniformly modifies the above baseline models: a simple fusion strategy of direct addition is used to merge the visible light and infrared modes. The modified baseline models are denoted as YOLOv8-Add, YOLOv10-Add, DETR-Add, and RT-DETR-Add, respectively. The evaluation metrics are precision, recall, and mean precision at different IoU thresholds. The quantitative comparison results are shown in Table 1.

[0123] Table 1. Comparative experimental results of CDA-DETR on the DroneVehicle dataset: Method P R mAP50 mAP50-95 YOLOv8-Add 71.1 69.6 71.8 50.2 YOLOv10-Add 72.4 70.6 73.4 50.4 DETR-Add 70.4 69.1 70.3 49.6 RT-DETR-Add 76.7 74.8 73.5 53.2 CDA-DETR 81.9 79.9 79.4 59.4

[0124] The experimental data in Table 1 clearly demonstrates that the CDA-DETR proposed in this study comprehensively surpasses the modified baseline model in all core indicators. A detailed analysis follows:

[0125] First, in terms of the mAP50 metric, which measures the overall localization and classification capabilities of the models, the pure convolutional architectures YOLOv8-Add and YOLOv10-Add achieved scores of 71.8% and 73.4%, respectively. Limited by the local receptive field of the convolutional kernels and the simplistic addition-fusion strategy, these models exhibit significant performance bottlenecks when handling large-scale background interference from a UAV perspective. In contrast, the baseline model RT-DETR-Add, employing a Transformer architecture, achieved an mAP50 of 73.5% thanks to its global receptive field provided by its self-attention mechanism. However, because it relies solely on direct addition operations and lacks an explicit complementary mechanism for the differences between the two modes, it remains extremely limited when facing information conflicts between infrared and visible light modes. Without adding any additional data, CDA-DETR achieved a significant increase in mAP50 to 79.4%, representing a 5.9% absolute performance improvement compared to the baseline RT-DETR-Add. Furthermore, on the more demanding high-precision localization metric mAP50-95, CDA-DETR reached 59.4%, surpassing the second-best baseline by 6.2%, fully demonstrating the model's superior performance in pixel-level bounding box regression of small targets.

[0126] Secondly, a deep analysis of precision and recall metrics reveals that CDA-DETR achieved 81.9% and 79.9% respectively, demonstrating a highly balanced and excellent detection level. This result strongly validates the mathematical effectiveness of the core mechanism presented in this paper compared to a simple addition strategy from a macro-data perspective. On one hand, the 79.9% recall rate is mainly attributed to the bidirectional dynamic complementary mechanism of the cross-modal difference attention module. This module completely eliminates the background noise pollution caused by direct addition, successfully rescuing missed detections caused by visible light failure in extremely low-light conditions at night, and more effectively overcoming the loss of target contours due to infrared thermal cross-effects in complex terrain during the day. Regardless of the type of single-modal attenuation, the module can adaptively extract robust features of the opposing modality, thereby reducing the missed detection rate across all scenarios. On the other hand, the 81.9% precision rate is mainly attributed to the semantically guided spatial weighted filtering mechanism. This mechanism effectively suppresses clutter and background noise in shallow features using deep semantics, reducing false positives.

[0127] In summary, compared with traditional single-modal networks and simple bimodal extensions, CDA-DETR can effectively overcome the interference of drastic target scale changes and extreme physical environments in UAV bimodal scenarios, demonstrating excellent feature robustness and extremely high detection accuracy in complex scenarios.

[0128] Example 3:

[0129] To more intuitively verify the detection performance of the proposed model in complex road and dense vehicle scenarios, this embodiment selects two highly representative sets of bimodal samples from the DroneVehicle validation set for qualitative and quantitative cross-analysis. The models involved in the comparison include the classic convolutional architecture YOLOv10-Add, the homologous baseline model RT-DETR-Add, and the CDA-DETR proposed in this study.

[0130] Figure 2 The detection results are shown in the scenario of a complex main road intersection. Figure 2 In (a), (b), and (c), the upper figure shows the visible light detection results, and the lower figure shows the infrared detection results. The scene features a dense distribution of targets and a complex background structure, placing high demands on the model's localization accuracy and background suppression capabilities. It can be observed that YOLOv10-Add generated many redundant detection boxes on this sample, resulting in relatively messy detection results. While the baseline model RT-DETR-Add can complete most target detections, it still exhibits some false positives and false negatives. According to the cross-union ratio threshold matching statistics, this baseline model generated 4 false positive predictions and missed 2 real targets in this sample. In contrast, the CDA-DETR proposed in this study demonstrates superior scene adaptability. It detected one more real target, reduced the number of false positives to 2, and reduced the number of false negatives to 1. This indicates that the proposed method has more stable target representation and discrimination capabilities in complex scenes.

[0131] Figure 3 Another set of supplementary samples with strong local interference was further presented, in which... Figure 3 In (a), (b), and (c), the upper figure shows the visible light detection results, and the lower figure shows the infrared detection results. In the detection results of the baseline model RT-DETR-Add, the model exhibited 3 significant false positives and 1 false negative. In contrast, CDA-DETR achieved complete coverage of all labeled targets within the sample, with zero false negatives and reducing the number of false positives to 1. These results demonstrate that the proposed method has significant advantages in suppressing local artifact interference and improving the recall of true targets.

[0132] The combined results from both sets of samples show that CDA-DETR exhibits more stable detection results in both RGB and infrared modes. Its qualitative performance is consistent with the quantitative experimental conclusions above, further verifying the robustness and advancement of the proposed architecture in UAV dual-modal target detection tasks.

[0133] Example 4:

[0134] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0135] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the dual-modal end-to-end target detection method for UAVs and complex scenarios described in the various embodiments of this application.

[0136] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned dual-modal end-to-end target detection method for UAVs and complex scenarios.

[0137] Example 5:

[0138] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned dual-modal end-to-end target detection method for UAVs and complex scenarios.

[0139] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0140] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0141] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the methods disclosed herein and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A dual-modal end-to-end target detection method for UAVs and complex scenarios, characterized in that, Includes the following steps: Step 1: Acquire and preprocess UAV dual-modal image data; Step 2: Construct the dual-modal target detection framework CDA-DETR; The dual-modal target detection framework CDA-DETR includes a two-stream convolutional network, a feature fusion layer, a multi-scale neck network, and a detection head. The two-stream convolutional network is used to achieve dual-modal input separation and parallel dual-branch feature extraction. The feature fusion layer embeds a cross-modal differential attention module (CDA) for deep feature complementation. The multi-scale neck network introduces a semantically guided spatial weighted filtering mechanism (SSWF) to perform top-down information transfer and multi-scale integration of the feature pyramid. The detection head is used to perform end-to-end prediction; Step 3: Perform forward inference and target detection based on the dual-modal target detection framework CDA-DETR.

2. The dual-modal end-to-end target detection method for UAVs and complex scenarios according to claim 1, characterized in that, Step 1 specifically involves: acquiring paired visible light and infrared images collected by the UAV, and ensuring absolute alignment of the bimodal images in pixel-level space; then, concatenating the paired visible light and infrared images in the channel dimension to construct a bimodal input tensor.

3. The dual-modal end-to-end target detection method for UAVs and complex scenarios according to claim 1, characterized in that, The dual-stream convolutional network first acquires paired visible light and infrared images, then concatenates them along the channel dimension to construct a bimodal input tensor. Where B represents the batch size, 6 is the total number of channels after splicing, the indices of the 6 channels are defined as 0-5 respectively, and H and W represent the spatial height and width of the bimodal input tensor respectively. After the bimodal input tensor X is input into a two-stream convolutional network, it is passed through an explicit channel separation function f split The bimodal input tensor is rigidly cut along the channel dimension, separating it into two independent data streams; the channel separation function f split As shown in formula (1): （1）； Among them, channel indices 0 to 2 are divided into visible light tensors X. vis Channel indices 3 to 5 are divided into infrared tensors X. ir Subsequently, the visible light tensor X vis and infrared tensor X ir The data are fed into parallel two-stream convolutional networks, where independent and parallel layer-by-layer downsampling feature extraction is performed simultaneously. The two-stream convolutional network consists of a series of convolutional layers connected in series with C2f modules. Two-stream convolutional networks output feature pyramids at four key scales: visible light features. With infrared features In this context, visible light features and infrared features at the same level together constitute a bimodal feature pair at that level, and the first-level feature pair... Let this be the shallow feature P1, and the third, fourth, and fifth level feature pairs... , , These are denoted as deep features P3, P4, and P5, respectively. Finally, the extracted deep features P3, P4, and P5 will be fed into the cross-modal differential attention module (CDA) as the main features containing high-level semantics. The visible light features in the shallow feature P1 will be... With infrared features Perform element-wise addition to generate a fused shallow feature layer. Then, it is fed into a multi-scale neck network for background noise suppression and edge detail integration.

4. The dual-modal end-to-end target detection method for UAVs and complex scenarios according to claim 3, characterized in that, The feature fusion layer is specifically defined as follows: Let the input visible light feature tensor be t1 and the aligned infrared feature tensor be t2. Cross-modal differential attention module (CDA) utilizes independent The convolutional layer combines batch normalization and ReLU activation function to perform feature space alignment and linear mapping on the input visible light feature tensor t1 and infrared feature tensor t2, generating query feature Q and key feature K respectively, as shown in formula (2) and formula (3): （2）； （3）； Calculate the element-wise absolute difference matrix of the visible light characteristic tensor t1 and the infrared characteristic tensor t2. ; It is fed into the squeeze-incentive attention module SE for channel dimension weight redistribution, and the difference value feature V focusing on the target ontology is constructed, as shown in formula (4): （4）； The cross-modal difference attention module CDA introduces spatial cosine similarity. By introducing the L2 norm, the query feature Q and the key feature K are normalized. The summation operation Sum is performed on the channel dimension to obtain the initial similarity distribution Attn, which measures the consistency of the bimodal spatial structure, as shown in formula (5): （5）； The cross-modal difference attention module CDA uses a constant 1 minus the initial similarity distribution Attn; it decays the weights of high similarity regions to near 0, while simultaneously reversing the activation of the weights of low similarity regions to approach 1, thereby generating a targeted difference significance attention mask matrix M', as shown in formula (6): （6）； The Cross-Modal Differential Attention Module (CDA) introduces a residual connection structure. It performs a pixel-wise multiplication (⊙) of the differential saliency attention mask matrix M' with the differential value feature V, then performs a weighted summation, and finally adds it element-wise to the differential value feature V itself, outputting the cross-modal fusion feature tensor at that scale. As shown in formula (7): （7）； After computation by three parallel cross-modal differential attention modules (CDA), the feature fusion layer finally outputs a deep multi-scale enhanced fusion feature that is deeply complementary and aligned. This deep multi-scale enhanced fusion feature is composed of cross-modal fusion feature tensors at three different depth scales. A set that is formed together.

5. The dual-modal end-to-end target detection method for UAVs and complex scenarios according to claim 4, characterized in that, The multi-scale neck network is connected to the output of the feature fusion layer and first receives... and the highest layer features After adjusting the spatial resolution through the upsampling layer, and The features are concatenated along the channel dimension and then fed into the C2f module for processing; the processed features are then upsampled again and compared with... The same concatenation and C2f processing are performed to complete the top-down integration of deep features, outputting an integrated deep feature tensor containing the global receptive field. ; Then, the integrated deep feature tensor is utilized. shallow features Perform spatial weighted filtering; integrate deep feature tensors The input is fed into a separate 1×1 convolutional layer and processed by the Sigmoid activation function. The processed feature values ​​are then projected onto the (0,1) interval to generate a spatial weight matrix. As shown in formula (8): （8）； Obtain the spatial weight matrix Then, it is compared with shallow features. Perform pixel-by-pixel multiplication along the spatial dimension to generate shallow enhancement features after noise reduction. As shown in formula (9): （9）； Finally, the shallow enhancement features With integration of deep feature tensors The components are concatenated along the channel dimension to output the final integrated noise-free multi-scale enhanced feature tensor.

6. The dual-modal end-to-end target detection method for UAVs and complex scenarios according to claim 5, characterized in that, The detection head specifically adopts the RT-DETR decoder structure and is connected to the output of the multi-scale neck network, including the forward inference stage and the backward propagation stage; In the forward inference stage, the detection head receives a noiseless multi-scale enhanced feature tensor output from the multi-scale neck network, performs self-attention and cross-attention decoding matrix operations, and then outputs a set of fixed-number prediction vectors containing the target bounding box coordinates and corresponding class confidence through a linear mapping layer, thereby achieving complete end-to-end feature parsing. In the backpropagation phase, a loss function is constructed to calculate the error between the detection head's predicted output and the labeled ground truth, thereby guiding the weight update of the CDA-DETR model; specifically, the loss function is composed of classification loss. Regression loss with bounding box The weighted mixed loss function L; The hybrid loss function L is shown in equation (10): （10）； in, and These are the weighting coefficients for the classification loss and the bounding box regression loss, respectively. For classification loss The Focal Loss function is used, as shown in Equation (11): （11）； in, This represents the model's predicted probability for the target category. As a class balance factor, As a focusing factor; For bounding box regression loss The combination of CIoU loss and smoothing L1 loss is adopted, as shown in formulas (12), (13), and (14): （12）； （13）； （14）； Where IoU is the intersection-over-union ratio of the predicted bounding box and the ground truth bounding box, and b is the coordinate of the center point of the predicted bounding box. The coordinates of the center point of the true bounding box. ρ is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, c is the diagonal length of the minimum bounding box, and v is the aspect ratio consistency factor. To balance the aspect ratio penalty term, x is the difference between the predicted coordinates and the true coordinates; The loss function is responsible for constraining the overlap between the predicted bounding box and the ground truth bounding box, the Euclidean distance between the center points, and the aspect ratio at the global feature scale; the smoothing L1 loss function is responsible for accurately narrowing down the absolute distance of the coordinates at the micro-pixel scale. Calculate the bounding box regression loss At that time, a scale-adaptive weighting mechanism based on the area of ​​the ground truth bounding box was introduced; the size of the area of ​​the ground truth bounding box was used as a dynamic evaluation benchmark for the inverse penalty coefficient: the smaller the target area, the larger the weight multiplier assigned to the target in the global regression loss; The scale-adaptive hybrid loss value is obtained by using a hybrid loss function, and the backpropagation of the entire CDA-DETR detection model and the update and iteration of the network weight parameters are completed.

7. The dual-modal end-to-end target detection method for UAVs and complex scenarios according to claim 1, characterized in that, Step 3 specifically involves: inputting the bimodal input tensor to be detected into the trained bimodal object detection framework CDA-DETR for forward inference; the bimodal tensor sequentially passes through a dual-stream convolutional network for feature extraction, through the cross-modal differential attention module CDA in the feature fusion layer for deep complementary fusion, and through the semantically guided spatial weighted filtering mechanism SSWF in the multi-scale neck network for background noise suppression and multi-scale integration. Then, the detection head receives a noise-free multi-scale enhanced feature pyramid, performs self-attention and cross-attention decoding matrix operations, and directly outputs a fixed number of prediction vectors containing the target bounding box coordinates and the corresponding class confidence. This model achieves end-to-end feature parsing, and only uses the confidence threshold to filter and parse the prediction vectors, ultimately outputting the object detection result.

8. A computer-readable storage medium storing executable instructions, characterized in that, When the instruction is executed, it causes the processor to perform the dual-modal end-to-end target detection method for UAVs and complex scenarios as described in any one of claims 1-7.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the dual-modal end-to-end target detection method for UAVs and complex scenarios as described in any one of claims 1-7.

Images