A multi-source remote sensing target detection method based on asymmetric manifold co-alignment and global calibration

Abstract

The application relates to the technical field of image processing, and discloses a multi-source remote sensing target detection method based on asymmetric manifold cooperation and global calibration, which comprises the following steps: collecting a remote sensing target image; inputting the remote sensing target image into a cooperative adaptive cross-modal network improved based on a YOLO12 target detection network to output the predicted frame coordinates of the remote sensing target; the cooperative adaptive cross-modal network comprises a double-branch backbone network, a neck network and a head network; wherein the double-branch backbone network comprises a visible light branch, an infrared thermal imaging branch and a spatial cooperative asymmetric parallel perception unit; the spatial cooperative asymmetric parallel perception unit is added to the neck network; and the detection head in the head network is a globally calibrated adaptive detection head. The application solves the problem of the topological shackles of rigid symmetrical fusion in the prior art, and cannot realize deep complementary calibration and reliable screening for heterogeneous data.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a multi-source remote sensing target detection method based on asymmetric manifold collaboration and global calibration. Background Technology

[0002] In recent years, with the rapid development of aerospace technology and Earth observation systems, remote sensing image analysis technology based on satellite remote sensing and high-altitude unmanned aerial vehicle platforms has become a core research hotspot in the field of computer vision, and has been widely applied in key areas such as urban planning, disaster relief, and precision agriculture. Under the macroscopic viewpoint of remote sensing, target detection tasks face unique physical constraints and algorithmic challenges. The most critical challenge lies in the multi-scale characteristics of targets in remote sensing images. A single high-resolution remote sensing image contains both macroscopic targets occupying a large number of pixels (such as large building complexes, bridges, and ports) and a large number of tiny targets with extremely low pixel counts (usually less than 32×32, or even 8×8 pixels) (such as vehicles, pedestrians, and isolated facilities). This extreme scale span, coupled with the overhead distortion of the remote sensing perspective, high-frequency background clutter, and the dense arrangement of targets, makes existing general-purpose target detection networks prone to the dilemma of "redundant features for large targets and diffuse features for small targets" when constructing feature pyramids and allocating receptive fields. The multi-scale integrated recognition accuracy and robustness in engineering applications still face significant bottlenecks.

[0003] In terms of data source, most mainstream remote sensing multi-scale detection networks are currently built upon single-modal images such as visible light (RGB). Visible light images, with their ultra-high resolution, rich colors, and fine texture information, have significant advantages in representing the visual details of targets. However, remote sensing Earth observation tasks typically need to meet all-day, all-weather engineering requirements. Visible light sensors are extremely sensitive to lighting conditions and atmospheric scattering. When faced with common remote sensing interferences such as low light (e.g., dawn / dusk, night), extreme weather (e.g., cloud cover, rain / snow), and complex lighting (e.g., shadows from tall buildings in cities), image quality deteriorates drastically, leading to severe loss of target features. In contrast, infrared thermal imaging (IR) technology relies on the difference in thermal radiation between the target and the background, possessing strong penetration and resistance to lighting interference, and can stably extract the salient contours of targets in all weather conditions. However, limited by the physical characteristics of sensors, infrared images generally have lower resolution and lack detailed textures and contextual semantics of the scene. Therefore, in view of the inherent environmental limitations of multi-scale target detection tasks from the perspective of remote sensing, the fusion technology of infrared thermal imaging (IR) and visible light (RGB) multimodal data has become an inevitable development trend to break through the observation barrier of a single sensor and improve the detection of complex scenes in all weather conditions.

[0004] Although multimodal fusion detection theoretically possesses immense potential, the current state of technology in this field is still in its early exploratory stages. Existing fusion paradigms exhibit significant limitations when addressing multi-scale remote sensing tasks. Currently, commonly used fusion strategies primarily focus on decision-level fusion and simple feature-level fusion.

[0005] The first type of decision layer (result layer) fusion only performs post-processing on the single-modal output results at the end of the network prediction (such as NMS voting). This approach completely severs the modal interaction feedback of deep networks in the feature extraction process, resulting in the loss of a large amount of implicit complementary semantics. When facing densely distributed multi-scale targets in remote sensing images, it is very easy to miss detection.

[0006] The second and most mainstream type is simple feature layer fusion, which typically uses channel concatenation (Concat) or element-wise addition (Add) operations directly in specific shallow or deep layers of the network. However, there is a significant spatial offset and representation gap between visible light and infrared thermal imaging in remote sensing images. The Concat operation indiscriminately stacks the two feature paths, which not only fails to eliminate the bias between heterogeneous sensors, but also injects high-frequency clutter from visible light and low-frequency thermal noise from infrared into the network, causing serious feature contamination. The Add operation, on the other hand, crudely aligns and mixes pixel-level features, destroying the inherent semantic independence of the two modalities.

[0007] More critically, most existing fusion strategies employ static, non-adaptive aggregation methods, lacking pixel-level reliability assessment mechanisms. In the complex context of remote sensing, the effectiveness of features from different modalities at different spatial locations and scale levels is dynamically changing (e.g., visible light is ineffective in cloud-covered areas, while infrared thermal imaging is effective). Furthermore, current fusion modules are mostly preprocessing plugins, detached from the main detection backbone, failing to achieve deep task coupling with the downstream multi-scale feature pyramid structure (FPN) and detection head. This fragmented architecture of "coarse upstream fusion and rigid downstream detection" cannot adaptively allocate multi-modal feature weights according to the actual needs of classification and regression tasks. Ultimately, in remote sensing scenarios, large-scale target boundaries are inaccurately located, and small-scale target features are overwhelmed by background noise, making it difficult to achieve truly high-performance multi-scale collaborative detection.

[0008] Therefore, the existing technology system is limited by the rigidity of symmetrical feature superposition and task decoupling mechanism, which makes it impossible to achieve in-depth complementary calibration and reliability screening for heterogeneous data, and it is difficult to meet the urgent needs of high sensitivity and high robustness for target detection in the field of view of actual complex UAV aerial photography. Summary of the Invention

[0009] The purpose of this invention is to solve the topological constraints of the existing "rigid symmetric fusion" technology, which makes it impossible to achieve in-depth complementary calibration and reliability screening for heterogeneous data, and to provide a multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration.

[0010] To achieve the above-mentioned objectives, the embodiments of the present invention provide the following technical solutions:

[0011] A multi-source remote sensing target detection method based on asymmetric manifold collaboration and global calibration includes the following steps:

[0012] Acquire remote sensing target images; input the remote sensing target images into a collaborative adaptive cross-modal network based on the improved YOLO12 target detection network, and output the predicted bounding box coordinates of the remote sensing targets;

[0013] The collaborative adaptive cross-modal network includes a dual-branch backbone network, a neck network, and a head network; wherein, the dual-branch backbone network includes a visible light branch, an infrared thermal imaging branch, and a spatial collaborative asymmetric parallel sensing unit; the neck network adds a spatial collaborative asymmetric parallel sensing unit; and the detection head in the head network is a global calibration task adaptive detection head.

[0014] In the above scheme, a collaborative adaptive cross-modal network (SAC-Net) integrating a dual-branch backbone network, a neck network, and a global calibration task adaptive detection head (OCTA-Head) is proposed. Spatial collaborative asymmetric parallel sensing units (SAPSE) are deployed in the dual-branch backbone network and the neck network to replace the traditional splicing operator, and connected to the global calibration task adaptive detection head (OCTA-Head) to achieve accurate calibration at the global scale.

[0015] Furthermore, the spatially collaborative asymmetric parallel sensing unit is represented by SAPSE;

[0016] The visible light branch includes Multiin-1, Conv-1, Conv-2, C3k2-1, Conv-3, C3k2-2, Conv-4, A2C2f-1, Conv-5, and A2C2f-2 connected in sequence;

[0017] The infrared thermal imaging branch includes Multiin-2, Conv-6, Conv-7, C3k2-3, Conv-8, C3k2-4, Conv-9, A2C2f-3, Conv-10, and A2C2f-4 connected in sequence;

[0018] A2C2f-2 and A2C2f-4 are connected to SAPSE-1 respectively.

[0019] Furthermore, the neck network includes A2C2f-5, A2C2f-6, A2C2f-7, A2C2f-8, SAPSE-2, SAPSE-3, Conv-11, A2C2f-9, Conv-12, and C3k2-5;

[0020] Feature map F1 output from A2C2f-2 is upsampled and fused with feature map F7 output from A2C2f-1 to obtain feature map G1; feature map F2 output from A2C2f-4 is upsampled and fused with feature map F3 output from A2C2f-3 to obtain feature map G2; feature map G2 is input into A2C2f-5 to obtain feature map G3, feature map G3 is upsampled and fused with feature map F5 output from C3k2-5 to obtain feature map G4; feature map G1 is input into A2C2f-7 to obtain feature map G5, feature map G5 is upsampled and fused with feature map F4 output from C3k2-2 to obtain feature map G6; feature map G4 is input into A2C2f-5. 2f-6 yields feature map G7, and feature map G6 is input into A2C2f-8 to obtain feature map G8; feature maps G7 and G8 are input together into SAPSE-3 to obtain feature map P3, and feature maps G3 and G5 are input together into SAPSE-2 to obtain feature map G10; feature map P3 is fused with feature map G10 after passing through Conv-11 to obtain feature map G11; feature map G11 is input into A2C2f-9 to obtain feature map P4; feature map P4 is fused with feature map F6 output from SAPSE-1 after passing through Conv-12 to obtain feature map G13; feature map G13 is input into C3k2-5 to obtain feature map P5.

[0021] Furthermore, OTCA-Head is used to represent the adaptive detection head for the global calibration task;

[0022] The head network includes OTCA-Head-1, OTCA-Head-2, and OTCA-Head-3;

[0023] The feature map output by SAPSE-3 is input to OTCA-Head-1 at P3; the feature map output by A2C2f-9 is input to OTCA-Head-2 at P4; and the feature map output by C3k2-5 is input to OTCA-Head-3 at P5.

[0024] Furthermore, the data processing flow of the spatially coordinated asymmetric parallel sensing unit is as follows:

[0025] (1) Asymmetric multi-level sensing field coding;

[0026] For the elementary flow X ptg Extracting edge features:

[0027] ;

[0028] Among them, F ptg The edge features of the primitive flow; D-DWConv 3×3,d=1 This is a depthwise dilated convolution operation;

[0029] For associated flow X prg Extract detailed features:

[0030] ;

[0031] Among them, F prg For associated flow detail features; D-DWConv 9×9,d=4 This is a sparse dilated convolution operation;

[0032] (2) Cross-modal collaborative calibration interaction;

[0033] Generate spatial alignment weights M cal :

[0034] ;

[0035] in, For sigmoid activation function; Conv 1×1 This is a convolution operation; This is pixel-by-pixel multiplication;

[0036] Using spatial alignment weight M cal Edge features of the elementary flow F ptg and associated flow details F prg Perform initial feature enhancement:

[0037] ;

[0038] ;

[0039] in, The edge features of the primitive flow after initial enhancement; For the preliminary enhanced details of the associated flow;

[0040] (3) Dimensionality reduction mapping of manifold space;

[0041] The edge features of the pre-enhanced primitive flow are processed through convolutional layers. and associated flow details Mapping to a low-dimensional manifold space:

[0042] ;

[0043] ;

[0044] in, The primitive flow edge features of a low-dimensional manifold space; For the associated flow detail features of low-dimensional manifold space; Let be the set of real numbers; C, H, and W represent the number of channels, height, and width of the feature map, respectively.

[0045] (4) Ternary reliability space gating;

[0046] Extracting ternary descriptors from low-dimensional manifold spaces:

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] Among them, F concat for and Characteristics of channel cascaded; S avg S is the mean descriptor; max S is the maximum value descriptor; var Mean is a variance descriptor. channel This indicates a mean pooling operation along the channel dimension; Max channel Represents the channel-level max pooling operation; Variation channel This indicates the calculation of variance in the channel dimension;

[0052] Generate collaborative weight graph :

[0053] ;

[0054] Among them, Conv 7×7 This is a convolution operation;

[0055] The collaborative weight graph is sliced ​​and converted into... ,in The weights of the primitive flow space; For associated flow spatial weights; These are the global confidence gating coefficients;

[0056] (5) Dynamic injection fusion and output;

[0057] Using collaborative weight graph Perform adaptive fusion:

[0058] ;

[0059] Among them, F fused Features of fusion;

[0060] Then confidence modulation is performed, and the original number of channels C is restored through a convolutional layer:

[0061] ;

[0062] Among them, F att To enhance attention;

[0063] Enhance attention map F att Injection back to the primitive flow X ptg :

[0064] ;

[0065] Where Y represents the fusion enhancement feature output by the spatially collaborative asymmetric parallel sensing unit.

[0066] In the above scheme, a spatial collaborative asymmetric parallel sensing unit (SAPSE) based on ternary reliability spatial gating is designed. It utilizes the parallel encoding of heterogeneous features of the asymmetric sensing field and dynamically evaluates pixel-level statistics through the ternary reliability spatial gating mechanism to achieve asymmetric interception and high-fidelity alignment of visible light background clutter and infrared thermal noise.

[0067] Furthermore, the inputs of SAPSE-1 are connected to A2C2f-4 and A2C2f-2 respectively, where the elementary flow X ptg It is the feature map F2 output by A2C2f-4, with associated flow X. prg It is the feature map F1 output by A2C2f-2;

[0068] The inputs of SAPSE-2 are connected to A2C2f-5 and A2C2f-7 respectively, where the primitive flow X ptg It is the feature map G3 output by A2C2f-5, with associated flow X. prg It is feature map G5 output by A2C2f-7;

[0069] The inputs of SAPSE-3 are connected to A2C2f-6 and A2C2f-8 respectively, where the primitive flow X ptg It is the feature map G7 output by A2C2f-6, with associated flow X. prg It is the feature map G8 output by A2C2f-8.

[0070] Furthermore, the data processing flow of the adaptive detection head for the global calibration task is specifically as follows:

[0071] (1) Global scale perception calibration stage;

[0072] Receive feature maps X from the neck network at three scales. i ∈{P3,P4,P5}, calculate the learnable normalized weights a for each level.i :

[0073] ;

[0074] Where W = {w3, w4, w5}, i = 3, 4, 5, W is a learnable parameter vector, and w i These are the learnable parameters of the i-th layer;

[0075] Through learnable normalized weights a i The calibrated feature map is as follows:

[0076] ;

[0077] in, For calibrating feature maps;

[0078] (2) Adaptive processing of the regression branch;

[0079] Based on calibration feature map Calculate the fundamental characteristics:

[0080] ;

[0081] Among them, Y base Basic features; BN is the normalization operation; GroupConv 3×3 For topological grouping convolution operations; Conv 1×1 This is a convolution operation;

[0082] Dual-channel pooling aggregation extracts global features:

[0083] ;

[0084] Where Z represents the global feature; Preset weights; Gap is for global average pooling calculation; Gmp is for global max pooling calculation;

[0085] By activating features through a fully connected (FC) layer, and remapping them back to the original number of channels C, a normalized preliminary channel attention score vector S is obtained:

[0086] ;

[0087] Where ReLU is the ReLU activation function; FC1 is the dimensionality reduction layer; and FC2 is the dimensionality increase layer.

[0088] After temperature coefficient The calibration yields the final channel attention vector E:

[0089] ;

[0090] in, Use the Sigmoid activation function; The bias coefficient for the regression task;

[0091] For the basic feature Y base After recalibration, the output is obtained through residual and linear correction:

[0092] ;

[0093] ;

[0094] Among them, Y reg_i express The corresponding regression feature results; This indicates the features after recalibration and weighting; "shortcut" indicates the residual join operation.

[0095] (3) Lightweight deep decoupling layer;

[0096] Generate regression plot:

[0097] ;

[0098] Among them, B i For Y-based reg_i The generated regression plot; B represents the batch size; reg_max is the maximum number of discrete distance grids; H i B i Height; W i B i The width;

[0099] In parallel, generate target classification probability maps:

[0100] ;

[0101] Among them, C i Based on The generated target classification probability map; H i C i Height; W i C i The width;

[0102] Output prediction tensor:

[0103] ;

[0104] Among them, O i This is the output prediction tensor; Concat is the concatenation operation.

[0105] (4) Bounding box decoding;

[0106] Calculate the distance from any boundary of the predicted bounding box to the center point:

[0107] ;

[0108] Where d represents the distance from any boundary of the predicted bounding box to the center point; reg_max-1 represents the upper limit of the integral summation, j=0,1,2,...,reg_max-1; softmax(B i,j ) represents the probability that the boundary of the predicted bounding box falls exactly at position j.

[0109] In the above scheme, a global calibration task adaptive detection head (OCTA-Head) based on temperature coefficient modulation is designed. It smooths the semantic gaps between pyramid layers by normalizing across scales and uses a dynamic attention mechanism with temperature coefficient modulation to adaptively allocate feature weights for classification and regression tasks, thereby resolving gradient conflicts between multi-scale tasks from the root.

[0110] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0111] (1) This invention breaks through the topological constraints of the existing technology of "rigid symmetric fusion" and proposes an adaptive purification and deconstruction mechanism for multimodal feature manifolds. Existing technologies generally rely on channel concat for modal superposition, which is prone to physical coupling of high-frequency noise. The SAPSE deployed in this invention achieves dynamic evaluation and pixel-level interception of statistical physical quantities of feature maps by introducing ternary reliability space gating. This technique successfully upgrades the traditional blind stacking fusion to high-fidelity complementarity with on-demand filtering, fundamentally blocking the cross-contamination of visible light clutter and infrared thermal noise, and endowing the model with extreme noise robustness under extreme lighting and meteorological interference.

[0112] (2) This invention constructs an asymmetric smooth span representation model with global perception, improving semantic collapse under multiple scales. Addressing the problems of "small target feature diffusion" and "large target local fragmentation" caused by large changes in target span under remote sensing overhead views, this invention proposes a cross-dimensional scale-breaking solution. Utilizing parallel perception through asymmetric convolution within SAPSE, combined with cross-level global scale normalization calibration in OCTA-Head, the model constructs a smooth feature weight dynamic distribution pool, i.e., a collaborative weight graph. This mechanism breaks down the semantic barriers between traditional pyramid levels, ensuring perfect consistency between micro-response and macro-profile across the entire size range, and significantly improving the recall rate of dense, small targets.

[0113] (3) This invention introduces a channel modulation architecture based on thermodynamic temperature evolution, solving the problem of backward gradient conflict at the detection head. In the final stage of target detection, the translation invariance of the classification task and the spatial sensitivity of the regression task are naturally mutually exclusive. The OCTA-Head of this invention innovatively incorporates a dynamic attention mechanism with temperature coefficient modulation. The network can adaptively and softly adjust the saturation range of the activation function according to the signal-to-noise ratio and convergence state of the current task features. This innovative approach achieves efficient scheduling and gradient decoupling of task-specific channel resources, breaking the performance limitations of traditional static detection heads and achieving more accurate bounding box extreme fitting under complex high IoU thresholds.

[0114] (4) This invention achieves a paradigm shift from unidirectional propagation to global adaptive calibration, establishing an excellent energy efficiency ratio. This architecture effectively avoids the problem of exponentially increasing computing power often associated with conventional multimodal models by cleverly embedding a low-parameter SAPSE and a decoupled OCTA-Head into the core network computation flow. Without sacrificing real-time inference efficiency, this invention completes end-to-end closed-loop calibration from upstream heterogeneous feature extraction and midstream multi-scale dynamic noise reduction to downstream global task adaptive prediction, providing a solution for all-weather high-precision Earth observation that combines high-dimensional computing power performance with ultimate engineering efficiency. Attached Figure Description

[0115] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0116] Figure 1 This is a structural diagram of the Collaborative Adaptive Cross-Modal Network (SAC-Net) of the present invention;

[0117] Figure 2 This is a data processing flowchart of the Spatial Collaborative Asymmetric Parallel Sensing Unit (SAPSE) of the present invention;

[0118] Figure 3 This is a data processing flowchart for the Global Calibration Task Adaptive Detection Head (OCTA-Head) of this invention;

[0119] Figure 4 This is a structural diagram of the traditional YOLO12 object detection network. Detailed Implementation

[0120] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0121] It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance, or suggesting any such actual relationship or order between these entities or operations. Additionally, the terms "connected," "linked," etc., can refer to a direct connection between elements, components, modules, etc., or an indirect connection via other elements, components, modules, etc.

[0122] Example 1:

[0123] This invention is achieved through the following technical solutions, such as... Figure 1 As shown, a multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration is proposed. The method acquires remote sensing target images, inputs the remote sensing target images into a collaborative adaptive cross-modal network based on the YOLO12 target detection network, and outputs the predicted bounding box coordinates of the remote sensing targets.

[0124] The proposed collaborative adaptive cross-modal network in this scheme is an improvement upon the traditional YOLO12 target detection network, such as... Figure 4 As shown, the traditional YOLO12 object detection network is a single-branch network that supports single-modal data detection. It consists of a backbone, a neck, and a head. The backbone employs a modular, progressive downsampling design, performing basic image transformations and downsampling, which are then fed into a stacking module for efficient feature learning. Finally, the A2C2f module aggregates the global context. The neck primarily performs feature fusion and enhancement, also utilizing the C3k2 module. It completes connections by aggregating features from different scales and optimizing the transfer process. The head performs bounding box and classification predictions on the refined feature maps and feature information passed to subsequent layers, ultimately outputting the results.

[0125] This solution improves the backbone network of the traditional YOLO12 object detection network. Under the condition of simultaneous input of bimodal (RGB and IR) data, it extracts and fuses bimodal features to form a Synergetic Adaptive Cross-modal Network (SAC-Net), achieving same-dimensional feature enhancement and cross-dimensional feature fusion at different stages. Figure 1 As shown, the collaborative adaptive cross-modal network includes a dual-branch backbone network, a neck network, and a head network.

[0126] The hierarchical connection relationship within the dual-branch backbone network is as follows: The dual-branch backbone network includes a visible light branch, an infrared thermal imaging branch, and a Spatial-Synergetic Asymmetric Parallel Sensing Engine (SAPSE); the visible light branch includes Multiin-1, Conv-1, Conv-2, C3k2-1, Conv-3, C3k2-2, Conv-4, A2C2f-1, Conv-5, and A2C2f-2 connected in sequence; the infrared thermal imaging branch includes Multiin-2, Conv-6, Conv-7, C3k2-3, Conv-8, C3k2-4, Conv-9, A2C2f-3, Conv-10, and A2C2f-4 connected in sequence; A2C2f-2 and A2C2f-4 are respectively connected to SAPSE-1.

[0127] The hierarchical connection relationship within the neck network is as follows: The neck network includes A2C2f-5, A2C2f-6, A2C2f-7, A2C2f-8, SAPSE-2, SAPSE-3, Conv-11, A2C2f-9, Conv-12, and C3k2-5; Feature map F1 output by A2C2f-2 is upsampled and fused with feature map F7 output by A2C2f-1 to obtain feature map G1; Feature map F2 output by A2C2f-4 is upsampled and fused with feature map F3 output by A2C2f-3 to obtain feature map G2; Feature map G2 is input into A2C2f-5 to obtain feature map G3; Feature map G3 is upsampled and fused with feature map F5 output by C3k2-5 to obtain feature map G4; Feature map G1 is input into A2C2f-7 to obtain... Feature map G5 is upsampled and fused with feature map F4 output from C3k2-2 to obtain feature map G6. Feature map G4 is input into A2C2f-6 to obtain feature map G7, and feature map G6 is input into A2C2f-8 to obtain feature map G8. Feature maps G7 and G8 are input together into SAPSE-3 to obtain feature map P3. Feature maps G3 and G5 are input together into SAPSE-2 to obtain feature map G10. Feature map P3 is fused with feature map G10 after passing through Conv-11 to obtain feature map G11. Feature map G11 is input into A2C2f-9 to obtain feature map P4. Feature map P4 is fused with feature map F6 output from SAPSE-1 after passing through Conv-12 to obtain feature map G13. Feature map G13 is input into C3k2-5 to obtain feature map P5.

[0128] The hierarchical connection relationship within the head network is as follows: The head network includes OTCA-Head-1, OTCA-Head-2, and OTCA-Head-3; the feature map P3 output by SAPSE-3 is input to OTCA-Head-1, the feature map P4 output by A2C2f-9 is input to OTCA-Head-2, and the feature map P5 output by C3k2-5 is input to OTCA-Head-3.

[0129] It should be noted that the "-1", "-2", etc. at the end of each layer in the Collaborative Adaptive Cross-Modal Network (SAC-Net) are only used to distinguish layers with the same structure. For example, Conv-1 and Conv-2 belong to the same Conv structure; SAPSE-1, SAPSE-2, and SAPSE-3 belong to the same SAPSE structure; and OTCA-Head-1, OTCA-Head-2, and OTCA-Head-3 belong to the same OTCA-Head structure.

[0130] Multiin is a preprocessing layer that receives input tensors from RGB and IR images, segments the input tensors by channel, and extracts data from different modalities. Conv is a convolutional layer used to extract local features from the input data and generate feature maps. C3k2 optimizes the information flow by segmenting the feature maps and applying different convolutional layers. A2C2f is an Area-Attention Enhanced Cross-Stage Partial with fast feature extraction module. It divides the input feature map into several regions, calculates local attention independently, and then concatenates and fuses them to reduce computational complexity and enhance the representation of small target features.

[0131] The Collaborative Adaptive Cross-Modal Network (SAC-Net) extends the traditional backbone network into a dual-branch backbone network. After upsampling in the neck network, the dual-modal features are fused. The two modalities are extracted through independent dual-branch backbone networks and neck networks. During fusion, the feature maps are merged through the operation of Spatial Collaborative Asymmetric Parallel Perception Unit (SAPSE). Finally, the feature maps are fed into the Omni-Calibrated Task-Adaptive Head (OCTA-Head).

[0132] The Collaborative Adaptive Cross-Modal Network (SAC-Net) constructed in this invention starts with a dual-branch backbone network. Visible light (RGB) and infrared thermal imaging (IR) are received and aligned via Multiin. Cascaded C2k2 and A2C2f methods are used to independently complete multi-scale feature encoding from microscopic details to macroscopic semantics within the dual branches. To address the noise pollution problem caused by heterogeneous modalities in remote sensing images, Spatial Collaborative Asymmetric Parallel Sensing Units (SAPSE) are deployed at the ends and neck of the dual-branch backbone network. The Spatial Collaborative Asymmetric Parallel Sensing Units (SAPSE) abandon the traditional rigid stitching method, utilizing their unique asymmetric parallel sensing field to independently align the dual-modal features and combining a ternary reliability spatial gating mechanism to dynamically evaluate pixel-level statistics, accurately intercepting visible light background clutter and infrared thermal noise. This approach ensures that multi-scale features maintain high-confidence semantic complementarity and high-fidelity information transmission throughout the pyramid upsampling and downsampling process.

[0133] The Spatial Collaborative Asymmetric Parallel Sensing Unit (SAPSE) is an asymmetric heterogeneous feature fusion unit designed to replace traditional linear superposition or splicing operations. SAPSE defines input as a stream of primitives and a concomitant stream, eliminating the representation gap between multi-source inputs through cross-domain manifold alignment, achieving high-fidelity complementary feature fusion. For example... Figure 2The diagram shows the data processing flow of SAPSE. Since the input of SAPSE in the Collaborative Adaptive Cross-Modal Network (SAC-Net) is a two-stream feature, the two-stream feature is defined as the primitive stream X. ptg and associated flow X prg The original feature domains, which respectively carry the core geometric structure and semantic skeleton of the target, and the filled feature domains, which contain detailed information about the environmental context, are adapted to the infrared thermal imaging branch and the visible light branch, respectively. The output of SAPSE is the fused and enhanced feature Y described by the primitive flow. The specific data processing flow of SAPSE is as follows:

[0134] (1) Asymmetric multi-level sensing field coding.

[0135] Heterogeneous receptive field information is extracted using convolution operators with different topologies in the primitive flow X. ptg In the local fine-grained computation, a depthwise dilated convolution with a core size k=3 and an expansion rate d=1 is used to focus on extracting high-frequency edge information, which can be expressed as:

[0136] ;

[0137] Among them, F ptg The edge features of the primitive flow; D-DWConv 3×3,d=1 This is a deep-diffuse convolution operation.

[0138] In the associated flow X prg In the global sparse sensing computation, a sparse dilated convolution with a kernel size k=9 and a dilation rate d=4 is used to capture a large range of contextual relationships. Its equivalent receptive field reaches 33×33, which can be expressed as:

[0139] ;

[0140] Among them, F prg For associated flow detail features; D-DWConv 9×9,d=4 This is a sparse dilated convolution operation.

[0141] It should be noted that, see Figure 1 The inputs of SAPSE-1 are connected to A2C2f-4 and A2C2f-2 respectively, where the primitive flow X ptg It is the feature map F2 output by A2C2f-4, with associated flow X. prg This is the feature map F1 output by A2C2f-2. The inputs of SAPSE-2 are connected to A2C2f-5 and A2C2f-7 respectively, where the primitive flow X... ptg It is the feature map G3 output by A2C2f-5, with associated flow X. prg This is feature map G5 output from A2C2f-7. The inputs of SAPSE-3 are connected to A2C2f-6 and A2C2f-8 respectively, where the primitive flow X...ptg It is the feature map G7 output by A2C2f-6, with associated flow X. prg It is the feature map G8 output by A2C2f-8.

[0142] (2) Cross-modal collaborative calibration interaction.

[0143] To eliminate spatial offset between sensors and enhance modal complementarity, this scheme introduces a calibration operator. By calculating the pixel-wise correlation of the two feature streams, a spatial alignment weight M is generated. cal , represented as:

[0144] ;

[0145] in, For sigmoid activation function; Conv 1×1 This is a convolution operation; This is a pixel-by-pixel multiplication.

[0146] Using spatial alignment weight M cal Edge features of the elementary flow F ptg and associated flow details F prg Preliminary feature enhancement is performed, represented as:

[0147] ;

[0148] ;

[0149] in, The edge features of the primitive flow after initial enhancement; This is a preliminary enhancement of the associated flow details.

[0150] (3) Dimensionality reduction mapping of manifold space.

[0151] To reduce computational cost while removing feature redundancy, convolutional layers are used to enhance the edge features of the primitive flow. and associated flow details Mapped to a low-dimensional manifold space, it can be represented as:

[0152] ;

[0153] ;

[0154] in, The primitive flow edge features of a low-dimensional manifold space; For the associated flow detail features of low-dimensional manifold space; Let be the set of real numbers; C, H, and W represent the number of channels, height, and width of the feature map, respectively.

[0155] (4) Ternary reliability space gating.

[0156] Extract ternary descriptors from the low-dimensional manifold space and dynamically evaluate the reliability of information at each pixel location, where channel concatenation is represented as:

[0157] ;

[0158] Among them, F concat for and Characteristics of cascaded channels.

[0159] Ternary descriptors extracted based on features derived from channel cascading:

[0160] ;

[0161] ;

[0162] ;

[0163] Among them, S avg The mean descriptor represents the average significance; S max The maximum value descriptor represents the local strongest feature; S var The variance descriptor characterizes modal reliability; Mean channel This indicates a mean pooling operation along the channel dimension; Max channel Represents the channel-level max pooling operation; Variation channel This indicates the calculation of variance in the channel dimension.

[0164] Subsequently, a collaborative weighting graph is generated. The ternary descriptors are concatenated and then convolved using a 7×7 scale to calculate the collaborative weight graph containing the ternary descriptors. , represented as:

[0165] ;

[0166] in, For Sigmoid activation function; Conv 7×7 This is a convolution operation; The weights of the primitive flow space; For associated flow spatial weights; is the global confidence gating coefficient.

[0167] The mean descriptor S avg Maximum value descriptor S max Variance descriptor S varBy concatenating along the channel dimension, a three-channel composite tensor with dimensions (3, H, W) is formed. This three-channel composite tensor is then input into a convolutional layer with 3 output channels and a scale of 7×7 to aggregate and map local spatial features. After the convolution operation, the output tensor is constrained to the (0, 1) interval by the Sigmoid activation function, resulting in a collaborative weight map with a dimension domain of (3, H, W). Ultimately, the channel dimension is used to... Perform a slice separation operation, and Unpacking yields three independent single-channel spatial weight matrices, representing the spatial weights of the primitive flow. Spatial weights of associated flows Global confidence gating coefficient .

[0168] (5) Dynamic injection fusion and output.

[0169] Finally, the collaborative weight graph is used. Perform adaptive feature fusion and inject the fusion result back into the primitive flow X. ptg Adaptive fusion is represented as:

[0170] ;

[0171] Among them, F fused This is a feature of fusion.

[0172] Then confidence modulation is performed, and the original number of channels C is restored through a convolutional layer, represented as:

[0173] ;

[0174] Among them, F att To enhance attention to the diagram.

[0175] Finally, a residual multiplication mechanism is used to enhance the attention graph F. att Injection back to the primitive flow X ptg The goal perception enhancement is achieved, represented as:

[0176] ;

[0177] Where Y represents the fusion enhancement feature output by SAPSE.

[0178] The Global Calibration Task Adaptive Detection Head (OCTA-Head) is a detection head specifically designed for the Collaborative Adaptive Cross-Modal Network (SAC-Net) constructed in this invention. Through cross-scale weight calibration, dual-path feature aggregation, and a task-adaptive channel attention mechanism with a temperature coefficient, it improves the localization accuracy of SAC-Net in multi-scale and complex backgrounds. This mechanism mainly consists of four processes: a global scale perception calibration stage, task-adaptive processing of the regression branch, lightweight deep decoupling layer prediction generation, and bounding box decoding. Figure 3 As shown, the overall calculation process is as follows:

[0179] (1) Global scale perception calibration stage.

[0180] OCTA-Head receives three scale feature maps P∈{P3,P4,P5} from the neck network. First, scale weight normalization is performed. The globally learnable parameter vector for each layer of OCTA-Head is preset to W={w3,w4,w5}, i=3,4,5; w i Let X be the learnable parameters of the i-th layer, which are iteratively updated during training based on the gradient backpropagation mechanism. i ∈{P3,P4,P5}, calculate the learnable normalized weights a for each level. i , represented as:

[0181] ;

[0182] Through learnable normalized weights a i The calibrated feature map is as follows:

[0183] ;

[0184] in, To calibrate the feature map.

[0185] (2) Adaptive processing of the regression branch.

[0186] Calibration feature map Entering the regression branch, we first perform topological grouping convolution to extract basic features. The number of groups, g=32, reduces computation and enhances feature diversity, as shown below:

[0187] ;

[0188] Among them, Y base Basic features; BN is the normalization operation; GroupConv 3×3 For topological grouping convolution operations; Conv 1×1 This is a convolution operation.

[0189] Subsequently, task-adaptive channel calculation is performed, introducing dual-channel pooling aggregation and a temperature coefficient calibration mechanism. Dual-channel pooling aggregation simultaneously extracts global contextual information and local salient features, represented as:

[0190] ;

[0191] Where Z represents the global feature; To preset weights, Gap is calculated using global average pooling; Gmp is calculated using global max pooling.

[0192] After channel extraction, the process proceeds to nonlinear mapping and temperature coefficient calibration, utilizing a learnable temperature coefficient. The saturation region of the activation function is dynamically adjusted, and the regression task bias coefficient is added. This allows the OCTA-Head to learn a channel distribution that is more sensitive to regression tasks. The activation features are first calculated through a fully connected layer (FC) (including a dimensionality reduction layer FC1 and an dimensionality increase layer FC2), and then remapped back to the original number of channels C, resulting in a normalized preliminary channel attention score vector S, expressed as:

[0193] ;

[0194] Where ReLU is the ReLU activation function.

[0195] After temperature coefficient The calibration yields the final channel attention vector E, expressed as:

[0196] ;

[0197] in, The activation function is Sigmoid, and this operation uses a dynamically updated temperature coefficient. Activation function calculation can be adjusted flexibly.

[0198] Finally, the basic feature Y... base After recalibration, the output, through residual and linear correction, is expressed as:

[0199] ;

[0200] ;

[0201] Among them, Y reg_i express The corresponding regression feature results; This indicates the features after recalibration and weighting; `` indicates element-wise multiplication; `shortcut` indicates a residual join operation. Because... The calibration feature maps corresponding to the three feature maps P3, P4, and P5 at different scales, i.e., i=3,4,5, will be output as three different scale regression feature results Y after the regression branching process described above. reg_i These are used to predict bounding boxes for targets of different sizes.

[0202] (3) Lightweight deep decoupling layer.

[0203] After processing through different branches, OCTA-Head generates regression map B for localization. i and classification map C used for identification i After studying Y reg_i Perform continuous convolution to generate regression map B i For each grid point, channel data for distributed focus loss (DFL) prediction is generated. The maximum number of discrete distance grids preset by the task is defined as reg_max, expressed as:

[0204] ;

[0205] Among them, B i For Y-based reg_i The generated regression plot; B represents the batch size; H i B i Height; W i B i The width.

[0206] In parallel, Import a lightweight classification branch consisting of depthwise separable convolutions, and output the target classification probability map C. i The number of target categories is defined as nc, and is expressed as:

[0207] ;

[0208] Among them, C i Based on The generated target classification probability map; H i C i Height; W i C i The width.

[0209] Ultimately, the regression will be shown in Figure B along the channel dimension. i With target classification probability map C i By splicing, a complete prediction tensor O is formed. i For subsequent decoding, it is represented as:

[0210] ;

[0211] Among them, O iis the output prediction tensor; Concat is the concatenation operation.

[0212] It should be noted that when i=3, O3 is the predicted tensor output by OCTA-Head-1; when i=4, O4 is the predicted tensor output by OCTA-Head-2; and when i=5, O5 is the predicted tensor output by OCTA-Head-3. The three OCTA-Heads independently decouple the classification and regression tasks, and the entire head network ultimately outputs three sets of predictions for different scales (different H values). i W i The bounding box prediction results and category prediction results of the target.

[0213] (4) Bounding box decoding.

[0214] Regression plot B i Instead of directly outputting pixel coordinates, it predicts the probability distribution of the distance between the bounding box and the center point. Let d be the distance from any boundary of the predicted bounding box to the center point. The OCTA-Head outputs a vector segment of length reg_max. Its decoding and reconstruction process relies on the expectation formula of the discrete distribution, expressed as:

[0215] ;

[0216] Where d represents the distance from any boundary of the predicted bounding box to the center point; reg_max-1 represents the upper limit of the integral summation (reg_max-1=15 in this embodiment), j=0,1,2,...,reg_max-1; softmax(B i,j ) represents the probability that the boundary of the predicted bounding box falls exactly at position j.

[0217] Based on this, the distances [d] from the left, right, top, and bottom boundaries of the predicted bounding box to the center point can be calculated respectively. left ,d right ,d top ,d bottom Then, coordinate transformation and true scale mapping are performed. Let the current regression graph B be... i The center coordinates of the grid points on the grid are (x anchor ,y anchor The stride of this feature map layer relative to the original input image is s. i These correspond to downsampling factors of 8, 16, and 32 for levels P3, P4, and P5, respectively. The distance from the predicted bounding box to the center point is converted into the absolute center point and width / height on the current feature map, expressed as:

[0218] ;

[0219] ;

[0220] ;

[0221] ;

[0222] Among them, (x center ,y center ) represents the coordinates of the absolute center point, w box h box This represents the absolute width and absolute height. Finally, multiply by the step size s. i Mapping back to the original scale, we obtain the final predicted bounding box coordinates [X,Y,W,H], represented as:

[0223] ;

[0224] The multi-scale feature streams (P3, P4, and P5) after deep fusion through the neck network ultimately converge into the Global Calibration Task Adaptive Detection Head (OCTA-Head). Addressing the hierarchical information discontinuity caused by extreme scale variations in remote sensing perspectives, the OCTA-Head first performs cross-level global scale normalization calibration. It balances the semantic weight distribution between feature layers through learnable weighted pooling, effectively preventing the suppression of small target features in deep networks. Subsequently, in the parallel decoding stage of classification and regression tasks, the OCTA-Head introduces a dynamic channel attention mechanism with temperature coefficient modulation. This mechanism adaptively adjusts the activation threshold based on the differences in feature sensitivity for different tasks. This fundamentally resolves gradient conflicts in backpropagation of converging features from both streams, achieving high-precision localization and stable identification of full-scale targets in complex backgrounds.

[0225] Example 2:

[0226] To systematically evaluate the performance advantages of the Collaborative Adaptive Cross-Modal Network (SAC-Net) proposed in Example 1, this example conducts comparative experiments on the integrated RGBTDronePerson dataset based on the optimization merging criterion. Qualitative performance analysis is performed on the enhanced SAC-Net. The experiments focus on examining multiple evaluation metrics, including precision (P), recall (R), and recognition accuracy (mAP) at different IoU thresholds. Due to the subtle target features and extremely small overall proportion in the dataset, detailed experimental results are summarized in Table 1 to further verify the performance of SAC-Net at different input scales.

[0227] Table 1 Ablation Experiment Results

[0228]

[0229] Experimental results show that the proposed SAC-Net significantly outperforms the traditional model (YOLO12) in terms of precision (P), recall (R), and accuracy (mAP) at different IoU thresholds, improving by 4%, 1.7%, 3.7%, and 1.8%, respectively. This verifies that SAC-Net has excellent multimodal image extraction and fusion detection capabilities.

[0230] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-source remote sensing target detection method based on asymmetric manifold collaboration and global calibration, characterized in that, Includes the following steps: Acquire remote sensing target images; input the remote sensing target images into a collaborative adaptive cross-modal network based on the improved YOLO12 target detection network, and output the predicted bounding box coordinates of the remote sensing targets; The collaborative adaptive cross-modal network includes a dual-branch backbone network, a neck network, and a head network; wherein, the dual-branch backbone network includes a visible light branch, an infrared thermal imaging branch, and a spatial collaborative asymmetric parallel sensing unit; the neck network adds a spatial collaborative asymmetric parallel sensing unit; and the detection head in the head network is a global calibration task adaptive detection head.

2. The multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration according to claim 1, characterized in that, The spatial collaborative asymmetric parallel sensing unit is represented by SAPSE; The visible light branch includes Multiin-1, Conv-1, Conv-2, C3k2-1, Conv-3, C3k2-2, Conv-4, A2C2f-1, Conv-5, and A2C2f-2 connected in sequence; The infrared thermal imaging branch includes Multiin-2, Conv-6, Conv-7, C3k2-3, Conv-8, C3k2-4, Conv-9, A2C2f-3, Conv-10, and A2C2f-4 connected in sequence; A2C2f-2 and A2C2f-4 are connected to SAPSE-1 respectively.

3. The multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration according to claim 2, characterized in that, The neck network includes A2C2f-5, A2C2f-6, A2C2f-7, A2C2f-8, SAPSE-2, SAPSE-3, Conv-11, A2C2f-9, Conv-12, and C3k2-5; Feature map F1 output from A2C2f-2 is upsampled and fused with feature map F7 output from A2C2f-1 to obtain feature map G1; feature map F2 output from A2C2f-4 is upsampled and fused with feature map F3 output from A2C2f-3 to obtain feature map G2; feature map G2 is input into A2C2f-5 to obtain feature map G3, feature map G3 is upsampled and fused with feature map F5 output from C3k2-5 to obtain feature map G4; feature map G1 is input into A2C2f-7 to obtain feature map G5, feature map G5 is upsampled and fused with feature map F4 output from C3k2-2 to obtain feature map G6; feature map G4 is input into A2C2f-5. 2f-6 yields feature map G7, and feature map G6 is input into A2C2f-8 to obtain feature map G8; feature maps G7 and G8 are input together into SAPSE-3 to obtain feature map P3, and feature maps G3 and G5 are input together into SAPSE-2 to obtain feature map G10; feature map P3 is fused with feature map G10 after passing through Conv-11 to obtain feature map G11; feature map G11 is input into A2C2f-9 to obtain feature map P4; feature map P4 is fused with feature map F6 output from SAPSE-1 after passing through Conv-12 to obtain feature map G13; feature map G13 is input into C3k2-5 to obtain feature map P5.

4. The multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration according to claim 3, characterized in that, OTCA-Head represents the adaptive detection head for the global calibration task; The head network includes OTCA-Head-1, OTCA-Head-2, and OTCA-Head-3; The feature map output by SAPSE-3 is input to OTCA-Head-1 at P3; the feature map output by A2C2f-9 is input to OTCA-Head-2 at P4; and the feature map output by C3k2-5 is input to OTCA-Head-3 at P5.

5. The multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration according to claim 3, characterized in that, The data processing flow of the spatial collaborative asymmetric parallel sensing unit is as follows: (1) Asymmetric multi-level sensing field coding; For the elementary flow X ptg Extracting edge features: ； Among them, F ptg The edge features of the primitive flow; D-DWConv 3×3,d=1 This is a depthwise dilated convolution operation; For associated flow X prg Extract detailed features: ； Among them, F prg For associated flow detail features; D-DWConv 9×9,d=4 This is a sparse dilated convolution operation; (2) Cross-modal collaborative calibration interaction; Generate spatial alignment weights M cal : ； in, For sigmoid activation function; Conv 1×1 This is a convolution operation; This is pixel-by-pixel multiplication; Using spatial alignment weight M cal Edge features of the elementary flow F ptg and associated flow details F prg Perform initial feature enhancement: ； ； in, The edge features of the primitive flow after initial enhancement; For the preliminary enhanced details of the associated flow; (3) Dimensionality reduction mapping of manifold space; The edge features of the pre-enhanced primitive flow are processed through convolutional layers. and associated flow details Mapping to a low-dimensional manifold space: ； ； in, The primitive flow edge features of a low-dimensional manifold space; For the associated flow detail features of low-dimensional manifold space; Let be the set of real numbers; C, H, and W represent the number of channels, height, and width of the feature map, respectively. (4) Ternary reliability space gating; Extracting ternary descriptors from low-dimensional manifold spaces: ； ； ； ； Among them, F concat for and Characteristics of channel cascaded; S avg S is the mean descriptor; max S is the maximum value descriptor; var Mean is a variance descriptor. channel This indicates a mean pooling operation along the channel dimension; Max channel Represents the channel-level max pooling operation; Variation channel This indicates the calculation of variance in the channel dimension; Generate collaborative weight graph : ； Among them, Conv 7×7 This is a convolution operation; The collaborative weight graph is sliced ​​and converted into... ,in The weights of the primitive flow space; For associated flow spatial weights; These are the global confidence gating coefficients; (5) Dynamic injection fusion and output; Using collaborative weight graph Perform adaptive fusion: ； Among them, F fused Features of fusion; Then confidence modulation is performed, and the original number of channels C is restored through a convolutional layer: ； Among them, F att To enhance attention; Enhance attention map F att Injection back to the primitive flow X ptg : ； Where Y represents the fusion enhancement feature output by the spatially collaborative asymmetric parallel sensing unit.

6. The multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration according to claim 5, characterized in that, The inputs of SAPSE-1 are connected to A2C2f-4 and A2C2f-2 respectively, where the primitive flow X ptg It is the feature map F2 output by A2C2f-4, with associated flow X. prg It is the feature map F1 output by A2C2f-2; The inputs of SAPSE-2 are connected to A2C2f-5 and A2C2f-7 respectively, where the primitive flow X ptg It is the feature map G3 output by A2C2f-5, with associated flow X. prg It is feature map G5 output by A2C2f-7; The inputs of SAPSE-3 are connected to A2C2f-6 and A2C2f-8 respectively, where the primitive flow X ptg It is the feature map G7 output by A2C2f-6, with associated flow X. prg It is the feature map G8 output by A2C2f-8.

7. The multi-source remote sensing target detection method with asymmetric manifold collaboration and global calibration according to claim 4, characterized in that, The data processing flow of the adaptive detection head for the global calibration task is as follows: (1) Global scale perception calibration stage; Receive feature maps X from the neck network at three scales. i ∈{P3,P4,P5}, calculate the learnable normalized weights a for each level. i : ； Where W = {w3, w4, w5}, i = 3, 4, 5, W is a learnable parameter vector, and w i These are the learnable parameters of the i-th layer; Through learnable normalized weights a i The calibrated feature map is as follows: ； in, For calibrating feature maps; (2) Adaptive processing of the regression branch; Based on calibration feature map Calculate the fundamental characteristics: ； Among them, Y base Basic features; BN is the normalization operation; GroupConv 3×3 For topological grouping convolution operations; Conv 1×1 This is a convolution operation; Dual-channel pooling aggregation extracts global features: ； Where Z represents the global feature; Preset weights; Gap is for global average pooling calculation; Gmp is for global max pooling calculation; By activating features through a fully connected (FC) layer, and remapping them back to the original number of channels C, a normalized preliminary channel attention score vector S is obtained: ； Where ReLU is the ReLU activation function; FC1 is the dimensionality reduction layer; and FC2 is the dimensionality increase layer. After temperature coefficient The calibration yields the final channel attention vector E: ； in, Use the Sigmoid activation function; The bias coefficient for the regression task; For the basic feature Y base After recalibration, the output is obtained through residual and linear correction: ； ； Among them, Y reg_i express The corresponding regression feature results; This indicates the features after recalibration and weighting; "shortcut" indicates the residual join operation. (3) Lightweight deep decoupling layer; Generate regression plot: ； Among them, B i For Y-based reg_i The generated regression plot; B represents the batch size; reg_max is the maximum number of discrete distance grids; H i B i Height; W i B i The width; In parallel, generate target classification probability maps: ； Among them, C i Based on The generated target classification probability map; H i C i Height; W i C i The width; Output prediction tensor: ； Among them, O i This is the output prediction tensor; Concat is the concatenation operation. (4) Bounding box decoding; Calculate the distance from any boundary of the predicted bounding box to the center point: ； Where d represents the distance from any boundary of the predicted bounding box to the center point; reg_max-1 represents the upper limit of the integral summation, j=0,1,2,...,reg_max-1; softmax(B i,j ) represents the probability that the boundary of the predicted bounding box falls exactly at position j.

Images