Infrared and visible dual-mode target detection method based on modal guidance

Abstract

The application discloses an infrared and visible light dual-mode target detection method based on modal guidance, which comprises the following steps: acquiring paired infrared images and visible light images from a video stream or a file; inputting the acquired infrared images and visible light images into an improved feature extraction network based on YOLOV11 respectively; obtaining infrared feature maps and visible light feature maps through the improved feature extraction network based on YOLOV11 respectively; inputting the infrared feature maps and the visible light feature maps into a target perception modal fusion module, performing cross-modal feature fusion through difference perception guided fusion and a global context perception attention mechanism, and obtaining fusion feature maps; and inputting the fusion feature maps into a detection head network to predict the bounding box and the class confidence of the target; the method has the characteristics of overcoming the perception limitation of a single mode and improving the target detection precision and robustness in complex scenes such as insufficient light and bad weather.

Description

Technical Field

[0001] This invention belongs to the field of multimodal target detection in computer vision, and particularly relates to a method for infrared and visible light dual-modal target detection based on modality guidance. Background Technology

[0002] Object detection is a fundamental task in computer vision, with significant value in applications such as autonomous driving, video surveillance, and remote sensing analysis. In recent years, single-modal object detection technologies have made remarkable progress, with both the R-CNN series of two-stage methods and the YOLO series of single-stage methods widely applied in various scenarios. However, visible light imaging is limited by its specific wavelength range. In low-light conditions such as nighttime, rain, fog, or adverse weather, the image signal-to-noise ratio drops sharply, leading to a significant increase in the false negative and false positive rates of detection models. To address this challenge, infrared imaging technology has been widely introduced to compensate for the limitations of visible light under adverse sensing conditions. Therefore, multimodal object detection methods (IVOD) using both visible light and infrared images have gradually become an important research direction.

[0003] However, current multimodal target detection methods using visible and infrared images still face numerous challenges. On the one hand, infrared images generally suffer from low resolution and a lack of texture detail, making it difficult for general feature extractors to extract effective texture and color information. On the other hand, existing research has a biased focus, concentrating too much on improving the subjective visual quality of fused images for the human eye, rather than optimizing the perceptual performance of detectors. This has resulted in a long-term underexploration of feature fusion mechanisms designed for detection tasks. Although existing methods have improved the perceptual performance of models to some extent, they still have significant shortcomings in cross-modal interaction. Common fusion strategies typically involve simply concatenating bimodal data into a four-channel input, or extracting features from each branch separately and then performing simple fusion. Such methods struggle to achieve deep coupling of semantic and structural information, resulting in insufficient mining and utilization of complementary features across modalities.

[0004] Therefore, there is an urgent need to provide a difference-guided method for fusing infrared and visible light features to solve the above problems. Summary of the Invention

[0005] To achieve the above objectives, the inventors provide a modality-guided dual-modal target detection method based on infrared and visible light, comprising the following steps: S1, acquire paired infrared and visible light images from a video stream or file; S2, input the acquired infrared image and visible light image into the feature extraction network based on YOLOV11; S3, infrared feature maps and visible light feature maps are obtained respectively through a feature extraction network based on YOLOV11; S4, the infrared feature map and the visible light feature map are input into the target perception modal fusion module, and cross-modal feature fusion is performed through difference perception guided fusion and global context perception attention mechanism to obtain fused feature map; S5, input the fused feature map into the detection head network to predict the target's bounding box and class confidence.

[0006] In a preferred embodiment of the present invention, the feature extraction network employs a learnable weighted convolutional module. The weighted convolutional layer modulates the standard convolutional kernel element-wise by introducing a learnable spatial weighting template. The modulated weighted convolutional kernel... The expression is: ; Where ⊙ represents element-wise multiplication, E is the standard convolution kernel, and Φ is the learnable spatial weighted template; The initial value of the learnable spatial weighted template Φ is set as a fixed template with center-enhancing properties, and its initial value is obtained by outer product operation on the vector α=[δ,…,1.0,…,δ]. Generate, where δ∈(0,1), and T denotes the transpose of the matrix; Based on this initialization, the entire feature extraction network is fed into an end-to-end training process to optimize the parameters.

[0007] In a preferred embodiment of the present invention, during the training process, the learnable spatial weighting template Φ is updated through a backpropagation algorithm, thereby learning the optimal spatial weighting pattern from the data, expressed as: ; in, Represents the space-weighted template matrix. Indicates will Set as trainable parameters. Generated from a two-dimensional weighted template matrix with fixed weights, used as the initial training values, the final expression for the convolution operation is: ; in, This represents the value of the output feature map Y at the b-th sample, the c-th output channel, and the (i, j)-th spatial location. Let represent the weight parameters of the convolution kernel at the c-th output channel, the m-th input channel, and the spatial position (u,v). This represents the weight of the spatially weighted template at position (u, v). represents the number of input channels, and k represents the spatial size of the convolution kernel.

[0008] As a preferred embodiment of the present invention, the difference-aware guided fusion includes the following steps: M1, calculate the pixel-by-pixel absolute difference between the infrared feature map and the visible light feature map to obtain the modal difference map; M2 passes the modality difference map sequentially through a convolutional layer, an activation function layer, and a sigmoid function to generate a spatially adaptive difference-guided weight map; M3 uses a difference-guided weight map to perform a weighted summation of the infrared feature map and the visible light feature map to obtain a local fusion feature map; M4 adds the infrared feature map to the visible light feature map to obtain the preliminary global joint feature map; M5 performs a global average pooling operation on the preliminary global semantic representation to obtain a channel-level statistical description vector; M6 inputs the channel-level statistical description vector into the multilayer perceptron and normalizes it using the Softmax function to obtain the cross-modal channel attention weights; M7 utilizes cross-modal channel attention weights to perform channel-weighted fusion of the infrared feature map and the visible light feature map to obtain a feature fusion map of infrared and visible light.

[0009] As a preferred embodiment of the present invention, the difference-aware guided fusion includes the following steps: M1, on the local fusion path, the target-aware modal fusion module calculates the pixel-by-pixel absolute difference map D between the infrared modal feature map and the visible light modal feature map, expressed as: ; in, and represent the infrared feature map and the visible light feature map, respectively; M2 takes the pixel-wise absolute difference map D as input to the lightweight weight generation submodule to generate a spatially adaptive difference-guided weight map F, expressed as: ; Where F represents the difference-guided weighting graph. and This represents the convolution operation. This represents the activation function. Represents the Sigmoid function; M3 uses a difference-guided weight map to perform pixel-level weighting on the infrared and visible light feature maps, resulting in a locally fused feature map, expressed as: ; in, ⊙ represents a local fusion feature map, and ⊙ represents element-wise multiplication.

[0010] As a preferred embodiment of the present invention, the weight generation submodule is characterized by comprising the following layers in sequence: a 3×3 convolutional layer with C input and output channels, a ReLU activation function, a 1×1 convolutional layer with C input channels and 1 output channel, and a Sigmoid activation function σ.

[0011] As a preferred embodiment of the present invention, the difference-aware guided fusion further includes the following steps: M4, along the global fusion path, adds the infrared and visible light feature maps channel by channel to obtain the preliminary global joint feature, expressed as: ; in, This indicates the initial global joint features. and These represent infrared feature maps and visible light feature maps, respectively. M5, for preliminary global joint features Perform a global average pooling operation to obtain a channel-level statistical description vector, expressed as: ; in, This represents a channel-level statistical description vector. This indicates a global average pooling operation; M6 takes the channel-level statistical description vector as input to a lightweight MLP network consisting of two 1×1 convolutional layers. The first layer compresses the number of channels from C to C / r, where the compression ratio r=8, and performs a non-linear transformation through ReLU activation. The second layer restores the number of channels to C×h, where h is the number of modalities, generates a fusion weight tensor, and obtains balanced cross-modal channel attention weights through Softmax normalization, expressed as: ; Where A represents the cross-modal channel attention weight, This represents a multilayer perceptron. Represents the normalization function; M7 utilizes cross-modal attention weights to weightedly combine infrared and visible light features to obtain globally attention-enhanced features, expressed as follows: ; in, This indicates a global attention enhancement feature. This represents the cross-modal channel attention weights corresponding to the infrared features. This represents the cross-modal channel attention weights corresponding to visible light features; The target perception modal fusion module superimposes local fusion features with global attention enhancement features to obtain a feature fusion map of infrared and visible light, expressed as: ; in, This represents a feature fusion diagram of infrared and visible light. Represents the local fusion feature map. This indicates a global attention enhancement feature.

[0012] Unlike existing technologies, the above-mentioned technical solution achieves the following beneficial effects: This method utilizes the difference information between modalities to construct guiding signals, realizing adaptive fusion of multimodal feature maps, thereby promoting full interaction and complementarity between features; and by introducing a difference-guided fusion mechanism and a learnable weighted convolution module, it achieves adaptive and complementary fusion of infrared and visible light features; at the same time, this method improves the existing weighted convolution module by combining the convolution kernel with a trainable prior template, enhancing the dynamic modeling capability and feature representation capability of convolution operations; this method can overcome the perception limitations of a single modality and improve the target detection accuracy and robustness in complex scenarios such as insufficient lighting and severe weather. Attached Figure Description

[0013] Figure 1 This is a framework diagram of the method described in a specific implementation. Figure 2 This is a structural diagram of the weighted convolutional module (wConv) in the infrared and visible light dual-modal target detection network described in the specific implementation embodiment; Figure 3 This is a structural diagram of the target perception modality fusion module TAMFusion in the infrared and visible light dual-modal target detection network described in the specific implementation embodiment; Figure 4 This is a schematic diagram illustrating the input, feature fusion, and detection results of the method described in a specific implementation. Detailed Implementation

[0014] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0015] Combination Figure 1This modality-guided dual-modal target detection method for infrared and visible light is constructed based on the YOLOv11n model. The input consists of spatiotemporally registered pairs of infrared and visible light images. The workflow begins with two separately weighted backbone feature extraction networks to extract features from the two input images. At this stage, this embodiment uses a learnable weighted convolutional module (lwConv) instead of a standard convolutional module to enhance the dynamic perception of key local features. Subsequently, the extracted feature maps from the infrared and visible light images are fed into a target perception modality fusion module (TAMFusion) for deep complementarity and fusion. Finally, the enhanced fused features are delivered to the detection head to complete target localization and classification.

[0016] Implementation of Learnable Weighted Convolutional Module (lwConv): Learnable weighted convolutional module (lwConv) aims to solve the problem that traditional convolutional kernels are spatially uniformly distributed and cannot dynamically perceive the importance of local structures.

[0017] like Figure 2 As shown, the traditional convolutional module (Conv) has a single, fixed convolutional kernel at its core. In contrast, the learnable weighted convolutional module (lwConv) proposed in this embodiment introduces a learnable spatial weighted template Φ on top of the traditional convolutional kernel. This module, by introducing learnable spatial weights, endows the convolutional operation with adaptive feature selection capabilities. Let the input feature map be... The convolution kernel is The output expression of a standard two-dimensional convolution is: ; in, This represents the value of the output feature map Y at the b-th sample, the c-th output channel, and the (i, j)-th spatial location. Let X represent the value of the input feature map X at the b-th sample, the m-th input channel, and the (i+u, j+v)-th spatial position. Let B represent the weight parameters of the convolution kernel at the c-th output channel, the m-th input channel, and the spatial location (u,v), and let B represent the batch size. Indicates the number of input channels. The output channel number is represented by H and W, which represent the spatial height and width of the input feature map, respectively, and k represents the spatial size of the convolution kernel.

[0018] In this calculation, the spatial locations within the convolution kernel The weights contribute equally to the output, failing to reflect the differences in structural importance at different positions and lacking a dynamic perception mechanism for local features.

[0019] To enhance the spatial structure modeling capability of convolutional kernels, this embodiment introduces a trainable spatial weighted template. First, a fixed template with center enhancement properties is generated as follows: For a convolution kernel of size k×k, a center enhancement weight vector α=[δ,…,1.0,…,δ] is defined. The length of this vector is k, its central element is 1.0, and the rest of the elements are δ.

[0020] In this embodiment, for the case k=3, α=[δ,1.0,δ], where δ takes the value of 0.75. Subsequently, the center-enhanced weight vector α and its transpose are used... Multiplying the matrices yields the initialized space-weighted template. Its expression is: ; in, This represents the constructed two-dimensional space weighted template matrix. This represents the transpose of the center-enhancing weight vector α.

[0021] Taking k=3 and δ=0.75 as an example, the calculated Φ is a 3x3 matrix with the highest weight at the center (1.0) and decreasing towards the surrounding areas.

[0022] To further enhance its dynamism and adaptability, this embodiment improves it into trainable parameters: The spatial weighting template Φ is a trainable parameter matrix of dimension k×k. During network training initialization, its value is assigned by Φ calculated above. During training, Φ is updated using the backpropagation algorithm to learn the optimal spatial weighting pattern from the data, expressed as: ; in, Represents the space-weighted template matrix. Indicates will Set as trainable parameters. Generated from a two-dimensional spatially weighted template matrix with fixed weights, and used as the initial training values, this allows the model to learn the optimal spatial weighting pattern from the data. The final expression for the convolution operation is: ; in, This represents the value of the output feature map Y at the b-th sample, the c-th output channel, and the (i, j)-th spatial location. Let X represent the value of the input feature map X at the b-th sample, the m-th input channel, and the (i+u, j+v)-th spatial position. Let represent the weight parameters of the convolution kernel at the c-th output channel, the m-th input channel, and the spatial position (u,v). This represents the weight of the spatially weighted template at position (u, v). The output channel number is represented by H and W, which represent the spatial height and width of the input feature map, respectively, and k represents the spatial size of the convolution kernel.

[0023] like Figure 2 As shown, the output of the wConv module is the result of calculating the input feature map using a convolutional kernel modulated with spatial weights Φ. This improved wConv module, while retaining the center-enhanced prior, introduces trainability, achieving a combination of local feature enhancement and data-driven spatial weighting. This provides a more robust convolutional feature representation for high-quality feature extraction from infrared and visible multimodal images.

[0024] To verify the effectiveness of the lwConv module, it was tested on a multi-scenario, multimodal dataset M. 3 Ablation experiments were conducted on FD. The experimental results are shown in Table 1. After introducing the weighted convolutional module wConv into the baseline network, the average accuracy (mAP) of target detection was improved. 50 The accuracy improved from 79.8% to 81.4%, an increase of 1.6 percentage points; the overall average accuracy (mAP) increased. 50-95 The accuracy improved from 52.3% to 53.8%, an increase of 1.5 percentage points. This data demonstrates that the lwConv module significantly improves network performance by enhancing its ability to model local detailed features.

[0025] Table 1: The improvement modules of this method in M 3 Comparison table of ablation experiments on the FD dataset: 79.8 52.3 √ 81.4 53.8 √ 84.3 56.1 √ √ 85.5 57.8 Table 1 sets four sets of baselines. A checkmark (√) indicates that the corresponding improved component is enabled, while a blank space indicates that the module is not used. The first set has no checkmarks and represents the original baseline network without lwConv and TAMFusion. When only the lwConv column is checked, only the backbone convolution module is replaced, and the feature fusion method remains unchanged. When only the TAMFusion column is checked, the backbone convolution maintains its original structure, and only the feature fusion strategy is upgraded. Checking both columns simultaneously indicates that the lwConv backbone and TAMFusion fusion modules are integrated at the same time.

[0026] lwConv indicates that the original convolutional module Conv is replaced with a learnable weighted convolutional module lwConv in the backbone network. TAMFusion indicates that pure feature concatenation is replaced with the object-aware modality fusion module TAMFusion.

[0027] The implementation of the target-aware modal fusion module (TAMFusion) is used to solve the problems of information interference and semantic inconsistency caused by the direct fusion of infrared and visible light features due to differences in semantics and detail distribution.

[0028] like Figure 3 As shown, this module receives features from both the infrared and visible light branches. ,in , and Do not represent the red feature map and the visible light feature map. B represents the dimension of the feature map, C represents the number of channels, and H and W represent the spatial height and width of the feature map, respectively. The effective integration of features is achieved through a parallel local and global fusion mechanism.

[0029] First, along the local fusion path, this module calculates the pixel-wise absolute difference map D of the two modal features, expressed as: ; Where D represents the pixel-by-pixel absolute difference map between infrared features and visible light features.

[0030] The difference map D effectively reflects regions with high information content in infrared and visible light. Difference map D is fed into a lightweight weight generation submodule, which consists of the following layers: a 3×3 convolutional layer with C input and output channels, a ReLU activation function, a 1×1 convolutional layer with C input channels and 1 output channel, and a Sigmoid activation function σ. Finally, a spatially adaptive difference-guided weight map F is generated, whose expression is: ; in, ⊙ represents a local fusion feature map, and ⊙ represents element-wise multiplication.

[0031] This mechanism achieves pixel-by-pixel modal weighted fusion in the spatial dimension, which can adaptively enhance the response capability of infrared in hot target areas while preserving the texture details in visible light images, thereby giving full play to the complementarity of the two modes.

[0032] Simultaneously, in the global fusion path, to capture long-range dependencies between modalities and the global semantic structure, this module implements a joint cross-modal channel attention mechanism. First, the infrared and visible light features are added channel by channel to obtain preliminary global joint features, expressed as: ; in, Represents the global joint feature. and These represent infrared feature maps and visible light feature maps, respectively.

[0033] Next to Perform a global average pooling operation to obtain a channel-level statistical description vector, expressed as: ; in, This represents a channel-level statistical description vector. This indicates a global average pooling operation.

[0034] Next, to capture the dependencies between channels and generate cross-modal attention weights, this embodiment inputs the channel-level statistical description vector into a lightweight MLP network consisting of two 1×1 convolutional layers. The first layer compresses the number of channels from C to C / r, where the compression ratio r=8, and achieves a non-linear transformation through ReLU activation; the second layer restores the number of channels to C×h, where h is the number of modalities, and generates a fused weight tensor. Balanced cross-modal channel attention weights are then obtained through Softmax normalization, expressed as: ; Where A represents the balanced cross-modal channel attention weights, This represents a multilayer perceptron. This represents the normalization function.

[0035] Obtaining cross-modal channel attention weights Subsequently, in order to feed back the channel-level weights to the original feature space and achieve adaptive fusion between modalities, this embodiment further utilizes attention weights to weightedly combine infrared and visible light features, thereby obtaining globally attention-enhanced features, expressed as: ; in, This indicates a global attention enhancement feature; This represents the cross-modal channel attention weights corresponding to the infrared features; This represents the cross-modal channel attention weights corresponding to visible light features.

[0036] This mechanism ensures the semantic consistency of fused features by modeling global dependencies between channels.

[0037] Finally, to balance local saliency and global consistency, the target perception modality fusion module (TAMFusion) superimposes the local fusion features with the global attention enhancement features to obtain a feature fusion map of infrared and visible light, expressed as: ; in, This represents a feature fusion diagram of infrared and visible light. Represents the local fusion feature map; This represents attention-weighted features.

[0038] like Figure 4 As shown, (a) is the visible light input image; (b) is the infrared input image; (c), (d), and (e) are the feature fusion maps output by the shallow, middle, and high layers of this method, respectively; (f) is the target detection result on the visible light image; and (g) is the target detection result on the infrared image. Final output... It is a local path With global path The result of the addition corresponds to Figure 4 (c), (d), and (e) are examples of this fusion strategy. This strategy fully leverages the complementary advantages of the two mechanisms: the local difference weighting module highlights the target response in the infrared image and the texture details in the visible light image in the spatial dimension, while the global attention mechanism coordinates the feature distribution of the two modalities at the semantic level, ensuring consistency between global structure and semantic information. By combining the two, the TAMFusion module achieves collaborative modeling of local and global features, effectively avoiding information shift or structural inconsistency problems caused by a single fusion strategy. The final output fused features are shown in the diagram. It balances target prominence, texture fidelity, and global structural integrity, providing a more stable and discriminative feature representation for subsequent tasks.

[0039] To further evaluate its advantages in multimodal fusion tasks, this paper... 3 The results were compared with state-of-the-art fusion models and object detection models on the FD (Multi-Modal Model Few-shot Dataset) and LLVIP (A Visible-infrared Paired Dataset for Low-lightVision) datasets, and are shown in Table 2.

[0040] Table 2: Comparison of Experiments between This Method and Multimodal Target Detection Models

[0041] This experiment was conducted in M 3 The proposed model was compared with state-of-the-art cross-modal fusion algorithms on the two major public infrared-visible datasets, FD and LLVIP. The experimental results are shown in Table 2. The comparison methods include classic image fusion networks such as Swin Fusion, PIAFusion, and U2Fusion, as well as recent high-performance cross-modal detection models such as YOLOv11n-RGBT, FD2Net, and DEYOLO-n, with mAP as the benchmark. 50As a core accuracy evaluation metric, this study comprehensively verifies the detection performance and fusion advantages of the model presented in this paper. In M 3 In the FD dataset, our method achieves an optimal mAP of 85.5. 50 As a result, compared to other comparative algorithms, the accuracy improved by 0.3% to 6.3%, demonstrating a significant advantage and fully showcasing the model's excellent cross-modal feature fusion capability. In complex scene tests on the LLVIP dataset, the proposed method's model achieved high mAP. 50 Achieving a score of 96.7, it is only slightly behind the best DEYOLO-n by a mere 0.1%, significantly outperforming most advanced algorithms such as Swin Fusion, PIAFusion, and YOLOv11n-RGBT-MCF. The combined results from both datasets demonstrate that, compared to traditional single-modal detection methods and existing cross-modal fusion models, TAMYOLO-n effectively mines complementary features from infrared and visible light images, achieving accurate feature alignment and fusion, significantly improving target detection accuracy in complex scenes, and exhibiting outstanding overall performance and robustness.

[0042] It should be noted that although the above embodiments have been described herein, this does not limit the scope of patent protection of the present invention. Therefore, any changes and modifications made to the embodiments described herein based on the innovative concept of the present invention, or equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included within the scope of patent protection of the present invention.

Claims

1. A method for detecting infrared and visible light dual-modal targets based on modal guidance, characterized in that, Includes the following steps: S1, acquire paired infrared and visible light images from a video stream or file; S2, input the acquired infrared image and visible light image into the feature extraction network based on YOLOV11; S3, infrared feature maps and visible light feature maps are obtained respectively through a feature extraction network based on YOLOV11; S4, the infrared feature map and the visible light feature map are input into the target perception modal fusion module, and cross-modal feature fusion is performed through difference perception guided fusion and global context perception attention mechanism to obtain fused feature map; S5, input the fused feature map into the detection head network to predict the target's bounding box and class confidence.

2. The infrared and visible light dual-modal target detection method based on modal guidance according to claim 1, characterized in that: The feature extraction network employs learnable weighted convolutional modules. These weighted convolutional layers modulate the standard convolutional kernel element-wise by introducing a learnable spatial weighting template. The modulated weighted convolutional kernels... The expression is: ； Where ⊙ represents element-wise multiplication, E is the standard convolution kernel, and Φ is the learnable spatial weighted template; The initial value of the learnable spatial weighted template Φ is set as a fixed template with center-enhancing properties, and its initial value is obtained by outer product operation on the vector α=[δ,…,1.0,…,δ]. Generate, where δ∈(0,1), and T denotes the transpose of the matrix; Based on this initialization, the entire feature extraction network is fed into an end-to-end training process to optimize the parameters.

3. The infrared and visible light dual-modal target detection method based on modal guidance according to claim 2, characterized in that: During training, the learnable spatial weighted template Φ is updated using the backpropagation algorithm, thereby learning the optimal spatial weighting pattern from the data. The expression is: ； in, Represents the space-weighted template matrix. Indicates will Set as trainable parameters. Generated from a two-dimensional weighted template matrix with fixed weights, and used as the initial values ​​for training, the final expression for the convolution operation is: ； in, This represents the value of the output feature map Y at the b-th sample, the c-th output channel, and the (i, j)-th spatial location. Let X represent the value of the input feature map X at the b-th sample, the m-th input channel, and the (i+u, j+v)-th spatial position. Let represent the weight parameters of the convolution kernel at the c-th output channel, the m-th input channel, and the spatial position (u,v). This represents the weight of the spatially weighted template at position (u, v). represents the number of input channels, and k represents the spatial size of the convolution kernel.

4. The infrared and visible light dual-modal target detection method based on modal guidance according to claim 1, characterized in that, The difference-aware guided fusion includes the following steps: M1, calculate the pixel-by-pixel absolute difference between the infrared feature map and the visible light feature map to obtain the modal difference map; M2 passes the modality difference map sequentially through a convolutional layer, an activation function layer, and a sigmoid function to generate a spatially adaptive difference-guided weight map; M3 uses a difference-guided weight map to perform a weighted summation of the infrared feature map and the visible light feature map to obtain a local fusion feature map; M4 adds the infrared feature map to the visible light feature map to obtain the preliminary global joint feature map; M5 performs a global average pooling operation on the preliminary global semantic representation to obtain a channel-level statistical description vector; M6 inputs the channel-level statistical description vector into the multilayer perceptron and normalizes it using the Softmax function to obtain the cross-modal channel attention weights; M7 utilizes cross-modal channel attention weights to perform channel-weighted fusion of the infrared feature map and the visible light feature map to obtain a feature fusion map of infrared and visible light.

5. The infrared and visible light dual-modal target detection method based on modal guidance according to claim 1, characterized in that, The difference-aware guided fusion includes the following steps: M1, along the local fusion path, the target perception modal fusion module calculates the pixel-wise absolute difference map D between the infrared modal feature map and the visible light modal feature map, expressed as: ; in, and These represent infrared feature maps and visible light feature maps, respectively. M2 takes the pixel-wise absolute difference map D as input to the lightweight weight generation submodule to generate a spatially adaptive difference-guided weight map F, expressed as: ； Where F represents the difference-guided weighting graph. and This represents the convolution operation. This represents the activation function. Represents the Sigmoid function; M3 uses a difference-guided weight map to perform pixel-level weighting on the infrared and visible light feature maps, resulting in a locally fused feature map, expressed as: ； in, ⊙ represents a local fusion feature map, and ⊙ represents element-wise multiplication.

6. The infrared and visible dual-modal target detection method based on modal guidance according to claim 5, characterized in that, The weight generation submodule consists of the following layers in sequence: a 3×3 convolutional layer with C input and output channels, a ReLU activation function, a 1×1 convolutional layer with C input channels and 1 output channel, and a Sigmoid activation function σ.

7. The infrared and visible dual-modal target detection method based on modal guidance according to claim 5, characterized in that, The difference-aware guided fusion also includes the following steps: M4, along the global fusion path, adds the infrared and visible light feature maps channel by channel to obtain the preliminary global joint feature, expressed as: ； in, This indicates the initial global joint features. and These represent infrared feature maps and visible light feature maps, respectively. M5, for preliminary global joint features Perform a global average pooling operation to obtain a channel-level statistical description vector, expressed as: ； in, This represents a channel-level statistical description vector. This indicates a global average pooling operation; M6 takes the channel-level statistical description vector as input to a lightweight MLP network consisting of two 1×1 convolutional layers. The first layer compresses the number of channels from C to C / r, where the compression ratio r=8, and performs a non-linear transformation through ReLU activation. The second layer restores the number of channels to C×h, where h is the number of modalities, generates a fusion weight tensor, and obtains balanced cross-modal channel attention weights through Softmax normalization, expressed as: ； Where A represents the cross-modal channel attention weight, This represents a multilayer perceptron. Represents the normalization function; M7 utilizes cross-modal attention weights to weightedly combine infrared and visible light features to obtain globally attention-enhanced features, expressed as follows: ； in, This indicates a global attention enhancement feature. This represents the cross-modal channel attention weights corresponding to the infrared features. This represents the cross-modal channel attention weights corresponding to visible light features; The target-sensing modal fusion module superimposes local fusion features with global attention enhancement features to obtain a feature fusion map of infrared and visible light, expressed as: ； in, This represents a feature fusion diagram of infrared and visible light. Represents the local fusion feature map. This indicates a global attention enhancement feature.

Images