An image fusion method based on cross-modal difference and double-axis attention network

Abstract

This invention belongs to the field of image fusion technology, specifically relating to an image fusion method based on cross-modal differential attention and a dual-axis attention network. The steps include: S1, constructing a global enhancement branch and a multi-scale enhancement branch; S2, for the input infrared and visible light images, performing feature extraction and processing through the global enhancement branch and the multi-scale enhancement branch respectively to obtain global enhancement features and multi-scale enhancement features; S3, adding the global enhancement features and multi-scale enhancement features element-wise to obtain infrared and visible light image enhancement features; S4, inputting the infrared and visible light image enhancement features into a decoder to output a fused infrared and visible light image. This invention can utilize cross-modal differential attention mechanisms and dual-axis attention mechanisms to fuse infrared and visible light images, improving the clarity and accuracy of image fusion.

Description

Technical Field

[0001] This invention belongs to the field of image fusion technology, specifically relating to an image fusion method based on cross-modal difference and dual-axis attention network. Background Technology

[0002] With the rapid development of computer vision technology, fusion techniques are playing an increasingly important role in modern applications. Infrared and visible light image fusion offers significant advantages in information integration and practical applications. First, their signals originate from different modalities, providing diverse information. Second, infrared and visible light images reveal almost all the intrinsic features of an object, making this fusion highly valuable in environmental monitoring, reconnaissance, and other fields with significant potential applications. However, existing fusion models cannot effectively distinguish the unique features of these two types of images. Furthermore, some attention mechanisms also exhibit limitations when simultaneously integrating features from both spatial and channel dimensions.

[0003] To improve the fusion effect of infrared and visible light images, deep learning-based image fusion techniques have gradually attracted attention in recent years. This technique can automatically extract deep features from images through networks. Deep learning networks can also learn how to effectively extract and fuse key information from infrared and visible light images from large amounts of data. However, existing fusion models cannot effectively distinguish the unique features of these two types of images. Secondly, some attention mechanisms also show limitations when simultaneously integrating features from spatial and channel dimensions. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide an image fusion method based on cross-modal differential and dual-axis attention networks, which can use cross-modal differential attention mechanism and dual-axis attention mechanism to perform infrared and visible light image fusion, thereby improving the clarity and accuracy of image fusion.

[0005] To achieve the above objectives, this invention provides an image fusion method based on cross-modal difference and dual-axis attention network, comprising the following steps:

[0006] S1. Construct a global enhancement branch and a multi-scale enhancement branch. The global enhancement branch includes a global pooling layer and residual connection, a convolutional-biaxial attention module, a cross-modal difference attention module 1, and a Transformer module 1, which are set up in sequence. The convolutional-biaxial attention module has two layers. The multi-scale enhancement branch includes a convolutional layer, a multi-scale-biaxial attention module, a cross-modal difference attention module 2, and a Transformer module 2, which are set up in sequence.

[0007] S2. For the input infrared and visible light images, feature extraction and processing are performed through the global enhancement branch and the multi-scale enhancement branch, respectively, to obtain global enhancement features and multi-scale enhancement features;

[0008] S3. Element-wise addition of global enhancement features and multi-scale enhancement features yields infrared and visible light image enhancement features;

[0009] S4. Input the infrared and visible light image enhancement features into the decoder and output the infrared and visible light fused image.

[0010] As a preferred embodiment of the present invention, in S2, when the input infrared and visible light images pass through the global enhancement branch, infrared global features and visible light global features are first extracted using a global pooling layer and residual connection. The two global features are then processed by a two-layer convolutional-dual-axis attention module to output infrared global dual-axis attention features and visible light global dual-axis attention features. The two global dual-axis attention features are then input to a cross-modal differential attention module to output infrared global differential attention features and visible light global differential attention features. Finally, the two global differential attention features are input to a Transformer module to output infrared global enhancement features and visible light global enhancement features. The two are then added element-wise to obtain the global enhancement features.

[0011] When the input infrared and visible light images pass through the multi-scale enhancement branch, the infrared shallow features and visible light shallow features are first extracted using convolutional layers. The two shallow features are then passed through a multi-scale dual-axis attention module, which outputs infrared multi-scale dual-axis attention features and visible light multi-scale dual-axis attention features. The two multi-scale dual-axis attention features are then input to a cross-modal differential attention module, which outputs infrared multi-scale differential attention features and visible light multi-scale differential attention features. Finally, the two multi-scale differential attention features are passed through a Transformer module, which outputs infrared multi-scale enhancement features and visible light multi-scale enhancement features. The two are then added element-wise to obtain the multi-scale enhancement features.

[0012] As a preferred embodiment of the present invention, global features are extracted using a global pooling layer and residual connections, as follows:

[0013] (1);

[0014] In the formula, x takes the value of or , Represents infrared images or visible light image ; This represents a global average pooling operation; Represents global infrared features or visible light global features .

[0015] As a preferred embodiment of the present invention, the extraction process of global dual-axis attention features is as follows:

[0016] First, the global dual-axis attention intermediate features are extracted through the first-layer convolutional dual-axis attention module, represented as:

[0017] (2);

[0018] In the formula, It is the ReLU activation function; This indicates a dual-axis attention module; Indicating infrared global dual-axis attention intermediate features Or visible light global dual-axis attention intermediate features ; This represents a 3×3 convolution;

[0019] The second-layer convolutional-biaxial attention module is used to... Further processing yields the global biaxial attention features, represented as:

[0020] (3);

[0021] In the formula, Indicating infrared global dual-axis attention features or visible light global dual-axis attention features .

[0022] As a preferred embodiment of the present invention, a dual-axis attention module Specifically:

[0023] First, perform average pooling on the input features:

[0024] (4);

[0025] In the formula, and These represent the vertical global context vector and the horizontal global context vector, respectively. This represents the input to the dual-axis attention module;

[0026] right and After processing, feature maps in the vertical and horizontal directions are obtained, represented as follows:

[0027] (5);

[0028] In the formula, This represents a splicing operation; This represents a batch normalization operation; This represents the decomposition of the feature map in the horizontal and vertical directions; , These represent feature maps in the vertical and horizontal directions, respectively.

[0029] right and The two feature weight maps are processed separately and obtained as follows:

[0030] (6);

[0031] In the formula, The value is or , represent or ; This represents the Sigmoid activation function; Feature weight map representing the vertical or horizontal direction , ;

[0032] Will and , Element-wise multiplication yields the output of the dual-axis attention module. :

[0033] (7).

[0034] As a preferred embodiment of the present invention, the shallow feature extraction process is as follows:

[0035] (8);

[0036] In the formula, Represents shallow infrared features or visible light shallow layer features .

[0037] As a preferred embodiment of the present invention, the extraction method of multi-scale biaxial attention features is as follows:

[0038] (9);

[0039] In the formula, This represents a multi-scale convolution operation; Indicating infrared global dual-axis attention features or visible light global dual-axis attention features .

[0040] As a preferred embodiment of the present invention, the formulas for obtaining global differential attention features and multi-scale differential attention features are as follows:

[0041] (10);

[0042] In the formula, the value of i is... or , represent or , represent or ; Indicating infrared global differential attention features or infrared multi-scale differential attention features ; Represents the global differential attention features of visible light Or visible light multi-scale differential attention features ; This indicates either Cross-modal Differential Attention Module 1 or Cross-modal Differential Attention Module 2, both with the same architecture, specifically:

[0043] First, the input features are processed to obtain the updated features, represented as follows:

[0044] (11);

[0045] In the formula, Infrared image features are represented as input to the cross-modal differential attention module. or visible light image features , Indicates updated infrared image features or visible light image features ; Represents a 1×1 convolution;

[0046] The enhanced features obtained through two convolutional layers are represented as follows:

[0047] (12);

[0048] In the formula, Indicates the enhanced infrared image features or visible light image features ;

[0049] Extract enhanced differential features, infrared-visible light differential features Represented as:

[0050] (13);

[0051] Visible-infrared difference characteristics Represented as:

[0052] (14);

[0053] Finally, the output features of the cross-modal differential attention are obtained, represented as follows:

[0054] (15);

[0055] (16);

[0056] In the formula, For the infrared output characteristics of cross-modal differential attention, The visible light output characteristics of cross-modal differential attention.

[0057] As a preferred embodiment of the present invention , After passing through Transformer module one, the infrared global enhancement features are obtained. and visible light global enhancement features The two are added element-wise to obtain the global enhanced feature. ;

[0058] , After passing through Transformer module 2, infrared multi-scale enhanced features are obtained. and visible light multiscale enhancement features The two are added element-wise to obtain multi-scale enhanced features. .

[0059] As a preferred embodiment of the present invention, in S3... , Element-wise addition yields the infrared and visible light image enhancement features F;

[0060] In S4, F is input into the decoder, and the decoding process is as follows:

[0061] (17);

[0062] In the formula, This represents a primary fusion feature;

[0063] The process of generating the fused image is as follows:

[0064] (18);

[0065] In the formula, This refers to the Transformer module; This represents the final fused image, which is a fused image of infrared and visible light.

[0066] The beneficial effects of this invention are:

[0067] In the task of fusion of infrared and visible light images, this invention effectively combines the advantages of convolutional neural networks and attention mechanisms, improving the clarity and visual quality of the fused images. It designs a dual-branch structure: the global enhancement branch enhances the feature representation of infrared and visible light images through global contextual information and cross-modal feature fusion; the multi-scale enhancement branch focuses on utilizing the advantages of multi-scale convolution to extract features from different levels of detail.

[0068] This invention designs a dual-axis attention module to process the height and width dimensions of the feature map. This design enables the model to effectively capture the importance of spatial and channel features. It also designs a cross-modal differential attention module to capture and process the difference features between infrared and visible light images to improve the fusion result. These designs make this invention valuable in the application of infrared and visible light image fusion technology, significantly improving the clarity and accuracy of image fusion, enhancing target recognition and judgment capabilities in fields such as public safety monitoring, intelligent transportation, and business analysis, thereby improving the overall system performance and resource utilization efficiency. Attached Figure Description

[0069] Figure 1 This is a flowchart illustrating the principle of this invention;

[0070] Figure 2 This is an exemplary fusion flowchart in an embodiment of the present invention. Detailed Implementation

[0071] The embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0072] Example 1: As Figure 1 As shown, an image fusion method based on cross-modal difference and dual-axis attention network includes the following steps:

[0073] S1. Construct a global enhancement branch and a multi-scale enhancement branch. The global enhancement branch includes a global pooling layer and residual connection, a convolutional-biaxial attention module, a cross-modal difference attention module 1, and a Transformer module 1, which are set up in sequence. The convolutional-biaxial attention module has two layers. The multi-scale enhancement branch includes a convolutional layer, a multi-scale-biaxial attention module, a cross-modal difference attention module 2, and a Transformer module 2, which are set up in sequence.

[0074] S2. For the input infrared and visible light images, feature extraction and processing are performed through the global enhancement branch and the multi-scale enhancement branch, respectively, to obtain global enhancement features and multi-scale enhancement features;

[0075] S3. Element-wise addition of global enhancement features and multi-scale enhancement features yields infrared and visible light image enhancement features;

[0076] S4. Input the infrared and visible light image enhancement features into the decoder and output the infrared and visible light fused image.

[0077] like Figure 2 As shown in S2, when the input infrared and visible light images pass through the global enhancement branch, the global features of infrared and visible light are first extracted using a global pooling layer and a residual connection. The two global features are then processed by a two-layer convolutional-dual-axis attention module, which outputs infrared global dual-axis attention features and visible light global dual-axis attention features. The two global dual-axis attention features are then input to the first cross-modal differential attention module, which outputs infrared global differential attention features and visible light global differential attention features. Finally, the two global differential attention features are input to the first Transformer module, which outputs infrared global enhancement features and visible light global enhancement features. The two are then added element-wise to obtain the global enhancement features.

[0078] When the input infrared and visible light images pass through the multi-scale enhancement branch, the infrared shallow features and visible light shallow features are first extracted using convolutional layers. The two shallow features are then passed through a multi-scale dual-axis attention module, which outputs infrared multi-scale dual-axis attention features and visible light multi-scale dual-axis attention features. The two multi-scale dual-axis attention features are then input to a cross-modal differential attention module, which outputs infrared multi-scale differential attention features and visible light multi-scale differential attention features. Finally, the two multi-scale differential attention features are passed through a Transformer module, which outputs infrared multi-scale enhancement features and visible light multi-scale enhancement features. The two are then added element-wise to obtain the multi-scale enhancement features.

[0079] Global features are extracted using a global pooling layer and residual connections, as follows:

[0080] (1);

[0081] In the formula, x takes the value of or , Represents infrared images or visible light image ; This represents a global average pooling operation; Represents global infrared features or visible light global features .

[0082] Global feature extraction, by considering macroscopic information of the entire image, such as brightness, contrast, texture, and edges, helps to describe the image content as a whole.

[0083] The process of extracting global biaxial attention features is as follows:

[0084] First, the global dual-axis attention intermediate features are extracted through the first-layer convolutional dual-axis attention module, represented as:

[0085] (2);

[0086] In the formula, It is the ReLU activation function; This indicates a dual-axis attention module; Indicating infrared global dual-axis attention intermediate features Or visible light global dual-axis attention intermediate features ; This represents a 3×3 convolution (convolution operation);

[0087] The second-layer convolutional-biaxial attention module is used to... Further processing yields the global biaxial attention features, represented as:

[0088] (3);

[0089] In the formula, Indicating infrared global dual-axis attention features or visible light global dual-axis attention features .

[0090] Dual-axis attention module Specifically:

[0091] First, perform average pooling on the input features:

[0092] (4);

[0093] In the formula, and These represent the vertical global context vector and the horizontal global context vector, respectively. This represents the input to the dual-axis attention module;

[0094] right and After processing, feature maps in the vertical and horizontal directions are obtained, represented as follows:

[0095] (5);

[0096] In the formula, This represents a splicing operation; This represents a batch normalization operation; This represents the decomposition of the feature map in the horizontal and vertical directions; , These represent feature maps in the vertical and horizontal directions, respectively.

[0097] right and The two feature weight maps are processed separately and obtained as follows:

[0098] (6);

[0099] In the formula, The value is or , represent or ; This represents the Sigmoid activation function; Feature weight map representing the vertical or horizontal direction , ;

[0100] Will and , Element-wise multiplication yields the output of the dual-axis attention module. :

[0101] (7).

[0102] Global dual-axis attention features apply a convolutional-dual-axis attention mechanism to global features, enabling the fused image to focus more on channel and spatial dimensional features while extracting global features. Batch normalization operation Standardizing feature distributions stabilizes training, improves fusion quality, and enhances the model's generalization ability. The ReLU activation function, through its advantages of introducing nonlinear features, improving computational efficiency, mitigating gradient vanishing, and achieving sparse activation, significantly enhances the modeling ability and training efficiency of deep networks in image fusion tasks.

[0103] The process of extracting shallow features is as follows:

[0104] (8);

[0105] In the formula, Represents shallow infrared features or visible light shallow layer features .

[0106] Shallow features play an important role in the fusion of infrared and visible light images. Compared with global features, shallow features focus more on local information of the image, mainly reflecting low-level visual features such as edges, texture, color, brightness, and contrast.

[0107] The extraction method for multi-scale biaxial attention features is as follows:

[0108] (9);

[0109] In the formula, This represents a multi-scale convolution operation; Indicating infrared global dual-axis attention features or visible light global dual-axis attention features .

[0110] Multi-scale dual-axis attention features apply a multi-scale dual-axis attention mechanism on top of shallow features, enabling the fused image to focus more on channel and spatial dimensional features while extracting shallow features, and to integrate information from multiple scales.

[0111] The formulas for obtaining global differential attention features and multi-scale differential attention features are as follows:

[0112] (10);

[0113] In the formula, the value of i is... or , represent or , represent or ; Indicating infrared global differential attention features or infrared multi-scale differential attention features ; Represents the global differential attention features of visible light Or visible light multi-scale differential attention features ; This indicates either Cross-modal Differential Attention Module 1 or Cross-modal Differential Attention Module 2, both with the same architecture, specifically:

[0114] First, the input features are processed to obtain the updated features, represented as follows:

[0115] (11);

[0116] In the formula, Infrared image features are represented as input to the cross-modal differential attention module. or visible light image features , Indicates updated infrared image features or visible light image features ; Represents a 1×1 convolution;

[0117] The enhanced features obtained through two convolutional layers are represented as follows:

[0118] (12);

[0119] In the formula, Indicates the enhanced infrared image features or visible light image features ;

[0120] Extract enhanced differential features, infrared-visible light differential features Represented as:

[0121] (13);

[0122] Visible-infrared difference characteristics Represented as:

[0123] (14);

[0124] Finally, the output features of the cross-modal differential attention are obtained, represented as follows:

[0125] (15);

[0126] (16);

[0127] In the formula, For the infrared output characteristics of cross-modal differential attention, The visible light output characteristics of cross-modal differential attention correspond to .

[0128] The two outputs of the cross-modal differential attention module fully utilize the difference features between the two modalities, enabling interaction between the two modalities and improving the quality of the fused image.

[0129] , After passing through Transformer module one, the infrared global enhancement features are obtained. and visible light global enhancement features The two are added element-wise to obtain the global enhanced feature. ;

[0130] , After passing through Transformer module 2, infrared multi-scale enhanced features are obtained. and visible light multiscale enhancement features The two are added element-wise to obtain multi-scale enhanced features. .

[0131] Transformer Module 1, Transformer Module 2, and All can be done using the existing Transformer model architecture. The Transformer module models the global dependencies between regions in an image through a powerful self-attention mechanism, which can efficiently extract complementary information, enhance semantic consistency, and improve the clarity and information integrity of the fused image in image fusion tasks.

[0132] In S3, , The infrared and visible light image enhancement features F are obtained by adding each element. The infrared and visible light enhancement features add the four features together, making full use of various features and avoiding feature loss.

[0133] In S4, F is input into the decoder, and the decoding process is as follows:

[0134] (17);

[0135] In the formula, This represents a primary fusion feature;

[0136] The process of generating the fused image is as follows:

[0137] (18);

[0138] In the formula, This refers to the Transformer module; This represents the final fused image, which is a fused image of infrared and visible light.

[0139] Example 2: This example is based on Example 1, and... In this process, a complementary guidance gating unit has been added, specifically:

[0140] After generating through equation (11) and Then, calculate their cross-modal complementary weights respectively:

[0141] (19);

[0142] (20);

[0143] In the formula, , These represent the complementary weighting coefficients for the infrared and visible light modes, respectively.

[0144] Applying the weights to the enhanced features obtained in equation (12) yields the gated enhanced features:

[0145] (twenty one);

[0146] In the formula, This represents element-wise multiplication; Indicates infrared gating enhancement features or visible light gated enhancement features ;

[0147] When calculating the differential features, gated enhancement features are used to replace the original enhancement features, and equations (13) and (14) are rewritten as follows:

[0148] (twenty two);

[0149] (twenty three);

[0150] Equations (15) and (16) are updated accordingly as follows:

[0151] (twenty four);

[0152] (25);

[0153] By introducing bidirectional complementary weight gating and dynamically calibrating the features of the current modality using contextual information from another modality, cross-modal collaboration can be enhanced. Suppress features that highly overlap with complementary modes in the current mode and enhance unique information (such as infrared target prominence and visible light texture details); the weighting mechanism of Equation (21) can focus the enhanced features on regions with significant cross-modal differences; only a lightweight weight generation layer (1×1 convolution + pooling) is added, maintaining computational efficiency.

[0154] Example 3: An image fusion device based on cross-modal difference and dual-axis attention network, comprising:

[0155] One or more processors;

[0156] Memory, used to store one or more computer programs;

[0157] When one or more programs are executed by one or more processors, the one or more processors perform the method in Embodiment 1 or Embodiment 2.

[0158] Example 4: A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method in Example 1 or Example 2.

Claims

1. An image fusion method based on cross-modal difference and dual-axis attention network, characterized in that, Includes the following steps: S1. Construct a global enhancement branch and a multi-scale enhancement branch. The global enhancement branch includes a global pooling layer and residual connection, a convolutional-biaxial attention module, a cross-modal difference attention module 1, and a Transformer module 1, which are set up in sequence. The convolutional-biaxial attention module has two layers. The multi-scale enhancement branch includes a convolutional layer, a multi-scale-biaxial attention module, a cross-modal difference attention module 2, and a Transformer module 2, which are set up in sequence. S2. For the input infrared and visible light images, feature extraction and processing are performed through the global enhancement branch and the multi-scale enhancement branch, respectively, to obtain global enhancement features and multi-scale enhancement features; S3. Element-wise addition of global enhancement features and multi-scale enhancement features yields infrared and visible light image enhancement features; S4. Input the infrared and visible light image enhancement features into the decoder and output the infrared and visible light fused image.

2. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 1, characterized in that, In S2, when the input infrared and visible light images pass through the global enhancement branch, the global features of infrared and visible light are first extracted using a global pooling layer and residual connections. The two global features are then processed by a two-layer convolutional-dual-axis attention module, which outputs infrared global dual-axis attention features and visible light global dual-axis attention features. The two global dual-axis attention features are then input to the first cross-modal differential attention module, which outputs infrared global differential attention features and visible light global differential attention features. Finally, the two global differential attention features are input to the first Transformer module, which outputs infrared global enhancement features and visible light global enhancement features. The two are then added element-wise to obtain the global enhancement features. When the input infrared and visible light images pass through the multi-scale enhancement branch, shallow infrared and visible light features are first extracted using convolutional layers. These two shallow features then pass through a multi-scale dual-axis attention module, outputting infrared and visible light multi-scale dual-axis attention features. These two multi-scale dual-axis attention features are then input to a cross-modal differential attention module, which outputs infrared and visible light multi-scale differential attention features. The two multi-scale differential attention features are processed by Transformer module 2 to output infrared multi-scale enhancement features and visible light multi-scale enhancement features. The two are added element-wise to obtain the multi-scale enhancement features.

3. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 2, characterized in that, Global features are extracted using a global pooling layer and residual connections, as follows: （1）； In the formula, x takes the value of or , Represents infrared images or visible light image ; This represents a global average pooling operation; Represents global infrared features or visible light global features .

4. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 3, characterized in that, The process of extracting global biaxial attention features is as follows: First, the global dual-axis attention intermediate features are extracted through the first-layer convolutional dual-axis attention module, represented as: （2）； In the formula, It is the ReLU activation function; This indicates a dual-axis attention module; Indicating infrared global dual-axis attention intermediate features Or visible light global dual-axis attention intermediate features ; This represents a 3×3 convolution; The second-layer convolutional-biaxial attention module is used to... Further processing yields the global biaxial attention features, represented as: （3）； In the formula, Indicating infrared global dual-axis attention features or visible light global dual-axis attention features .

5. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 4, characterized in that, Dual-axis attention module Specifically: First, perform average pooling on the input features: （4）； In the formula, and These represent the vertical global context vector and the horizontal global context vector, respectively. This represents the input to the dual-axis attention module; right and After processing, feature maps in the vertical and horizontal directions are obtained, represented as follows: （5）； In the formula, This represents a splicing operation; This represents a batch normalization operation; This represents the decomposition of the feature map in the horizontal and vertical directions; , These represent feature maps in the vertical and horizontal directions, respectively. right and The two feature weight maps are processed separately and obtained as follows: （6）； In the formula, The value is or , represent or ; This represents the Sigmoid activation function; Feature weight map representing the vertical or horizontal direction , ; Will and , Element-wise multiplication yields the output of the dual-axis attention module. : （7）。 6. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 5, characterized in that, The process of extracting shallow features is as follows: （8）； In the formula, Represents shallow infrared features or visible light shallow layer features .

7. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 6, characterized in that, The extraction method for multi-scale biaxial attention features is as follows: （9）； In the formula, This represents a multi-scale convolution operation; Indicating infrared multi-scale dual-axis attention features Or visible light multi-scale biaxial attention features .

8. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 7, characterized in that, The formulas for obtaining global differential attention features and multi-scale differential attention features are as follows: （10）； In the formula, the value of i is... or , represent or , represent or ; Indicating infrared global differential attention features or infrared multi-scale differential attention features ; Represents the global differential attention features of visible light Or visible light multi-scale differential attention features ; This indicates either Cross-modal Differential Attention Module 1 or Cross-modal Differential Attention Module 2, both with the same architecture, specifically: First, the input features are processed to obtain the updated features, represented as follows: （11）； In the formula, Infrared image features are represented as input to the cross-modal differential attention module. or visible light image features , Indicates updated infrared image features or visible light image features ; Represents a 1×1 convolution; The enhanced features obtained through two convolutional layers are represented as follows: （12）； In the formula, Indicates the enhanced infrared image features or visible light image features ; Extract enhanced differential features, infrared-visible light differential features Represented as: （13）； Visible-infrared difference characteristics Represented as: （14）； Finally, the output features of the cross-modal differential attention are obtained, represented as follows: （15）； （16）； In the formula, For the infrared output characteristics of cross-modal differential attention, The visible light output characteristics of cross-modal differential attention.

9. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 8, characterized in that, , After passing through Transformer module one, the infrared global enhancement features are obtained. and visible light global enhancement features The two are added element-wise to obtain the global enhanced feature. ; , After passing through Transformer module 2, infrared multi-scale enhanced features are obtained. and visible light multiscale enhancement features The two are added element-wise to obtain multi-scale enhanced features. .

10. The image fusion method based on cross-modal difference and dual-axis attention network according to claim 9, characterized in that, In S3, , Element-wise addition yields the infrared and visible light image enhancement features F; In S4, F is input into the decoder, and the decoding process is as follows: （17）； In the formula, This represents a primary fusion feature; The process of generating the fused image is as follows: （18）； In the formula, This refers to the Transformer module; This represents the final fused image, which is a fused image of infrared and visible light.

Images