Infrared and visible light image fusion method and system based on high and low frequency separation enhancement

Abstract

The present invention discloses a method and system for fusion of infrared and visible light images based on high- and low-frequency separation and enhancement, and relates to the field of image processing technology. The method comprises the steps of: obtaining an infrared image and a visible light image to be fused; using an image fusion network to fuse the infrared image and the visible light image to be fused to obtain a fused image; wherein the image fusion step comprises: extracting shallow features of the infrared image and the visible light image, processing the shallow features using a feature separation algorithm to obtain high-frequency features and low-frequency features, enhancing the high-frequency features using a feature enhancement algorithm, and fusing the enhanced high-frequency features with the low-frequency features to obtain a fused image. The present invention separates the high- and low-frequency features, enhances the high-frequency features using a self-attention mechanism, and then cross-fuses the features, thereby highlighting important information such as texture details and improving the image fusion quality.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an infrared and visible light image fusion method and system based on high and low frequency separation enhancement. Background Art

[0002] In the era of information explosion, sensor technology has developed rapidly. Now a single sensor can only provide specific types of information, but cannot provide multimodal information. Image fusion is to fuse the information obtained by multiple sensors to improve the comprehensiveness, accuracy and reliability of information acquisition, eliminate redundant information, compensate and solve the limitations of a single sensor, and generate more comprehensive, more informative and higher quality images.

[0003] The image fused with the visible light image and the infrared image has both rich texture details and thermal information of the target. At the same time, because both are easy to obtain and the cost is low, the fusion of infrared and visible light images has a wider range of applications compared to the fusion of images of other modalities. The fused image can be applied to many fields such as intelligent transportation, target detection, image enhancement, remote sensing, etc.

[0004] Traditional methods are mainly based on multi-scale variation, sparse representation, subspace and other fusion methods. However, traditional methods often find it difficult to balance the preservation and enhancement of image information, which can easily lead to information loss or redundancy. In addition, traditional methods may not be able to fully utilize emerging technologies such as deep learning to automatically extract and optimize features, thus limiting their adaptability and accuracy in complex scenes.

[0005] As deep learning has received more and more attention, many methods have applied deep learning to infrared and visible light image fusion in recent years. By combining Convolutional Neural Network (CNN), Autoencoder (AE), Generative Adversarial Network (GAN) and Transformer, the visual performance of the fusion results is improved, breaking through the limitations of traditional feature extraction methods. Although these methods have achieved good results, they often ignore important information such as edge and texture details, lack separate processing of high-frequency features, and cannot better highlight edge texture information. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for fusion of infrared and visible light images based on high and low frequency separation and enhancement, which separates the high and low frequency features, enhances the high frequency features using the self-attention mechanism and then cross-fuses the features, thereby highlighting important information such as texture details and improving the image fusion quality.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0008] The first aspect of the present invention provides an infrared and visible light image fusion method based on high and low frequency separation enhancement, comprising the following steps:

[0009] Acquire the infrared image and visible light image to be fused;

[0010] The infrared image to be fused and the visible light image are fused by using an image fusion network to obtain a fused image;

[0011] Among them, the image fusion step includes: extracting shallow features of infrared images and visible light images, processing the shallow features using a feature separation algorithm to obtain high-frequency features and low-frequency features, enhancing the high-frequency features using a feature enhancement algorithm, and fusing the enhanced high-frequency features with the low-frequency features to obtain a fused image.

[0012] The second aspect of the present invention provides an infrared and visible light image fusion system based on high and low frequency separation enhancement, comprising:

[0013] An image acquisition module, configured to acquire an infrared image and a visible light image to be fused;

[0014] An image processing module is configured to input the infrared image to be fused and the visible light image into an image fusion network for image fusion to obtain a fused image;

[0015] Among them, the image processing module includes:

[0016] A shallow feature extraction module is configured to extract shallow features of the infrared image and the visible light image;

[0017] The feature separation and enhancement module is configured to process shallow features using a feature separation algorithm to obtain high-frequency features and low-frequency features, and to enhance the high-frequency features using a feature enhancement algorithm;

[0018] The cross fusion module is configured to fuse the enhanced high-frequency features with the low-frequency features to obtain a fused image.

[0019] One or more of the above technical solutions have the following beneficial effects:

[0020] The invention discloses an infrared and visible light image fusion method and system based on high- and low-frequency separation and enhancement. The shallow features of the infrared image and the visible light image are extracted by a shallow feature extraction module; the shallow features are enhanced by a feature separation and enhancement module to realize high-frequency feature enhancement, and important information such as texture details are highlighted; the feature separation and enhancement module includes three identical high-frequency enhancement modules; the high-frequency enhancement module includes an average pooling layer, a maximum pooling layer, a self-attention layer, and a high- and low-frequency fusion layer. The pooling layer is used to separate high- and low-frequency features, the self-attention layer enhances high-frequency features to highlight texture details, and the high- and low-frequency fusion layer realizes the fusion of high-frequency features and low-frequency features; the cross-fusion module uses a cross-attention mechanism to fuse infrared and visible light features, which can better fuse deep features and contain rich contextual information; the convolution enhancement module enhances the fused features, and can effectively extract and enhance important information.

[0021] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0023] Figure 1 This is a flow chart of an infrared and visible light image fusion method based on high and low frequency separation enhancement in Embodiment 1 of the present invention;

[0024] Figure 2 is a schematic diagram of an image fusion network in Embodiment 1 of the present invention;

[0025] Figure 3 Schematic diagram of a shallow feature extraction module in Embodiment 1 of the present invention;

[0026] Figure 4 is a schematic diagram of a feature separation enhancement module in Embodiment 1 of the present invention;

[0027] Figure 5 is a schematic diagram of a high frequency enhancement module in Embodiment 1 of the present invention;

[0028] Figure 6 Schematic diagram of a high-low frequency fusion module in Embodiment 1 of the present invention

[0029] Figure 7 is a schematic diagram of a cross-fusion module in Embodiment 1 of the present invention;

[0030] Figure 8 Schematic diagram of a convolution enhancement module in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0031] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or their combinations;

[0033] Embodiment 1:

[0034] Embodiment 1 of the present invention provides an infrared and visible light image fusion method based on high and low frequency separation enhancement, such as Figure 1 As shown, it includes obtaining an infrared image and a visible light image to be fused, inputting the obtained infrared image and visible light image to be fused into an image fusion network for image fusion, and obtaining a fused image. The present invention can effectively enhance and highlight the high-frequency information of the infrared and visible light fused image.

[0035] This embodiment proposes an algorithm for fusion of infrared and visible light images. The algorithm is implemented by building an image fusion network. The image fusion network is a depth estimation network, such as Figure 2 As shown in the figure, the overall framework of the depth estimation network is based on the encoder-decoder structure, in which the encoder includes a shallow feature extraction module based on Restormer Block and a feature separation and enhancement module, which are respectively used to capture the shallow information of the image and enhance the high-frequency features. The enhanced infrared image and visible light image features are then subjected to a cross fusion module to obtain fused features, which are then enhanced through a convolution enhancement module, and finally the fused image is output through the decoder.

[0036] like Figure 2As shown in the figure, the shallow feature extraction module extracts shallow features of infrared and visible light images. The feature separation and enhancement module consists of three high-frequency enhancement modules. The high-frequency enhancement module divides the features into high-frequency features and low-frequency features through pooling operations, and enhances the high-frequency features through the self-attention mechanism to highlight high-frequency features such as edges and textures. The features after the self-attention mechanism are fused through the cross-fusion module. The cross-fusion module designs a fusion strategy based on the cross-attention mechanism, which can better fuse the deep features between infrared and visible light images and retain rich contextual information. At the same time, in the fusion process, the dimension of the feature map is channel by channel through deformation and transposition operations, which simplifies the number of parameters and shortens the running time. The fused features output by the cross-fusion module are input into the convolution enhancement module and then into the decoder to obtain the final fused image. The convolution enhancement module uses convolutional layers, depth-separable convolutional layers and reverse bottleneck structure ConvFFN to highlight the details and global structure in the fused features, and better promote the fusion of infrared and visible light features. The image reconstruction module, which is the decoder part, adopts the same structure as the shallow feature extraction module to reconstruct the fused image with fused features.

[0037] The specific steps are as follows:

[0038] Step 1: Obtain the infrared image and visible light image to be fused.

[0039] After the infrared image and the visible light image are aligned, they can be input into the image fusion network.

[0040] Step 2: Use the image fusion network to fuse the infrared image and the visible light image to obtain a fused image.

[0041] In this embodiment, the image fusion network includes an encoder, a cross-fusion module, a convolution enhancement module and a decoder. The encoder includes a shallow feature extraction module and a feature separation enhancement module, and the decoder has the same structure as the shallow feature extraction module; the feature separation enhancement module mainly includes three high-frequency enhancement modules; after the infrared image and the visible light image to be fused are respectively subjected to the shallow feature extraction module to extract shallow features, the enhanced infrared image and visible light image features are obtained through the three high-frequency enhancement modules in the feature separation enhancement module, the enhanced infrared image and visible light image features are subjected to the cross-fusion module to obtain fusion features, the fusion features are enhanced through the convolution enhancement module, and then the fusion image is output through the decoder.

[0042] Step 2.1: Extract shallow features of infrared images and visible light images.

[0043] In this embodiment, the shallow features contain more pixel information, such as fine-grained information such as image color, texture, edge, and angular information.

[0044] The structure of the shallow feature extraction module is as follows Figure 3 As shown in the figure, the module is based on Restormer Block, which includes a convolutional layer, a normalization layer, a multi-depth convolution head transposed self-attention layer (Multi-Dconv Head Transposed Attention, MDTA), a normalization layer and a gated deep convolution feed-forward network (Gated-DconvFeed-Forward Network, GDFN) connected in sequence.

[0045] In this embodiment, the acquired infrared and visible light images are first input into the encoder, and the shallow features are extracted by the shallow feature extraction module. and . It can be expressed as:

[0046] .

[0047] in represents the shallow feature extraction module, and represents the input infrared and visible light images, and Respectively represent the shallow features extracted from infrared images and visible light images, R is the set of real numbers, , and Represent the length, width and channel dimensions respectively.

[0048] Step 2.2: Use the feature separation enhancement module to process shallow features using the feature separation algorithm to obtain high-frequency features and low-frequency features.

[0049] In this embodiment, after obtaining the shallow features of the infrared image and the visible light image, a feature separation and enhancement module is used to separate the high-frequency and low-frequency features of the shallow features and enhance the high-frequency features. The structure of the feature separation and enhancement module is as follows: Figure 4 As shown, the feature separation enhancement module consists of three high-frequency enhancement modules. After the features pass through the three high-frequency enhancement modules, the three outputs are summed element by element.

[0050] High frequency enhancement module such as Figure 5 As shown, it includes average pooling layer, maximum pooling layer, self-attention mechanism and high and low frequency fusion module.

[0051] This embodiment uses a pooling layer in the process of high and low frequency separation, while the existing technology mostly uses wavelet transform for decomposition. Compared with wavelet transform, the pooling layer has the advantages of high computational efficiency, good integration with neural network architecture, simple and easy implementation, good learnability and translation invariance. The pooling operation is more suitable for end-to-end training, can provide multi-scale feature extraction capabilities, has low computational complexity, and introduces nonlinearity and dimensionality reduction effects, which helps to improve the generalization ability and performance of the model.

[0052] Step 2.2.1: Original shallow features Downsampling is performed through the average pooling layer to obtain low-frequency features ,The average pooling layer can perform feature compression and remove image noise.

[0053] Step 2.2.2: Then use bilinear interpolation upsampling to get the low-frequency features of the original dimension Finally, using the original shallow features Subtract low frequency features , get high frequency features By this method, high-frequency features can be obtained simply and effectively, and the formula is as follows:

[0054] ,

[0055] .

[0056] in, represents the average pooling operation, Represents the extracted shallow features, representing the infrared image , which represents a visible light image , Represents a bilinear interpolation upsampling operation.

[0057] Step 2.3: Use feature enhancement algorithm to enhance high-frequency features.

[0058] like Figure 5 As shown, step 2.3.1: After extracting the high-frequency features, the self-attention mechanism is used to enhance them to obtain the enhanced high-frequency features .

[0059] Step 2.3.2: Use the high- and low-frequency fusion module to fuse the enhanced high-frequency features with the low-frequency features to obtain the initial fused features.

[0060] The high and low frequency fusion module structure is as follows Figure 6 As shown, it includes a multi-channel module (MCB), a channel attention module of a bilinear interpolation upsampling layer, Convolutional layer, Convolution layer. In the high-low frequency fusion module, in order to better fuse the high- and low-frequency features, this embodiment uses a multi-channel module (Multi-Channel Block, MCB). The multi-channel block consists of two ways, such as Figure 6 As shown, one way is Convolution kernel, which pays more attention to the low-frequency information and edge contours of the image; three Convolution kernels focus more on the high-frequency information of the image. After the low-frequency features pass through three MCBs, they are restored to the original dimension through bilinear interpolation upsampling and fused with the high-frequency features that pass through one MCB. After fusion, they are input into Convolutional layers, channel attention layers, and Convolutional layer: Channel attention can adaptively enhance the focus on useful features and weaken the influence of useless features, which helps to improve the quality of feature representation.

[0061] The specific formula is as follows:

[0062] .

[0063] In the formula, It represents the low-frequency characteristics after three MCBs. It represents the high frequency characteristics after an MCB. Indicates The output of the high frequency enhancement module. represents a multi-channel block, express convolution, express convolution, represents channel attention, representing infrared images , which represents a visible light image In a specific implementation, the attention mechanism in this embodiment adopts the Transformer attention mechanism.

[0064] Step 2.3.3: The initial fused features are again subjected to the maximum pooling layer for maximum pooling, and the obtained high-frequency features are again enhanced by the self-attention mechanism. The low-frequency features of the output after the maximum pooling and the high-frequency features after the self-attention mechanism are again fused using the high-low frequency fusion module with the same structure as before to obtain the enhanced features. The reason why the maximum pooling is used here instead of the average pooling is that the maximum pooling can better retain important features and details, so that the final fused image has a clearer texture.

[0065] After passing through the three high-frequency enhancement modules, the enhanced features and shallow features of the three outputs are summed element by element. At the same time, residual splicing is added to prevent information loss during the downsampling process and ensure the integrity of the information:

[0066] .

[0067] in, Represents the summed features, representing the infrared image , which represents a visible light image .

[0068] Step 2.4: Fuse the enhanced high-frequency features with the low-frequency features to obtain a fused image.

[0069] In order to better fuse the depth features between infrared and visible light images and retain rich context information, this embodiment designs a cross fusion module, such as Figure 7 As shown:

[0070] Step 2.4.1: Obtain enhanced infrared features and visible light features, that is, the summed features , where represents the infrared image ,for , which represents a visible light image ,for .

[0071] Step 2.4.2: Calculate the key vector and numerical vector of infrared features and visible light features respectively.

[0072] In a specific embodiment, a convolutional layer is used to enhance the infrared features and visible light features. Perform convolution and get the key Sum as follows:

[0073] ,

[0074] .

[0075] in, , ,when Represents a visible light image, when represents an infrared image, Reshape represents a reshape operation.

[0076] Step: 2.4.3: Calculate the common query vector by fusing the infrared features with the visible light features .

[0077] For public query vectors , the features are concatenated and fused, and then convolved through the convolution layer. This allows cross-modal features to be better aligned, helping the model to better capture information during feature fusion, while also including information from another modality. The formula is:

[0078] .

[0079] Where, the public query vector , express Operation refers to splicing in the channel dimension.

[0080] Step 2.4.4: Multiply the common query vector by the transpose of the key vectors of the infrared feature and the visible light feature element by element to obtain the feature map of the infrared feature and the visible light feature.

[0081] Then, calculate the single-mode attention map :

[0082] .

[0083] In the formula , the dimension of the feature map is channel by channel, which simplifies the number of parameters and shortens the running time. Represents a transpose operation.

[0084] Step 2.4.5: Multiply the feature maps of the infrared feature and the visible light feature by the numerical vector of the same modality element by element, and then add them element by element with the input features of the same modality to obtain the infrared feature and the visible light feature containing the information of the other modality.

[0085] Then, the obtained feature attention map is compared with the value Multiply them and add them to the enhanced infrared features and visible light features, so that the obtained features contain both the information of the original features and the information of the other modality.

[0086] Step 2.4.6: Fuse the infrared features and visible light features to obtain fused features.

[0087] Finally, the obtained features are fused along the channel dimension and the fused features are obtained through the convolution layer. This process is expressed as:

[0088] ,

[0089] .

[0090] in, represents the output features, represents the fusion feature, Stands for element-wise addition.

[0091] In order to better enhance the representation of fused features, this implementation uses a convolutional enhancement module to better highlight the details and global structures in the image. The convolutional enhancement module structure is as follows: Figure 8 As shown, there are two A convolutional layer and a The depth separable convolution layer and the reverse bottleneck structure ConvFFN are composed of two Convolutional layer. In ConvFFN, the first convolutional layer is used to expand the channel to four times the original value. becomes , and then use the second convolutional layer to restore to the original channel dimension, that is, Restore to , so that more abundant feature information is captured through feature expansion, and the fusion of infrared and visible light features is better promoted. The calculation formula of the whole process is as follows:

[0092] .

[0093] in Depth can be separated convolution, Represents the enhanced fusion features. The GELU activation function is used after each convolution layer.

[0094] The final fusion feature Input into the decoder to reconstruct the fused image. The decoder has the same structure as the shallow feature extraction module. The fused features are input to reconstruct the fused image. :

[0095] .

[0096] in, For the decoder.

[0097] In a specific implementation, after the high-frequency and low-frequency features are fused, a maximum pooling layer operation is performed again, which is the same as the pooling operation principle in the previous shallow feature extraction process, and will not be described in detail here. The specific expression formula is:

[0098] .

[0099] The reason for using the maximum pooling layer here is that maximum pooling can better preserve important features and details, so that the final fused image has clearer texture.

[0100] This embodiment uses a feature separation enhancement module and a convolution enhancement module to enhance high-frequency features and fusion features respectively, so that the algorithm has the ability to enhance detail information in the scene. The feature separation enhancement module is composed of three high-frequency enhancement modules. The high-frequency enhancement module divides the features into high-frequency features and low-frequency features through pooling operations, and enhances the high-frequency features through a self-attention mechanism to highlight high-frequency features such as edges and textures. The convolution enhancement module uses convolution layers, depth-separable convolution layers, and a reverse bottleneck structure ConvFFN to highlight the details and global structure in the fusion features, while better promoting the fusion of infrared and visible light features; and uses a cross-fusion module to better fuse deep features and include rich contextual information.

[0101] Embodiment 2:

[0102] Embodiment 2 of the present invention provides an infrared and visible light image fusion system based on high and low frequency separation enhancement, including:

[0103] An image acquisition module, configured to acquire an infrared image and a visible light image to be fused;

[0104] An image processing module is configured to input the infrared image to be fused and the visible light image into an image fusion network for image fusion to obtain a fused image;

[0105] Among them, the image processing module includes:

[0106] A shallow feature extraction module is configured to extract shallow features of the infrared image and the visible light image;

[0107] The feature separation and enhancement module is configured to process shallow features using a feature separation algorithm to obtain high-frequency features and low-frequency features, and to enhance the high-frequency features using a feature enhancement algorithm;

[0108] The cross fusion module is configured to fuse the enhanced high-frequency features with the low-frequency features to obtain a fused image.

[0109] In this embodiment, Figure 2 As shown in the figure, the image fusion network includes an encoder, a cross-fusion module, a convolution enhancement module and a decoder. The encoder includes a shallow feature extraction module and a feature separation enhancement module, and the decoder has the same structure as the shallow feature extraction module; the feature separation enhancement module mainly includes three high-frequency enhancement modules; after the infrared image and the visible light image to be fused are respectively subjected to the shallow feature extraction module to extract shallow features, the enhanced infrared image and visible light image features are obtained through the three high-frequency enhancement modules in the feature separation enhancement module, the enhanced infrared image and visible light image features are subjected to the cross-fusion module to obtain fusion features, the fusion features are enhanced through the convolution enhancement module, and then the fusion image is output through the decoder.

[0110] Preferably, the high-frequency enhancement module includes an average pooling layer, a maximum pooling layer, a self-attention layer, a bilinear interpolation upsampling layer, and a high- and low-frequency fusion layer.

[0111] Preferably, the high-low frequency fusion layer includes a multi-channel module (MCB), a bilinear interpolation upsampling layer channel attention module, Convolutional layer, Convolutional layer.

[0112] Preferably, the multi-channel module includes two convolutional layers, one routing indivual Convolutional layer composition, one routing indivual Convolutional layer.

[0113] Preferably, the convolution enhancement module includes indivual Convolutional layer, indivual Depthwise separable convolutional layers, A reverse bottleneck structure ConvFFN.

[0114] The steps involved in the above embodiment 2 correspond to those in the method embodiment 1. For the specific implementation method, please refer to the relevant description part of the embodiment 1.

[0115] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. Infrared and visible light image fusion method based on high and low frequency separation enhancement, characterized in that: The following steps are involved: Acquire the infrared image and visible light image to be fused; The infrared image to be fused and the visible light image are fused by using an image fusion network to obtain a fused image; The image fusion step includes: extracting shallow features of the infrared image and the visible light image, processing the shallow features using a feature separation algorithm to obtain high-frequency features and low-frequency features, enhancing the high-frequency features using a feature enhancement algorithm, and fusing the enhanced high-frequency features with the low-frequency features to obtain a fused image; The specific steps of using the feature enhancement algorithm to enhance high-frequency features are: The self-attention mechanism is used to enhance the high-frequency features to obtain enhanced high-frequency features; Use the high- and low-frequency fusion module to fuse the enhanced high-frequency features with the low-frequency features to obtain the initial fused features; The initial fusion features are again pooled through the maximum pooling layer, and the obtained high-frequency features are enhanced again through the self-attention mechanism; The low-frequency features of the output after maximum pooling and the high-frequency features after the self-attention mechanism are fused again using the high- and low-frequency fusion module with the same structure as before to obtain enhanced features.

2. The infrared and visible light image fusion method based on high and low frequency separation enhancement as claimed in claim 1, characterized in that: The image fusion network is a depth estimation network with an encoder-decoder structure.

3. The infrared and visible light image fusion method based on high and low frequency separation enhancement as claimed in claim 1, characterized in that: The specific steps of using the feature separation algorithm to process shallow features include: The original shallow features are downsampled through the average pooling layer to obtain low-frequency features; Use bilinear interpolation upsampling to obtain low-frequency features of the original dimension; The high-frequency features are obtained by subtracting the low-frequency features from the original shallow features.

4. The infrared and visible light image fusion method based on high and low frequency separation enhancement as claimed in claim 1, characterized in that: A feature separation enhancement module is adopted to enhance the high-frequency features using a feature enhancement algorithm, and the feature separation enhancement module is composed of three high-frequency enhancement modules.

5. The infrared and visible light image fusion method based on high and low frequency separation enhancement as claimed in claim 4, characterized in that: The high-frequency enhancement module includes an average pooling layer, a maximum pooling layer, a self-attention mechanism, and a high- and low-frequency fusion module.

6. The infrared and visible light image fusion method based on high and low frequency separation enhancement as claimed in claim 5, characterized in that: The high-low frequency fusion module includes a multi-channel module, a channel attention module of a bilinear interpolation upsampling layer, and a convolution layer.

7. The infrared and visible light image fusion method based on high and low frequency separation enhancement as claimed in claim 6, characterized in that: After the shallow features pass through three high-frequency enhancement modules, the enhanced features and shallow features of the three outputs are summed element by element. At the same time, residual splicing is added to prevent information loss during the downsampling process.

8. The infrared and visible light image fusion method based on high and low frequency separation enhancement as claimed in claim 6, characterized in that: The specific steps of fusing the enhanced high-frequency features with the low-frequency features are: Obtain enhanced infrared and visible light features; Calculate the key vector and numerical vector of infrared features and visible light features respectively; The common query vector is calculated by fusing the infrared features with the visible light features; Multiply the common query vector by the transpose of the key vectors of the infrared feature and the visible light feature element by element to obtain a feature map of the infrared feature and the visible light feature; The feature graphs of infrared features and visible light features are multiplied element by element with the numerical vector of the same modality, and then added element by element with the input features of the same modality to obtain infrared features and visible light features containing information of another modality; The obtained infrared features are fused with the visible light features to obtain fused features.

9. Infrared and visible light image fusion system based on high and low frequency separation enhancement, characterized in that: include: An image acquisition module, configured to acquire an infrared image and a visible light image to be fused; An image processing module is configured to input the infrared image to be fused and the visible light image into an image fusion network for image fusion to obtain a fused image; Among them, the image processing module includes: A shallow feature extraction module is configured to extract shallow features of the infrared image and the visible light image; The feature separation and enhancement module is configured to process shallow features using a feature separation algorithm to obtain high-frequency features and low-frequency features, and to enhance the high-frequency features using a feature enhancement algorithm; A cross fusion module is configured to fuse the enhanced high-frequency features with the low-frequency features to obtain a fused image; The specific steps of using the feature enhancement algorithm to enhance high-frequency features are: The self-attention mechanism is used to enhance the high-frequency features to obtain enhanced high-frequency features; Use the high- and low-frequency fusion module to fuse the enhanced high-frequency features with the low-frequency features to obtain the initial fused features; The initial fusion features are again pooled through the maximum pooling layer, and the obtained high-frequency features are enhanced again through the self-attention mechanism; The low-frequency features of the output after maximum pooling and the high-frequency features after the self-attention mechanism are fused again using the high- and low-frequency fusion module with the same structure as before to obtain enhanced features.

Images