Infrared and visible light image gating fusion method based on illumination perception network

Abstract

The invention discloses an infrared and visible light image gating fusion method based on an illumination perception network, and the method comprises the steps: designing an illumination perception network, and calculating the intensity probability of the visible light intensity of a current scene; and then, firstly, selecting an output channel by using light intensity probabilities, if the output channel is a fusion channel, constructing illumination perception loss by using the probabilities, and guiding the fusion channel to fuse the infrared and visible light images. In a fusion channel part, a multi-attribute sharing attention mechanism module is designed, and an attention fusion module is embedded in different scale feature layers to avoid loss of different scale target areas. Secondly, human brain multi-modal information co-processing is simulated in a multi-attribute sharing attention mechanism module part, a multi-attribute sharing attention module is designed, the network operation speed is increased, and the requirement for real-time performance is met.

Description

Technical Field

[0001] The present invention belongs to the field of infrared and visible light image processing, and in particular relates to a gating and fusion method of infrared and visible light images based on a light perception network. Background Art

[0002] Infrared images can reflect the thermal radiation information of the target and distinguish the target from the background by the difference in pixel intensity. Therefore, the contrast between the target and the background is high and it is not easily affected by bad weather. However, it has the disadvantages of unclear detail information, low texture contrast, poor imaging performance and poor visibility. Visible light imaging sensors image based on the difference in visible light reflected by objects. The difference in visible light reflected by objects is mainly caused by the different structures of the objects. Therefore, visible light images can well reflect the texture details of objects. They have the advantages of high resolution, rich texture details and low noise. They can capture rich appearance information, but they are easily disturbed by bad weather. The fusion of infrared and visible light images can make up for the shortcomings of the imaging capabilities of their respective imaging sensors. By combining the target intensity of infrared images and the texture details of visible light images, complementary information of the same scene can be obtained at the same time, and the recognition efficiency of targets in complex environments can be improved. Therefore, it is widely used in medical imaging, remote sensing, target recognition, industrial detection and other fields.

[0003] According to different fusion theories, infrared and visible light image fusion technology can be divided into the following fusion algorithms: methods based on multi-scale transformation, methods based on sparse representation, methods based on neural networks, methods based on subspace, methods based on saliency, hybrid models and other methods. Although there have been many studies on infrared and visible light image fusion technology, no matter which type of fusion algorithm, they all face a common problem, that is, excessive pursuit of the contrast of the fused image will lead to the loss of the target intensity of the infrared image, which will make it difficult to distinguish the target from the background in the fused image. Excessive pursuit of a higher contrast between the target and the background will lead to too high a proportion of infrared information in the fused image. Infrared images contain a lot of noise information, which will directly affect the spatial resolution of the fusion result and make it difficult to detect and identify subsequent targets. Although simply adding two images can avoid the problem of information loss during the fusion process, the fusion result does not conform to the visual habits of the human eye. Therefore, the current difficulty in the fusion of infrared and visible light images is that no algorithm can cope with various complex changes in conditions at the same time. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for gating and fusing infrared and visible light images based on an illumination perception network in view of the deficiencies of the prior art, comprising the following steps:

[0005] Step 1: Design a light perception subnetwork N IA, using the illumination perception subnetwork N IA Calculate the visible light image I vis The probability of belonging to daytime is P d and the probability P of belonging to the evening n ;

[0006] Step 2: Design a lighting allocation mechanism based on the visible light image I vis The illumination probability is used to calculate the weight W of the infrared image. ir and the weight W of the visible light image vis ;

[0007] Step 3: Design a channel selection module based on light perception, use the channel selection mechanism to determine the channel type of the output result, and realize the adaptive selection of the output channels of infrared images, visible light images, and fused images;

[0008] Step 4: If the output channel selected by the channel selection module is an image fusion channel, the infrared and visible light images are multi-scale decomposed using a guided filter to obtain multi-scale features of the infrared and visible light images;

[0009] Step 5, input the multi-scale feature map obtained by decomposition into the fusion network, the fusion network includes a multi-attribute shared attention feature extraction module and a feature reconstruction network, the multi-attribute shared attention feature extraction module includes a multi-attribute shared attention mechanism module MASA based on channel attention and spatial attention sharing and a residual network ResNet, the multi-attribute shared attention feature extraction module is used to extract and fuse the spatial feature information and channel feature information of the infrared image and the visible light image, and finally reconstruct the fused image through a feature reconstruction network composed of the residual network ResNet.

[0010] In step 1, the light perception subnetwork N IA The illumination perception subnetwork N includes four convolutional layers, one global average pooling layer and two fully connected layers. IA A 4×4 convolution kernel with a stride of 2 is used to extract lighting information. Leaky Relu is used as the activation function in all convolutional layers. Then, a global average pooling operation is used to integrate the lighting information. The last two fully connected layers calculate the lighting probability based on the lighting information and autonomously evaluate the current input visible light image I according to the lighting probability. vis The probability of belonging to daytime is P d and the probability P of belonging to the evening n ; Use cross entropy loss to constrain the illumination perception subnetwork N IA The training process of the illumination perception subnetwork N IA The loss function L is expressed as:

[0011] L=-[Z*log(y)+(1-Z)*log(1-y)]

[0012] Where Z represents the illumination label of the input image, y = [P n ,P d ] represents the illumination perception subnetwork N IA Output.

[0013] In step 2, the weight W of the infrared image is calculated using the following formula: ir and the weight W of the visible light image vis :

[0014]

[0015]

[0016] Step 3 includes: setting two empirical thresholds T ir and T vis ,when When , directly select the infrared single channel as the output and output the infrared image; when When , directly select the visible light single channel as the output and output the visible light image; when When , the system selects the image fusion channel and outputs the infrared and visible light fusion image.

[0017] In step 4, when performing multi-scale decomposition, the relationship between the input image I and its multi-scale graph is I = I1 + I2 + ... + I n , where I n represents the nth multi-scale map of the input image I.

[0018] In step 5, the multi-attribute shared attention mechanism module MASA includes a spatial attention fusion module, a channel attention feature fusion module and a multi-attribute shared attention module;

[0019] The spatial attention fusion module is used to obtain a fusion feature map f of infrared and visible light images by extracting spatial feature information. spatial ;

[0020] The channel attention feature fusion module is used to obtain the channel attention feature fusion result f of the infrared and visible light images by extracting the channel feature information. channel ;

[0021] The multi-attribute shared attention module is used to group attributes according to the degree of semantic similarity, and the features of the same group of attributes share the same attention branch to calculate the fused feature f fuse .

[0022] In step 5, the fused feature f is calculated using the following formulafuse :

[0023] f fuse =0.5×(f channel +f spatial ).

[0024] In step 5, after obtaining the fused image, return to the illumination perception network N IA , calculate the probability that the fused image belongs to the daytime and the probability that it belongs to the nighttime, define the illumination loss function, and feed back the fusion network training process according to the illumination loss function. The illumination loss function L illum It is expressed as:

[0025]

[0026] in and Respectively represent the intensity loss of infrared image and the intensity loss of visible light image, W ir and W vi They represent the illumination perception weights of infrared images and visible light images respectively. The intensity loss aims to measure the difference between the fused image and the input image at the pixel level and is defined as:

[0027]

[0028] Where H and W represent the height and width of the input image, respectively. f represents the fused image, I vi and I ir Represent visible light image and infrared image respectively.

[0029] In step 5, the spatial attention fusion module performs the following steps:

[0030] Step 5-1-1, input infrared feature map f1 and visible light feature map f2, the dimension is H×W×C, extract the spatial statistical information of the feature map, perform global average pooling and global maximum pooling on the infrared feature map f1 and visible light feature map f2 in the channel dimension, and concatenate the pooling results in the channel dimension to obtain a pooling feature map with a dimension of H×W×2, where H and W represent the height and width of the pooling feature map respectively, and C represents the number of channels;

[0031] Step 5-1-2, calculate the spatial weight based on the spatial statistical information of the infrared feature map f1 and the visible light feature map f2: the spatial attention fusion module uses a convolution layer with an output channel of 1, a convolution kernel size of 1, and a step size of 1 to extract the spatial feature information of the pooled feature map, and then uses the extracted feature information as the spatial weight information v1 of the infrared feature map f1 and the spatial weight information v2 of the visible light feature map f after the sigmoid activation function; the convolution layer weights are shared when extracting features from the pooled feature maps of the infrared feature map f1 and the visible light feature map f2;

[0032] Step 5-1-3, enhance the spatial region information of the infrared feature map f1 and the visible light feature map f2 according to the spatial weight information, and then add the enhanced feature maps to obtain the fused feature map f spatial , the expression is:

[0033] f spatial =v1×f1+v2×f2.

[0034] The channel attention feature fusion module performs the following steps:

[0035] Step 5-2-1, input infrared feature map f1 and visible light feature map f2, the dimension is H×W×C, perform average pooling operation on infrared feature map f1 and visible light feature map f2 to complete the statistics of channel information, and the obtained outputs are recorded as feature S1 and feature S2 respectively;

[0036] Step 5-2-2, use a convolution layer with an output dimension of d, a convolution kernel size of 1, and a step size of 1 to perform a convolution operation on features S1 and S2 to obtain more compact features Z1 and Z2; the dimension d of features Z1 and Z2 is controlled by the ratio r and the minimum value l, expressed as:

[0037]

[0038] Step 5-2-3, perform convolution operation on features Z1 and Z2 through a convolution layer with an output channel of C, a convolution kernel size of 1, and a step size of 1, and perform softmax operation on the output results to obtain the channel weight w1 of the infrared feature map f1 and the channel weight w2 of the visible light feature map f2; the convolution layer weights are shared when extracting features from the channel information of the infrared feature map f1 and the visible light feature map f2;

[0039] Step 5-2-4, perform weighted calculation on the infrared feature map f1 and the visible light feature map f2 according to the channel weight to obtain the channel attention feature fusion result f channel :

[0040] f channel =v1×f1+v2×f2.

[0041] Beneficial effects:

[0042] The present invention discloses a method for gating and fusion of infrared and visible light images based on an illumination perception network. Aiming at the problem that there is a large amount of redundant information in infrared and visible light images and the full-process fusion mechanism will greatly increase the algorithm operation cost, an adaptive gating and fusion network of infrared and visible light images based on illumination perception is studied to realize efficient adaptive gating and fusion of multi-dimensional information. First, an illumination perception subnetwork is designed to calculate the probability of obtaining the illumination intensity of the visible light image of the current scene. Then, by constructing an adaptive gating mechanism, the output channel is adaptively selected according to the result of illumination perception, and the feature information of infrared and visible light images is integrated to realize the adaptive utilization of infrared channel, visible light channel and fusion channel. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0044] Figure 1 The figure is a schematic flow chart of the steps for implementing the present invention.

[0045] Figure 2 The figure is a schematic diagram of the structure of a light sensing network according to an embodiment of the present invention.

[0046] Figure 3 The figure is a schematic diagram of a fusion network structure according to an embodiment of the present invention.

[0047] Figure 4 A schematic diagram of a spatial attention feature fusion module according to an embodiment of the present invention.

[0048] Figure 5 A schematic diagram of a channel attention feature fusion module according to an embodiment of the present invention.

[0049] Figure 6 A schematic diagram of a multi-attribute shared attention module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] This embodiment discloses a method for gating and fusion of infrared and visible light images based on a light perception network. In view of the problem that there is a large amount of redundant information in infrared and visible light images and the full-process fusion mechanism will greatly increase the algorithm computing cost, an infrared and visible light image adaptive gating and fusion network based on light perception is studied to achieve efficient adaptive gating and fusion of multi-dimensional information. First, a light perception subnetwork is designed to calculate the probability of obtaining the light intensity of the visible light image of the current scene. Then, by constructing an adaptive gating mechanism, the output channel is adaptively selected according to the result of light perception, and the infrared and visible light image feature information is integrated to achieve adaptive utilization of infrared channels, visible light channels, and fusion channels. Figure 1 As shown, the specific steps include:

[0051] Step 1: Input infrared and visible light images and use the light perception network N IA In one embodiment of the present invention, the structure of the illumination sensing sub-network is as follows: Figure 2 As shown in the figure, the illumination perception subnetwork consists of four convolutional layers, one global average pooling layer, and two fully connected layers. A 4×4 convolution kernel with a stride of 2 is used to extract illumination information. All convolutional layers use Leaky Relu as the activation function, and then a global average pooling operation is used to integrate the illumination information. The last two fully connected layers calculate the illumination probability based on the illumination information. The currently input visible light image I is autonomously evaluated based on the illumination probability. vis The probability of belonging to daytime is P d and the probability P of belonging to the evening n The light perception network uses cross entropy loss to constrain the training process of the light perception sub-network. The loss function of the light perception sub-network is expressed as:

[0052] L=-[Z*log(y)+(1-Z)*log(1-y)]

[0053] Z represents the illumination label of the input image, y = [P n ,P d ] represents the output of the light perception sub-network.

[0054] The illumination perception subnetwork is used to evaluate the illumination conditions of the scene where the visible light image is taken, and output the probability P that the visible light image belongs to daytime. d and the probability P of belonging to the evening n .

[0055] Step 2: Calculate the weight of the infrared image using the illumination distribution mechanism and the weights of the visible light image Using the light perception network, we calculated the weights of infrared and visible light images in a large number of data sets, and set two empirical thresholds T for extremely poor and excellent lighting conditions based on experience. ir and Tvis

[0056] Step 3: Use the channel selection mechanism to determine the output channel. When , it is considered that the scene lighting conditions of the input image are extremely poor, and the visible light image contains less effective information. If it is still fused with the infrared image at this time, it will seriously affect the image quality of the fusion result. Therefore, the system directly selects the infrared single channel as the output and outputs the infrared image; when When , it is considered that the lighting conditions of the shooting scene of the input image are very good, and the visual effect of the visible light image is excellent. If it is fused with the infrared image at this time, it will only introduce more noise information. Therefore, the system directly selects the visible light single channel as the output and outputs the visible light image; when When the infrared and visible light images are taken under the lighting conditions, the system considers that the infrared and visible light images have complementary advantages. The system selects the image fusion channel and transmits the infrared and visible light images to the image fusion module.

[0057] Step 4: If the output channel selected by the channel selection module is the image fusion channel, the infrared and visible light images are decomposed into multi-scale using the guided filter to obtain the multi-scale features of the infrared and visible light images. The relationship between the input image I and its multi-scale map is I = I1 + I2 + ....... + I n .

[0058] Step 5: Input the decomposed multi-scale feature map into the multi-attribute shared attention feature fusion module to extract the spatial attention features and channel attention features, and use the illumination condition probability to design the loss function to guide the feature fusion network training and obtain the feature fusion image. Figure 3 shown.

[0059] In one embodiment, step 5 includes:

[0060] Step 5-1, the spatial attention feature fusion of the multi-scale features f1 and f2 of the infrared and visible light images. The schematic diagram of the spatial attention feature fusion network in this implementation is as follows: Figure 4As shown. First, the spatial statistical information of the feature map is extracted by performing global average pooling and global maximum pooling on f1 and f2 in the channel dimension, and then concatenating the pooling results in the channel dimension to obtain a pooled feature map with a dimension of H×W×2. Then, the spatial weight is calculated based on the spatial statistical information of f1 and f2. This module uses a convolutional layer with an output channel of 1, a convolution kernel size of 1, and a step size of 1 to extract the spatial feature information of the pooled feature map. After that, the extracted feature information is passed through a sigmoid activation function and used as the spatial weight information v1 and v2 of the feature maps f1 and f2. Here, the convolution layer weights are shared when extracting features from the pooled feature maps of f1 and f2. Finally, the spatial region information of features f1 and f2 is enhanced according to the spatial weights, and the enhanced feature maps are added together to obtain the fused feature map f spatial , whose expression is:

[0061] f spatial =v1×f1+v2×f2

[0062] Step 5-2, channel attention feature fusion of multi-scale features f1 f2 of infrared and visible light images. The schematic diagram of the spatial attention feature fusion network in this implementation is as follows: Figure 5 As shown. First, average pooling is performed on f1 and f2 to complete the statistics of channel information. Then, a convolutional layer with an output channel of d, a convolution kernel size of 1, and a step size of 1 is used to perform convolution operations on features S1 and S2 to obtain more compact features Z1 and Z2. The dimension d of features Z1 and Z2 can be controlled by the ratio r and the minimum value l, which is expressed as:

[0063]

[0064] After that, the features Z1 and Z2 are convolved through a convolutional layer with an output channel of C, a convolution kernel size of 1, and a stride of 1, and the output result is subjected to a softmax operation to obtain the channel weights v1 and v2 of different features. Similar to the spatial attention module, the convolutional layer weights for feature extraction and transformation of the channel statistics of f1 and f2 are shared here.

[0065] Finally, the features f1 and f2 are weighted according to the channel weight information to obtain the channel attention feature fusion result f channel :

[0066] f channel =v1×f1+v2×f2

[0067] Step 5-3, design a multi-attribute shared attention module. The modules are grouped according to the degree of semantic similarity. The features of the same group of attributes share the same attention module to calculate the features for fusion, so as to reduce the running cost. The schematic diagram of the multi-attribute shared attention module in this implementation is as follows: Figure 6 shown.

[0068] Step 5-4, fuse the two fused feature maps to obtain the final fused feature f fuse , the calculation process of fusion features is:

[0069] f fuse =0.5×(f channel +f spatial )

[0070] Step 5-5, after obtaining the fused image, return to the illumination perception network, calculate the illumination probability, define the illumination loss function, and iteratively train the fusion network based on the feedback of the loss function to obtain the final fused image.

[0071] The present invention provides a method for gating and fusing infrared and visible light images based on an illumination sensing network. There are many methods and approaches to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for gating and fusion of infrared and visible light images based on an illumination perception network, characterized in that: The following steps are involved: Step 1: Design a light perception subnetwork N IA , using the illumination perception subnetwork N IA Calculate the visible light image I vis The probability of belonging to daytime is P d and the probability P of belonging to the evening n ; Step 2: Design a lighting allocation mechanism based on the visible light image I vis The illumination probability is used to calculate the weight W of the infrared image. ir and the weight W of the visible light image vis ; Step 3: Design a channel selection module based on light perception, use the channel selection mechanism to determine the channel type of the output result, and realize the adaptive selection of the output channels of infrared images, visible light images, and fused images; Step 4: If the output channel selected by the channel selection module is an image fusion channel, the infrared and visible light images are multi-scale decomposed using a guided filter to obtain multi-scale features of the infrared and visible light images; Step 5, input the multi-scale feature map obtained by decomposition into the fusion network, the fusion network includes a multi-attribute shared attention feature extraction module and a feature reconstruction network, the multi-attribute shared attention feature extraction module includes a multi-attribute shared attention mechanism module MASA based on channel attention and spatial attention sharing and a residual network ResNet, the multi-attribute shared attention feature extraction module is used to extract and fuse the spatial feature information and channel feature information of the infrared image and the visible light image, and finally reconstruct the fused image through a feature reconstruction network composed of the residual network ResNet.

2. The method according to claim 1, characterized in that In step 1, the light perception subnetwork N IA The illumination perception subnetwork N includes four convolutional layers, one global average pooling layer and two fully connected layers. IA A 4×4 convolution kernel with a stride of 2 is used to extract lighting information. Leaky Relu is used as the activation function in all convolutional layers. Then, a global average pooling operation is used to integrate the lighting information. The last two fully connected layers calculate the lighting probability based on the lighting information and autonomously evaluate the current input visible light image I according to the lighting probability. vis The probability of belonging to daytime is P d and the probability P of belonging to the evening n ; Use cross entropy loss to constrain the illumination perception subnetwork N IA The training process of the illumination perception subnetwork N IA The loss function L is expressed as: L=-[Z*log(y)+(1-Z)*log(1-y)] Where Z represents the illumination label of the input image, y = [P n ,P d ] represents the illumination perception subnetwork N IA Output.

3. The method according to claim 2, characterized in that In step 2, the weight W of the infrared image is calculated using the following formula: ir and the weight W of the visible light image vis :

4. The method according to claim 3, characterized in that: Step 3 includes: setting two empirical thresholds T ir and T vis ,when When , directly select the infrared single channel as the output and output the infrared image; when When , directly select the visible light single channel as the output and output the visible light image; when When , the system selects the image fusion channel and outputs the infrared and visible light fusion image.

5. The method according to claim 4, characterized in that In step 4, when performing multi-scale decomposition, the relationship between the input image I and its multi-scale graph is I = I1 + I2 + ... + I n , where I n represents the nth multi-scale map of the input image I.

6. The method according to claim 5, characterized in that In step 5, the multi-attribute shared attention mechanism module MASA includes a spatial attention fusion module, a channel attention feature fusion module and a multi-attribute shared attention module; The spatial attention fusion module is used to obtain a fusion feature map f of infrared and visible light images by extracting spatial feature information. spatial ; The channel attention feature fusion module is used to obtain the channel attention feature fusion result f of the infrared and visible light images by extracting the channel feature information. channel ; The multi-attribute shared attention module is used to group attributes according to the degree of semantic similarity, and the features of the same group of attributes share the same attention branch to calculate the fused feature f fuse .

7. The method according to claim 6, characterized in that In step 5, the fused feature f is calculated using the following formula fuse : f fuse =0.5×(f channel +f spatial )。 8. The method according to claim 7, characterized in that In step 5, after obtaining the fused image, return to the illumination perception network N IA , calculate the probability that the fused image belongs to the daytime and the probability that it belongs to the nighttime, define the illumination loss function, and feed back the fusion network training process according to the illumination loss function. The illumination loss function L illum It is expressed as: in and Respectively represent the intensity loss of infrared image and the intensity loss of visible light image, W ir and W vi They represent the illumination perception weights of infrared images and visible light images respectively. The intensity loss aims to measure the difference between the fused image and the input image at the pixel level and is defined as: Where H and W represent the height and width of the input image, respectively. f represents the fused image, I vi and I ir Represent visible light image and infrared image respectively.

9. The method according to claim 8, characterized in that In step 5, the spatial attention fusion module performs the following steps: Step 5-1-1, input infrared feature map f1 and visible light feature map f2, the dimension is H×W×C, extract the spatial statistical information of the feature map, perform global average pooling and global maximum pooling on the infrared feature map f1 and visible light feature map f2 in the channel dimension, and concatenate the pooling results in the channel dimension to obtain a pooling feature map with a dimension of H×W×2, where H and W represent the height and width of the pooling feature map respectively, and C represents the number of channels; Step 5-1-2, calculate the spatial weight based on the spatial statistical information of the infrared feature map f1 and the visible light feature map f2: the spatial attention fusion module uses a convolution layer with an output channel of 1, a convolution kernel size of 1, and a step size of 1 to extract the spatial feature information of the pooled feature map, and then uses the extracted feature information as the spatial weight information v1 of the infrared feature map f1 and the spatial weight information v2 of the visible light feature map f after the sigmoid activation function; the convolution layer weights are shared when extracting features from the pooled feature maps of the infrared feature map f1 and the visible light feature map f2; Step 5-1-3, enhance the spatial region information of the infrared feature map f1 and the visible light feature map f2 according to the spatial weight information, and then add the enhanced feature maps to obtain the fused feature map f spatial , the expression is: f spatial =v1×f1+v2×f2。 10. The method according to claim 9, characterized in that The channel attention feature fusion module performs the following steps: Step 5-2-1, input infrared feature map f1 and visible light feature map f2, the dimension is H×W×C, perform average pooling operation on infrared feature map f1 and visible light feature map f2 to complete the statistics of channel information, and the obtained outputs are recorded as feature S1 and feature S2 respectively; Step 5-2-2, use a convolution layer with an output dimension of d, a convolution kernel size of 1, and a step size of 1 to perform a convolution operation on features S1 and S2 to obtain more compact features Z1 and Z2; the dimension d of features Z1 and Z2 is controlled by the ratio r and the minimum value l, expressed as: Step 5-2-3, perform convolution operation on features Z1 and Z2 through a convolution layer with an output channel of C, a convolution kernel size of 1, and a step size of 1, and perform softmax operation on the output results to obtain the channel weight w1 of the infrared feature map f1 and the channel weight w2 of the visible light feature map f2; the convolution layer weights are shared when extracting features from the channel information of the infrared feature map f1 and the visible light feature map f2; Step 5-2-4, perform weighted calculation on the infrared feature map f1 and the visible light feature map f2 according to the channel weight to obtain the channel attention feature fusion result f channel : f channel =v1×f1+v2×f2。

Images