A Multi-channel Image Fusion Method and System Based on Coupled Learning under Ultra-low Illumination

Abstract

This invention provides a multi-channel image fusion method and system based on coupled learning under ultra-low illumination conditions. The method includes preprocessing a dataset of low-light images with a preset sample size, followed by photometric feature extraction. This includes: constructing a multi-level attention model for light perception information extraction; constructing the photometric probability of the output image from the photometric information extraction network; building an auxiliary module for extracting common important features of the same target; and using convolutional dimensionality reduction to retain the optimal features selected from each channel image. Based on the images contained in the low-light image dataset, a multi-path image fusion framework is trained, including a synthesis learning stage and a reverse optimization stage, fusing the images to form the multi-path image fusion framework. The preprocessed images are input into the multi-path image fusion framework, and the fused image is output. The multi-channel image fusion method proposed in this invention can effectively preserve the light intensity and foreground / background information in each image while reducing the phenomenon of missing effective information and redundant invalid information during the network learning process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image fusion processing, and particularly relates to a multi-channel image fusion method and system based on coupled learning under ultra-low illumination, a computer medium and a computer. BACKGROUND

[0002] The low-light image fusion method is a technology for fusing images captured by low-light sensors (night vision devices, infrared cameras, etc.) and images captured by visible light sensors (ordinary cameras). Such fusion can provide clearer and more detailed images in insufficiently illuminated or completely dark environments, which helps to enhance the performance of target detection, recognition and surveillance applications. The low-light image fusion method has a wide range of applications in military, security, night driving and other fields. It can help people better perceive and understand the environment in low-light conditions, and improve the efficiency and accuracy of task execution.

[0003] At present, existing low-light image fusion methods generally include:

[0004] 1. Most of them use traditional fusion methods, such as multi-scale transform fusion, subspace-based fusion, and saliency-based fusion. Although these methods have achieved good fusion results, these methods rely on predefined transformation modes and corresponding scale components for source image decomposition and reconstruction, and there is no corresponding standard to evaluate the pros and cons of these transformation modes. Therefore, blindly selecting transformation modes cannot well preserve the high-quality information of source images. In addition, the current fusion rules are mostly too simple, resulting in some artifacts in the fusion results, and the quality of the fusion images cannot be guaranteed.

[0005] 2. A few use deep learning-based fusion methods. Compared with traditional fusion methods, deep learning-based fusion aims to provide information-rich fusion images by combining complementary information from different sensors. Existing learning-based fusion methods try to construct various loss functions to preserve the complementary features of different channel source images, but ignore the discovery of the mutual relationship between different channels, resulting in redundant or even invalid information in the fusion results. Moreover, they mostly focus on strengthening the network as the number of network layers increases, while ignoring the importance of feature transmission, leading to degradation of important information and compromised fusion results.

[0006] Therefore, there is an urgent need for a multi-channel image fusion method that can effectively preserve the illumination intensity and foreground / background information in the image while reducing the loss of valid information and the redundancy of invalid information in the learning process. SUMMARY

[0007] In order to be able to effectively preserve the light intensity and foreground / background information in the image, and reduce the phenomenon of missing effective information and redundant invalid information in the learning process, one aspect of the present application provides a multi-channel image fusion method based on coupled learning under ultra-low illumination, comprising:

[0008] Step S1: according to the preset sample number of low-light image data set, data preprocessing is carried out;

[0009] Step S2: according to the low-light image data set, photometric feature information extraction is carried out, including: constructing a multi-level attention model for light perception information extraction, and then constructing a photometric probability of the output image of the photometric information extraction network, constructing an auxiliary module for extracting the same target public important features, and then using the method of convolution dimension reduction to retain the selected optimal features;

[0010] Step S3: according to the pictures contained in the low-light image data set, training a multi-channel image fusion framework, including: a synthetic learning stage and a reverse optimization stage, and then fusing the synthetic learning stage and the reverse optimization stage to form a multi-channel image fusion framework;

[0011] Step S4: inputting the pictures after data preprocessing into the multi-channel image fusion framework, and outputting the fused images.

[0012] Preferably, the method of data preprocessing is:

[0013] Step S10: using a method based on semi-automatic feature matching to register the multi-channel images, and outputting registration pictures with the same size as the calibrated camera;

[0014] Step S11: cutting the public region of the registration pictures to retain the preset target;

[0015] Step S12: using the LPDM method to enhance the images below the preset illumination value.

[0016] Preferably, the method of using a method based on semi-automatic feature matching to register the multi-channel images is:

[0017] Step S100: using a method of manually selecting control points to combine the images of different channels with the calibrated camera images, manually selecting a group of corresponding points with a preset number of obvious features and prominent targets, and then calculating the corresponding homography matrix through the least square method;

[0018] Step S101: for any group of images, using an encoding method to realize automatic image registration under the same experimental environment by using the homography matrix.

[0019] Preferably, the method of photometric feature information extraction is:

[0020] Step S20: constructing a multi-level attention model for illumination perception information extraction, based on a model pre-trained on an ImageNet dataset, and then training the multi-level attention model through a micro-light image dataset;

[0021] Step S21: using a ResNet-50 network to build a luminosity information extraction network, outputting luminosity probabilities of corresponding images, so as to determine a current fusion base image using the trained multi-level attention model for a to-be-fused image and a fusion process image;

[0022] Step S22: generating luminosity information extracted from each channel image according to the luminosity information extraction network, constructing an auxiliary module for extracting the same target public important features, and then using a convolution dimension reduction method to retain the selected optimal features.

[0023] Preferably, the method of the synthetic learning stage comprises:

[0024] Step S30: given a preset number of registered and aligned pictures, importing the pictures into a multi-channel image fusion framework;

[0025] Step S31: fusing the pictures through the multi-channel image fusion framework to generate a fusion image.

[0026] Preferably, the method of the reverse optimization stage comprises:

[0027] Step S32: constraining the fusion image by learning foreground and background features in the source image;

[0028] Step S33: combining the features of the source image to provide optimal fusion weights, obtaining an image rich in information, sufficient in details, and prominent in targets.

[0029] Preferably, the method of constraining the fusion image by learning foreground and background features in the source image comprises:

[0030] Step S320: generating a mask of a preposed background for an EBCMOS image, and superimposing the mask with an infrared image and a visible light black and white image, respectively;

[0031] Step S321: generating a mask of a postposed background for the infrared image and the visible light black and white image, superimposing the mask with the EBCMOS image, respectively, setting positive and negative samples, evaluating the similarity of the features, and setting the score of the similarity of the positive and negative samples of each channel image as a loss function of the channel image:

[0032]

[0033] Wherein, N represents the number of layers of VGG-19 network, M represents the number of negative samples set in each kind of channel image, V, E, I represent the visible light black and white image, EBCMOS image, infrared image respectively, w i represents the weight under the i-th channel image, δ j represents the foreground and background features of the fusion image, represents the positive sample of the source image, represents the m-th negative sample of the source image.

[0034] Preferably, the method for combining the features of the source image to provide the optimal fusion weight and obtaining the image with rich information, sufficient details and prominent target is:

[0035] Step S330: learning to minimize the structural similarity of the source image and the fusion image, and the structural similarity is constrained by SSIM, and the loss function is defined as:

[0036] L SSIM = w a S(I V , I F ) + w b S(I R , I F )

[0037] Wherein, S represents the calculation process of SSIM, w a , w b , respectively, represent the fusion proportion coefficient of the source image;

[0038] Step S331: learning to minimize the intensity similarity of the source image and the fusion image, and the intensity similarity is constrained by MSE, and the loss function is defined as:

[0039] L N = w c ·||I V -I F ||+ w d ·||I R -I F ||

[0040] Wherein, w c , w d respectively represent the fusion proportion coefficient of the source image.

[0041] Preferably, in the method for fusing the synthesis learning stage and the reverse optimization stage to form the multi-channel image fusion framework, the method is characterized in that:

[0042] Step S300: through the joint learning of the synthesis learning stage and the reverse optimization stage, minimizing the joint loss in the form of weighted sum, and optimizing the performance of the multi-channel image fusion model;

[0043] Step S301: The synthetic learning stage and the reverse optimization stage are fused to form a multi-channel image fusion framework.

[0044] The application also provides a multi-channel image fusion system based on coupled learning under ultra-low illumination, comprising:

[0045] The data processing module is configured to perform data preprocessing according to a preset sample number of micro-light image data sets.

[0046] The multi-level attention module is configured to extract luminosity feature information according to the micro-light image data sets, and comprises: a multi-level attention model for extracting illumination perception information is constructed, and then a luminosity probability of an output image of a luminosity information extraction network is constructed, an auxiliary module for extracting common important features of the same target is constructed, and then a method of convolutional dimension reduction is used to retain the optimal features selected.

[0047] The fusion framework module is configured to train a multi-channel image fusion framework according to pictures contained in the micro-light image data sets, and comprises: a synthetic learning stage and a reverse optimization stage, and then the synthetic learning stage and the reverse optimization stage are fused to form a multi-channel image fusion framework.

[0048] The image fusion module is configured to input the pictures after data preprocessing into the multi-channel image fusion framework, and output fused images.

[0049] The application also provides a computer medium, wherein the computer medium stores a computer program, and the computer program is executed by a processor to realize the multi-channel image fusion method based on coupled learning under ultra-low illumination.

[0050] The application also provides a computer comprising the computer medium.

[0051] The above technical solutions of the application have the following advantages compared with the prior art:

[0052] 1. The multi-level attention model for extracting illumination perception information is introduced, which effectively retains the luminosity information and important features of each channel image in the training process, exhibits the superiority of the utilization of the light perception of the target image, and reduces the phenomenon of missing effective information and redundant invalid information of the network in the learning process.

[0053] 2. The multi-channel image fusion method based on coupled learning is proposed, which fully retains the high-quality information of each channel image, excludes miscellaneous information, and improves the fusion performance of the model. DETAILED DESCRIPTION

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1 The diagram illustrates the process of a multi-channel image fusion method.

[0056] Figure 2 The diagram shows the module connections of a multi-channel image fusion system.

[0057] Figure 3 The flowchart shown is for a multi-channel image fusion method.

[0058] Figure 4 The diagram shows a flowchart of a data preprocessing method.

[0059] Figure 5 The flowchart shown is a semi-manual feature matching method for multi-path image registration.

[0060] Figure 6 The flowchart shown is a method for extracting photometric feature information.

[0061] Figure 7 The diagram shows a flowchart of the synthetic learning phase method.

[0062] Figure 8 The diagram shown is a reference to the synthetic learning phase process.

[0063] Figure 9 The flowchart shown is for the reverse optimization phase method.

[0064] Figure 10 The diagram shows the flowchart of the coupled learning part of the inverse optimization phase.

[0065] Figure 11 The flowchart shown is a part of the adaptive weight learning method in the inverse optimization stage.

[0066] Figure 12 The flowchart shown is a method for integrating the synthetic learning phase and the inverse optimization phase.

[0067] Explanation of reference numerals in the attached figures:

[0068] 1. Data processing module; 2. Multi-level attention module; 3. Fusion framework module; 4. Image fusion module. Detailed Implementation

[0069] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0070] refer to Figure 1 , Figures 2-12 As shown, for a camera with multiple channels: visible light monochrome channel, EBCMOS channel, and infrared channel, and a real-time low-light image dataset D = {(V i E i I i The specific implementation method is as follows:

[0071] Step S1: Perform data preprocessing based on a real-time low-light image dataset with N samples.

[0072] Among them, reference Figure 4 As shown, the preprocessing process is as follows:

[0073] Because the images captured by different cameras are of different sizes and have not yet been registered and aligned, therefore:

[0074] Step S10: Use a semi-manual feature matching method to register the multi-path images and output a registered image with the same size as the calibration camera;

[0075] Step S11: Crop the common area of ​​the registered image, keeping only the target object;

[0076] Step S12: Images with extremely low contrast are enhanced using the LPDM algorithm.

[0077] Among them, reference Figure 5 As shown, the process of semi-manual feature matching multi-path image registration is as follows:

[0078] Step S100: We manually select 4 sets of corresponding points with obvious features and prominent targets among a set of images from different pathways by manually controlling the point selection. In MATLAB, we calculate the homography matrix using the least squares method and find the transformation relationship.

[0079] Among them, the manual control point selection method is used to select a set of corresponding points with obvious features and prominent targets between different channel images, denoted as (x i y i ) and (x′ i y′ i ), where i represents the control point number;

[0080] The homography matrix H is calculated using the least squares method, and the coordinates of corresponding points are expressed as homogeneous coordinates, where x and x′ represent the coordinates of the input and output images, respectively.

[0081]

[0082] Normalized homogeneous coordinates:

[0083] Step S101: For any set of images in this scene, use the encoding method and the homography matrix obtained in step S100 to achieve automated image registration.

[0084] The automated image registration is achieved based on an encoding method: For any set of images, images I and I′ are represented as matrices of size m×n and m′×n′, respectively; assuming I is the input image and I′ is the output image:

[0085]

[0086]

[0087] , where k represents the number of control points selected;

[0088] An affine transformation is performed on image I′, and pixel mapping is performed based on the homography matrix H and the homogeneous coordinates of the image to obtain the registered image I′. registered :

[0089]

[0090] , where h ij Elements representing the homography matrix H;

[0091] For example: Suppose we have two images I and I′ of the same size, and their corresponding control points (x). i y i ) and (x′ i y′ i We manually selected two control points (10, 10) and (100, 100), and calculated the homography matrix H using the least squares method.

[0092]

[0093] Represent images I and I′ as matrices respectively:

[0094] I=[I(10,10),I(100,100)]

[0095] I′=[I′(10,10),I′(100,100)]

[0096] The image I′ is obtained by performing an affine transformation on the homography matrix H. registered :

[0097]

[0098] Finally, it is compared with the original image I to observe whether the image has achieved the correct registration effect, and then the common target area is cropped.

[0099] Step S2: Based on the low-light image dataset, perform photometric feature extraction, including: constructing a multi-level attention model for light perception information extraction, then constructing the photometric probability of the output image of the photometric information extraction network, constructing an auxiliary module for extracting common important features of the same target, and then using convolutional dimensionality reduction to retain the selected optimal features.

[0100] In existing fusion methods, researchers only consider the fusion effect on the target object, ignoring the significant visual impact of illumination information in each image. Furthermore, in deep learning, as the network depth increases, effective information in the images is lost, especially illumination information. Therefore, a multi-level attention model for extracting illumination-aware information is constructed, which can preserve the effective photometric features of each path image globally. The existence of photometric features is essentially for selecting the base image for fusion. In daytime scenes, the light source comes from the visible light image; in nighttime scenes, the light source is preserved in the infrared image. Compared to existing methods that all use visible light images as the base image for fusion, the loss of illumination information remains unresolved. Illumination awareness, however, can adaptively preserve the optimal photometric information, providing strong support for the fusion process.

[0101] refer to Figure 6 As shown, the photometric feature information extraction process is as follows:

[0102] Step S20: Construct a multi-level attention model for extracting illumination perception information, and then train the multi-level attention model using a low-light image dataset, so that the multi-level attention model can benefit from the knowledge transfer of image fusion;

[0103] Step S21: Construct a photometric information extraction network using the ResNet-50 network structure to output the photometric probability of the corresponding image, so as to determine the current fusion base image using the trained multi-level attention model for the image to be fused and the fusion process image;

[0104] Step S22: Based on the photometric information extraction network, construct an auxiliary module to extract common important features of the same target in each channel image, and then use convolutional dimensionality reduction to retain the selected optimal features, which facilitates subsequent fusion.

[0105] Step S3: Based on the images contained in the low-light image dataset, train a multi-path image fusion framework, including a synthesis learning stage and an inverse optimization stage, and then fuse the synthesis learning stage and the inverse optimization stage to form a multi-path image fusion framework.

[0106] Before training the input neural network, a target mask is generated for each set of images to facilitate coupled learning. The multi-path image fusion framework uses the VGG-19 network and is divided into two stages: the synthesis learning stage and the inverse optimization stage. In addition, to reduce the inconsistency between input and output, the images are subjected to ReLU activation and upsampling operations as the final output of the network.

[0107] Phase 1: As Figures 7-12 As shown, the general process of the synthesis learning phase is as follows:

[0108] First, given two registered and aligned images, the images are fed into the network;

[0109] Then, through the multi-level attention blocks proposed in the second step, a fused image is generated.

[0110] The second stage, the inverse optimization stage, is divided into two parts: the coupled learning part and the adaptive weight learning module;

[0111] The coupling learning part constrains the fused image by learning the foreground / background features in the source image, while the adaptive weight learning part changes the fixed weight allocation method of the existing fusion method so that the network finds the most suitable fusion weights during the training process.

[0112] First, the coupling learning part is mainly used to learn specific foreground / background information from the source image. For EBCMOS images, the obvious target outline in the foreground is the part we are interested in, while the information in the rest is redundant. For visible light black and white images and infrared images, we tend to retain the vivid texture details in the background, while the dark foreground is undesirable.

[0113] Specifically, we generate a foreground background mask for the EBCMOS image and overlay it onto the infrared image and the visible light monochrome image, respectively; we generate a background mask for the infrared image and the visible light monochrome image and overlay it onto the EBCMOS image, respectively. We set positive and negative samples, evaluate the similarity of features, and set the score of the similarity between the positive and negative samples of each path image as the loss function of that path image:

[0114]

[0115] Where N represents the number of layers in the VGG-19 network, M represents the number of negative samples set in each path image, and V, E, and I represent visible light monochrome image, EBCMOS image, and infrared image, respectively. i δ represents the weight of the image under the i-th path. j Represents the foreground / background features of the fused image. Represents positive samples of the source image. This represents the m-th negative sample of the source image.

[0116] Second, the goal of the adaptive weight learning part is to combine the advantageous features of the source image to provide optimal fusion weights, resulting in an image that is information-rich, detailed, and highlights the target. For the fusion task between multi-path images, the learning process minimizes the structural similarity and intensity similarity between the source image and the fused image. Structural similarity is constrained by SSIM, and intensity similarity is constrained by MSE. The loss functions are defined as follows:

[0117] L SSIM =w a S(I V I F )+w b S(I R I F )

[0118] L N =w c ·||I V -I F ||+w d ·||I R -I F ||

[0119] Where S represents the SSIM calculation process, w a w b w c w d These represent the fusion ratio coefficients of the source images. In existing fusion methods, they are often conventionally set to fixed values. However, fixed values ​​cannot fully utilize the characteristics of the source images and will cause loss of detail. Therefore, in the above L... SSIM L N An adaptive weight is set in the loss function to better balance the parts of the source image involved in the fusion.

[0120] Finally, by integrating the synthesis learning stage and the inverse optimization stage, an inverse optimization multi-path image fusion framework is proposed. The learning objective of the multi-path image fusion framework is to minimize the joint loss and optimize the multi-path image fusion performance of the model by leveraging the joint learning of the synthesis learning stage and the inverse optimization stage. The joint loss is a weighted sum of the losses of the learning modules.

[0121] Step S4: Input the image after data preprocessing into the multi-path image fusion framework and output the fused image.

[0122] The fourth step involves end-to-end multi-path image fusion, with the following process: input the preprocessed image into the trained network, and the network outputs the fused image.

[0123] One aspect of this invention also provides a multi-channel image fusion system based on coupling learning under ultra-low illumination, using the aforementioned multi-channel image fusion method based on coupling learning under ultra-low illumination, with reference to... Figure 2 As shown, it includes:

[0124] The data processing module is used to preprocess data based on a dataset of low-light images with a preset sample size.

[0125] The multi-level attention module is used to extract photometric feature information based on the low-light image dataset. This includes: constructing a multi-level attention model for light perception information extraction, then constructing the photometric probability of the output image of the photometric information extraction network, constructing an auxiliary module for extracting common important features of the same target, and then using convolutional dimensionality reduction to retain the selected optimal features.

[0126] The fusion framework module is used to train a multi-path image fusion framework based on the images contained in the low-light image dataset. It includes a synthesis learning stage and an inverse optimization stage, and then merges the synthesis learning stage and the inverse optimization stage to form a multi-path image fusion framework.

[0127] The image fusion module is used to input the preprocessed images into the multi-path image fusion framework and output the fused image.

[0128] In one aspect, the present invention also provides a computer medium storing a computer program, which is executed by a processor to implement the aforementioned multi-channel image fusion method based on coupling learning under ultra-low illumination.

[0129] The present invention also provides a computer, including the aforementioned computer medium.

[0130] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A multi-channel image fusion method based on coupled learning under ultra-low illumination, characterized by, The method comprises the following steps: Step S1: data preprocessing according to a preset number of micro-light image data sets; Step S2: photometric feature information extraction according to the micro-light image data set, comprising: constructing a multi-level attention model for light perception information extraction, then constructing a photometric information extraction network to output the photometric probability of the image, constructing an auxiliary module for extracting common important features of the same target, and then using a convolution dimension reduction method to retain the selected optimal features; Step S3: training a multi-channel image fusion framework according to the pictures contained in the micro-light image data set, comprising: a synthetic learning stage and a reverse optimization stage, and then fusing the synthetic learning stage and the reverse optimization stage to form a multi-channel image fusion framework; The method of the synthetic learning stage comprises: Step S30: a preset number of registration aligned pictures are given, and the pictures are imported into the multi-channel image fusion framework; Step S31: the pictures are fused by the multi-channel image fusion framework to generate a fused image; The method of the reverse optimization stage comprises: Step S32: the fused image is constrained by learning the foreground and background features in the source image; Step S33: combining the features of the source image, the optimal fusion weight is provided to obtain a fused image with rich information, sufficient details and prominent target; Step S4: inputting the pictures after data preprocessing into the multi-channel image fusion framework to output the fused image.

2. The multi-channel image fusion method based on coupled learning under ultra-low illumination according to claim 1, characterized in that, The method of data preprocessing comprises: Step S10: using a semi-automatic feature matching based method to register the multi-channel images, and outputting registration pictures with the same size as the calibrated camera; Step S11: cropping the common area of the registration pictures to retain the preset target; Step S12: using an LPDM method to enhance the image when the image is lower than the preset illumination value.

3. The multi-channel image fusion method based on coupled learning under ultra-low illumination according to claim 2, characterized in that, The method of using a semi-automatic feature matching based method to register the multi-channel images comprises: Step S100: using a manual control point selection method to combine the images of different channels with the calibrated camera images, manually selecting a group of corresponding points with obvious features and prominent targets, and then calculating the corresponding homography matrix by the least square method; Step S101: for any group of images, using an encoding method to realize automatic image registration under the same experimental environment by using the homography matrix.

4. The multi-channel image fusion method based on coupled learning under ultra-low illumination according to claim 1, characterized in that, The method of photometric feature information extraction comprises: Step S20: constructing a multi-level attention model for light perception information extraction, and using a pre-trained model on an ImageNet data set as a basis, and then training the multi-level attention model by using a micro-light image data set; Step S21: using a ResNet-50 network structure to construct a photometric information extraction network to output the photometric probability of the corresponding image, so as to determine the current fusion basis image by using the trained multi-level attention model for the image to be fused and the fusion process image; Step S22: generating photometric information extracted from each channel image according to the photometric information extraction network, constructing an auxiliary module for extracting common important features of the same target, and then using a convolution dimension reduction method to retain the selected optimal features.

5. The multi-channel image fusion method based on coupled learning under ultra-low illumination according to claim 1, characterized in that, The method for constraining the fusion image by learning the foreground / background features in the source image is: Step S320: A mask of the pre-positioned background is generated for the EBCMOS image, and is superimposed with the infrared image and the visible light black and white image respectively; Step S321: A mask of the post-positioned background is generated for the infrared image and the visible light black and white image, and is superimposed with the EBCMOS image respectively, then positive and negative samples are set, the similarity of the features is evaluated, and the score of the similarity of the positive and negative samples of each channel image is set as the loss function of the channel image: Wherein, N represents the number of layers of VGG-19 network, M represents the number of negative samples set in each kind of channel image, V, E, I represent visible light black and white image, EBCMOS image, infrared image respectively, wi represents the weight under the i-th channel image, δj represents the foreground and background features of the fusion image, represents the positive sample of the source image, δjm represents the m-th negative sample of the source image.

6. The multi-channel image fusion method based on coupled learning under ultra-low illumination according to claim 1, characterized in that, The method for providing optimal fusion weights by combining the features of the source image to obtain an image rich in information, sufficient in details and prominent in targets is: Step S330: The structural similarity between the source image and the fusion image is learned to be minimized, and the structural similarity is constrained by SSIM, and the loss function is defined as: Wherein, S represents the calculation process of SSIM, wa , wb , respectively represent the fusion proportion coefficient of the source image; Step S331: The intensity similarity between the source image and the fusion image is learned to be minimized, and the intensity similarity is constrained by MSE, and the loss function is defined as: wherein, wc , wd respectively represent the fusion proportionality coefficients of the source images.

7. The multi-channel image fusion method based on coupled learning under ultra-low illumination according to claim 5 or 6, characterized in that, The method for fusing the synthesis learning stage and the reverse optimization stage to form a multi-channel image fusion framework is: Step S300: Through joint learning of the synthesis learning stage and the reverse optimization stage, the performance of the multi-channel image fusion framework is optimized by minimizing the joint loss in the form of weighted sum; Step S301: The synthesis learning stage and the reverse optimization stage are fused to form a multi-channel image fusion framework.

8. A multi-channel image fusion system based on coupled learning under ultra-low illumination, characterized in that, Comprise: A data processing module configured to perform data preprocessing on a preset sample number of micro-light image dataset; A multi-level attention module configured to extract photometric feature information from the micro-light image dataset, comprising: constructing a multi-level attention model for extracting light perception information, then constructing a photometric probability of a network output image for extracting photometric information, constructing an auxiliary module for extracting common important features of the same target, and then using a convolution dimension reduction method to retain the optimal features selected; A fusion framework module configured to train a multi-channel image fusion framework according to the pictures contained in the micro-light image dataset, comprising: a synthesis learning stage and a reverse optimization stage, and then fusing the synthesis learning stage and the reverse optimization stage to form a multi-channel image fusion framework; the method of the synthesis learning stage comprises: given a preset number of registration aligned pictures, and importing the pictures into the multi-channel image fusion framework; the pictures are fused by the multi-channel image fusion framework to generate a fusion image; the method of the reverse optimization stage comprises: constraining the fusion image by learning the foreground / background features in the source image; combining the features of the source image to provide optimal fusion weights to obtain a fusion image rich in information, sufficient in details and prominent in targets; An image fusion module configured to input the pictures after data preprocessing into the multi-channel image fusion framework to output the fused images.

Images