A multi-exposure image fusion method based on a multi-scale autoencoder

Abstract

The application discloses a multi-exposure image fusion method based on a multi-scale self-encoder, which comprises the following steps: 1. data preparation, data preprocessing and construction of a multi-scale self-encoder network, mainly including a multi-scale encoder and a decoder, wherein the encoder comprises a convolution and an activation function, a visual converter, and mainly performs multi-scale feature extraction; the encoder is a multi-scale fusion dense cross-connection convolution and an activation function, and mainly performs image reconstruction; 2. fusion is performed on input multi-exposure image pairs, including network training and multi-exposure image fusion. The application can fully utilize complementary and redundant information in overexposed images and underexposed images to fuse high-quality images with better quality, provide images with better quality for human eye observation, and provide support for computer vision tasks such as image segmentation and classification, thereby assisting human eye recognition and computer analysis and processing researches.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-exposure image fusion, and in particular to a multi-exposure image fusion method based on a multi-scale autoencoder. BACKGROUND

[0002] The brightness in natural scenes is usually quite different, and due to the limitations of imaging devices, the dynamic range of a single image is much lower than that of a natural scene. The scene being photographed can be affected by light, weather, solar altitude and other factors, and overexposure and underexposure often occur. A single image cannot fully reflect the light and shade of the scene, and some information can be lost, resulting in unsatisfactory imaging. It is still challenging to solve the problem of incomplete dynamic range matching in existing imaging devices, display monitors and the dynamic response of the human eye to real natural scenes. Multi-exposure image fusion (MEF) technology provides a simple, economical and efficient way to overcome the contradiction between HDR imaging and low dynamic range (LDR) display. It avoids the complexity of imaging hardware circuit design and reduces the weight and power consumption of the entire device, and it improves image quality and has important applications in the field of digital photography. MEF is a process of fusing multiple images with different exposures to produce a single visually pleasing and high-quality fused image. Specifically, MEF is a process of fusing multiple images with different exposures to produce a single visually pleasing and high-quality fused image. MEF is similar to other image fusion tasks, such as medical image fusion and remote sensing image fusion, as they all combine important information from multiple source images to produce a high-quality fused image. The main difference between these image fusion tasks is the source images, which contain different information to be fused. In contrast, the source images of MEF are images with different exposures. MEF has attracted widespread attention due to its effectiveness in promoting high-quality images.

[0003] Multi-exposure image fusion, as an important branch of image fusion, its main task is to fuse multiple images with different exposure levels under the same scene to obtain a high dynamic range high-quality image. The existing methods have the following problems: first, the traditional method-based method has poor robustness due to the artificially designed feature extractor and rules, and the effect is poor under different scenes, which may appear uneven light and dark and artifacts; the second is the deep learning-based method, which relies heavily on multi-exposure dataset training, while the multi-exposure dataset is relatively small compared to the natural dataset, which cannot fully train the network with more layers and more parameters; previous methods are mostly based on convolution, which lacks global information; and there is a lack of multi-scale feature fusion and interaction.

[0004] By using multi-exposure image fusion, all the important feature information can be obtained from a single image, facilitating human perception and subsequent image processing, such as object detection, object segmentation, and edge extraction. Therefore, the implementation of multi-exposure image fusion technology is of great significance. Summary of the Invention

[0005] To overcome the problems existing in the multi-exposure image fusion of current image fusion methods, this invention provides a multi-exposure image fusion method based on a multi-scale autoencoder. The aim is to fully utilize the complementary and redundant information of images with different exposure levels to provide better image feature representation and reconstruct higher quality images, thereby providing better quality images for human visual observation and supporting computer vision tasks such as image segmentation and classification.

[0006] To solve the above problems, the present invention adopts the following technical solution:

[0007] The present invention provides a multi-exposure image fusion method based on a multi-scale autoencoder, characterized by the following steps:

[0008] Step 1: Obtain P RGB natural images and convert them to grayscale images, denoted as {I1, I2, ..., I... p ,…,I P} and used as the training set, where I p This represents the p-th grayscale image;

[0009] Step 2: Construct a multi-scale autoencoder network, including a multi-scale encoder and a decoder;

[0010] Step 2.1: The multi-scale encoder includes W convolutional blocks A1, A2, ..., A w ,…,A W X convolutional blocks N1, N2, ..., N x ,…,N X And Y visual converters Trans1, Trans2, ..., Trans y ,…,Trans Y , where A w Let A represent the w-th convolutional block, and let A represent the w-th convolutional block. w Includes a convolutional layer with an A×A kernel and a ReLU activation function; N x This represents the x-th convolutional block; and the x-th convolutional block N x Includes a convolutional layer with an N×N kernel and a ReLU activation function; Trans y This represents the y-th visual converter; Y = W - 2;

[0011] Step 2.1.1: The p-th grayscale image I pThe input is input into the multi-scale encoder, and after being sequentially processed by the first convolutional block A1 and the first convolutional block N1, a primary shallow feature map is obtained After being sequentially processed by Y visual converters Trans1, Trans2, …, Trans y ,…,Trans Y , Y primary deep feature maps are obtained , wherein represents the yth primary deep feature map;

[0012] Step 2.1.2: input into the second convolutional block A2 for processing to obtain a shallow feature map with a channel number of C

[0013] input into W-2 convolutional blocks A3, …, A w ,…,A W for processing, respectively, to obtain deep feature maps with a channel number of C , wherein represents the w-2th deep feature map; W-1 comprehensive feature maps are composed of the shallow feature map and the deep feature map

[0014] Step 2.1.3: after the W-1th comprehensive feature map is processed by the Xth convolutional block N X , the X-1th multi-scale feature map is obtained

[0015] Step 2.1.4: after the W-1th comprehensive feature map is up-sampled, the W-2th up-sampled feature is obtained After the up-sampled feature is subjected to an element-wise addition operation with the W-2th comprehensive feature map , the W-2th intermediate feature map is obtained After the W-2th intermediate feature map is processed by the X-1th convolutional block N X-1 , the X-2th feature is obtained

[0016] Step 2.1.5: after is up-sampled, the W-3th up-sampled feature is obtained After the W-3th up-sampled feature is subjected to an element-wise addition operation, the W-3th intermediate feature map is obtained The (W-3)th intermediate feature map After the (X-2)th convolutional block N X-2 The (X-3)th feature is obtained

[0017] Step 2.1.6: sequentially process the X-1th multi-scale feature maps obtained in step 2.1.5 according to the process of step 2.1.5 to obtain X-1th multi-scale feature maps The X-1th multi-scale feature map is represented as

[0018] Step 2.2: the decoder is composed of P convolutional blocks and an output block Conv, wherein the P convolutional blocks are upper triangular densely connected, and are sequentially represented as wherein Decoder (i,j) represents the convolutional block in the ith column and the jth row, and the convolutional block Decoder (i,j) includes a convolutional layer with a convolution kernel of N×N and a ReLU activation function; I=J=X-2,

[0019] The output block Conv includes a convolutional layer with a convolution kernel of A×A and a ReLU activation function;

[0020] Step 2.2.1: sequentially process the X-1th multi-scale feature maps output by the encoder corresponding to the rows of the decoder are represented as 1, 2, …, j, …, J; then are sequentially represented as wherein, represents the multi-scale feature output by the jth decoder;

[0021] Step 2.2.2: the input of the decoder Decoder (1,j) in the first column and the jth row is and the up-sampling feature map The output of the decoder Decoder (1,j) in the first column and the jth row is the feature I (1,j) ;

[0022] Step 2.2.3: the input of the convolutional block Decoder (i,j) in the jth row in the remaining columns except the first column is the jth multi-scale feature map the up-sampling feature map of the feature I (i-1,j+1) output by the decoder in the i-1th column of the j+1th row and the feature map I (i-1,j) … I (1,j) spliced by the decoders in the jth row and the i-1th column, the jth row and the i-2th column to the jth row and the first column; the convolutional block Decoder(i,j) The output of the output block Conv is a feature map I (i,j) , thereby obtaining The feature I (I,1) is obtained after the P convolutional blocks of the decoder (I,1) After the processing of the output block Conv, the output result O p is obtained.

[0023] Step 3: Construct the total loss function L of the multi-scale autoencoder network using formula (1):

[0024] L = L ssim + λL pixel (1)

[0025] In formula (1), λ represents a weight coefficient of the pixel loss, L ssim represents a structural similarity loss function and is obtained from formula (2), and L pixel represents a pixel loss function and is obtained from formula (3).

[0026] L ssim = 1-SSIM(I p ,O p ) (2)

[0027]

[0028] In formula (2), SSIM represents structural similarity.

[0029] Step 4: Based on the training set, the multi-scale autoencoder network is trained using a back propagation algorithm, and the total loss function L is calculated to adjust the network parameters until a maximum iteration number is reached, thereby obtaining the trained multi-scale autoencoder network.

[0030] Step 5: Obtain B pairs of multi-exposure images and convert to Ycbcr color domain to obtain an image pair in which only the Y channel is retained, thereby obtaining the preprocessed B pairs of multi-exposure images {(I o1 ,I u1 ), (I o2 ,I u2 ), …, (I ob ,I ub ), …, (I oB ,I uB )}, wherein (I ob ,I ub ) represents the bth pair of multi-exposure images, I ob represents the bth Y channel overexposure image, and I ub represents the bth Y channel underexposure image.

[0031] Step 6: Input the multi-exposure image pair {(Io1 ,I u1 ),(I o2 ,I u2 ),…,(I ob ,I ub ),…,(I oB ,I uB} are input into the trained multi-scale encoder respectively for processing to obtain S scales of overexposed image features {Io f1 , Io f2 , …, Io fs , …, Io fS} and underexposed image features {Iu f1 , Iu f2 , …, Iu fs , …, Iu fS}, wherein Io fs represents the s-th overexposed image feature, and Iu fs represents the s-th underexposed image feature.

[0032] The s-th overexposed image feature Io fs and the s-th underexposed image feature Iu fs are added and averaged to obtain the s-th fusion feature f s , so as to obtain a fusion feature set {f1, f2, …, f s , …, f S} and input into the trained decoder, so as to obtain a fusion result {Output1, Output2, …, Output b , …, Output B}, wherein Output b represents the fusion result of the b-th Y channel overexposed image I ob and the b-th Y channel underexposed image I ub .

[0033] {Output1, Output2, …, Output b , …, Output B} is converted from the Ycbcr domain to the RGB domain, and finally an exposure-uniform color image {Result1, Result2, …, Result b , …, Result B} is obtained, wherein Result b represents the b-th color image result.

[0034] The electronic device comprises a memory and a processor, and is characterized in that the memory is used for storing a program supporting the processor to execute the multi-exposure image fusion method, and the processor is configured to execute the program stored in the memory.

[0035] The computer readable storage medium stores a computer program, and the computer program is characterized in that when the computer program is run by a processor, the steps of the multi-exposure image fusion method are executed.

[0036] Compared with the prior art, the beneficial effects of the present application are embodied in that:

[0037] 1. The present application proposes a unified network framework to realize the fusion tasks of overexposed images and underexposed images, fully utilizes the redundancy and complementary information between different modal images, and fuses high-quality images. Compared with the existing method which needs to train the network on multi-exposure images, the present application only needs to train on normal natural image data sets (such as 2014MS-COCO data set) to realize the high-quality fusion of multi-exposure images, thereby avoiding the dependence on multi-exposure data sets and the training of more layers of parameters.

[0038] 2. The present application designs a multi-scale feature combining CNN and transformer from top to bottom and from bottom to top, effectively extracts local and global features, pyramid multi-scale extracts better features of pictures with multi-scale changes, and better enables each feature of different scales to have strong semantic information, and comprehensively integrates the details of the bottom layer and the semantic information of the high layer, thereby bringing better detail performance to the fusion result.

[0039] 3. The present application designs a decoder composed of upper triangular dense connection and up-sampling, which can effectively fuse multi-scale features, fully utilize deep features, and retain more information of different scales extracted from the encoder network to prevent the network from losing shallow features while extracting deeper features, so that the feature information extracted by the network is more comprehensive, and then the multi-scale features obtained by the decoder can be fully utilized to strengthen the image quality of fusion. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The flow chart of the multi-exposure image fusion method based on multi-scale autoencoder of the present application;

[0041] Figure 2 The network structure diagram of the present application;

[0042] Figure 3 The fusion structure diagram of the present application;

[0043] Figure 4This is a schematic diagram of the encoder of the present invention;

[0044] Figure 5 This is a schematic diagram of the decoder structure of the present invention. Detailed Implementation

[0045] In this embodiment, the general flow of a multi-exposure image fusion method based on a multi-scale autoencoder is as follows: Figure 1 As shown, it includes the following steps:

[0046] Step 1: After converting P RGB natural images to grayscale images, denote them as {I1, I2, ..., I...} p ,…,I P}, where I p This represents the p-th grayscale image.

[0047] Step 2: Build using, for example Figure 2 The multi-scale autoencoder network shown includes a multi-scale encoder and a decoder;

[0048] Step 2.1: The multi-scale encoder consists of W convolutional blocks A1, A2, ..., A w ,…,A W X convolutional blocks N1, N2, ..., N x ,…,N X And Y visual converters Trans1, Trans2, ..., Trans y ,…,Trans Y , where A w Let A represent the w-th convolutional block, and let A be the w-th convolutional block. w Includes a convolutional layer with an A×A kernel and a ReLU activation function; N x This represents the x-th convolutional block; and the x-th convolutional block N x Includes a convolutional layer with an N×N kernel and a ReLU activation function; Trans y This represents the y-th visual converter; Y = W - 2; in this embodiment, as... Figure 4 As shown, W=5, X=5, Y=3.

[0049] Step 2.1.1: The p-th grayscale image I p The input is processed by the first convolutional block A1 and the first convolutional block N1 in sequence in the multi-scale encoder to obtain the primary shallow feature map. Then it passes through Y visual transformers in sequence: Trans1, Trans2, ..., Trans... y ,…,Trans Y After processing, Y primary deep feature maps are obtained. in, represents the y-th primary deep feature map.

[0050] Step 2.1.2: the is input into the 2nd convolutional block A2 for processing, to obtain a shallow feature map with a channel number of C

[0051] is input into the 2nd convolutional block A2 for processing, to obtain a shallow feature map with a channel number of C is sequentially input into the W-2 convolutional blocks A3,..., A w ,..., A W , to obtain a deep feature map with a channel number of C wherein, represents the w-2-th deep feature map; the shallow feature map and the deep feature map are combined to obtain W-1 comprehensive feature maps

[0052] Step 2.1.3: the W-1 comprehensive feature maps are input into the Xth convolutional block N X , to obtain the X-1 multi-scale feature maps

[0053] Step 2.1.4: the W-1 comprehensive feature maps are up-sampled to obtain the W-2 up-sampled features The up-sampled features are subjected to an element-wise addition operation with the W-2 comprehensive feature maps , to obtain the W-2 intermediate feature maps The W-2 intermediate feature maps are input into the X-1 convolutional block N X-1 , to obtain the X-2 features

[0054] Step 2.1.5: the are up-sampled to obtain the W-3 up-sampled features The up-sampled features are subjected to an element-wise addition operation with the W-3 comprehensive features , to obtain the W-3 intermediate feature maps The W-3 intermediate feature maps are input into the X-2 convolutional block N X-2 , to obtain the X-3 features

[0055] Step 2.1.6: the are sequentially processed according to the process of Step 2.1.5, to obtain the X-1 multi-scale feature maps This represents the (x-1)th multi-scale feature map.

[0056] like Figure 4 The encoder shown has input I. p An image of size 256×256×1 is processed by A1 to output a feature map of size 236×256×16, and then processed by N1 to output a primary shallow feature map of size 256×256×32, denoted as . After passing through Trans1, Trans2, and Trans3 sequentially, the primary deep feature maps are 128×128×64, 64×64×128, and 32×32×256, respectively, denoted as […]. The visual converter is a standard visual converter. Next... The shallow feature map 256×256×128 was obtained by passing A2, and is denoted as... Primary deep feature map Deep feature maps of 128×128×128, 64×64×128, and 32×32×128 were obtained through convolutional blocks A3, A4, and A5, respectively denoted as... Shallow feature map and deep feature map Composed of 4 comprehensive feature maps

[0057] 4th comprehensive feature map After processing by the 5th convolutional block N5, the 4th multi-scale feature map, 32×32×256, is obtained, denoted as...

[0058] For feature maps Upsampling yields upsampled feature maps upsampled feature map Then, combine with the third comprehensive feature map After performing element-wise addition, the intermediate feature map is obtained as 64×64×128, denoted as intermediate feature map After the fourth convolution N4, the third multi-scale feature is obtained, which is 64×64×128, denoted as...

[0059] For intermediate feature maps Upsampling yields upsampled feature maps upsampled feature map Then, combine with the second comprehensive feature map After performing element-wise addition, an intermediate feature map of 128×128×128 is obtained, denoted as... intermediate feature map After the third convolution N3, the second multi-scale feature is obtained, which is 128×128×64, denoted as...

[0060] For intermediate feature maps Upsampling yields upsampled feature maps upsampled feature map Then, combine with the first comprehensive feature map After performing element-wise addition, the intermediate feature map is obtained as 256×256×128, denoted as intermediate feature map After the third convolution N², the first multi-scale feature is obtained, which is 256×256×32, denoted as... This yielded four multi-scale features.

[0061] Step 2.2: As Figure 5 As shown, the decoder consists of P convolutional blocks and one output block Conv. The P convolutional blocks are densely connected in an upper triangular configuration and are denoted as follows: Among them, Decoder (i,j) Let represent the convolutional block in the i-th column and j-th row, and let represent the convolutional block Decoder in the i-th column and j-th row. (i,j) It includes a convolutional layer with an N×N kernel and a ReLU activation function; I = J = X⁻².

[0062] The output block Conv consists of a convolutional layer with an A×A kernel and a ReLU activation function.

[0063] Step 2.2.1: Convert the X-1 multi-scale features output by the encoder The corresponding decoder row indices are denoted as 1, 2, ..., j, ..., J; then... Recorded in order as in, This represents the multi-scale feature of the j-th output.

[0064] In the specific design, In the decoder's description, it is referred to as To facilitate alignment with the rows.

[0065] Step 2.2.2: Decoder in column 1, row j (1,j) The input is and its upsampled feature map Decoder in column 1, row j (1,j) The output is feature I (1,j) ;

[0066] Upsampling twice and After merging, input Decoder (1,3) Decoder13 Feature map I is obtained (1,3) ,at the same time, Upsampling twice and After merging, input Decoder (1,2) Feature map I is obtained (1,2) ; Upsampling twice and After merging, input Decoder (1,1) Feature map I is obtained (1,1) .

[0067] Step 2.2.3: Decoder for the convolutional block in row j of all columns except the first column. (i,j) The input is the feature map of the j-th scale. The decoder output feature I in row (j+1) and column (i-1) (i-1,j+1) upsampled feature map And the feature map I output by the decoder from row j, column i-1, row j, column i-2 to row j, column 1. (i-1,j) …I (1,j) The concatenated feature map; Decoder (i,j) The output is feature map I (i,j) Therefore, from After passing through P convolutional blocks in the decoder, feature I is obtained. (I,1) Feature I (I,1) After further processing by the output block Conv, the output result O is obtained. p .

[0068] In this embodiment, feature map I (1,3) Upsampling twice and and I (1,2) After concatenating the vectors, input them into the Decoder. (2,2) Feature map I is obtained (2,2) ;

[0069] Feature Map I (1,2) Upsampling twice and and I (1,1) After concatenating the vectors, input them into the Decoder. (2,1) Feature map I is obtained (2,1) ;

[0070] Feature Map I (2,2) Upsampling twice and I (1,1) and I (2,1) After concatenating the vectors, input them into the Decoder. (3,1) Feature map I is obtained (3,1) ;

[0071] Decoder (1,1), Decoder (1,2) , Decoder (1,3) , Decoder (2,1) , Decoder (2,2) , Decoder (3,1) The input and output channels of the multi-scale auto-encoder network are (96, 32), (192, 64), (384, 128), (128, 32), (256, 64), and (160, 32).

[0072] Feature map I (3,1) The input and output block Conv obtains an output result O p .

[0073] Step 3: Construct the total loss function L of the multi-scale auto-encoder network by using formula (1):

[0074] L = L ssim + λL pixel (1)

[0075] In formula (1), λ represents a weight coefficient of the pixel loss, L ssim represents a structural similarity loss function, and is obtained by formula (2), and L pixel represents a pixel loss function, and is obtained by formula (3);

[0076] L ssim = 1-SSIM(I p ,O p ) (2)

[0077]

[0078] In formula (2), SSIM represents structural similarity.

[0079] Step 4: Based on the training set, the multi-scale auto-encoder network is trained by using a back propagation algorithm, and the total loss function L is calculated to adjust the network parameters until a maximum iteration number is reached, so as to obtain the trained multi-scale auto-encoder network;

[0080] Step 5: Obtain a pair of multi-exposure images B and convert to a pair of images in which only the Y channel is reserved after the Ycbcr color domain, so as to obtain a pair of preprocessed multi-exposure images {(I o1 ,I u1 ), (I o2 ,I u2 ), …, (I ob ,I ub ), …, (I oB ,I uB )}, wherein (I ob ,I ub ) represents the bth pair of multi-exposure images, Iob overexposed image of the bth Y channel, I ub underexposed image of the bth Y channel;

[0081] Step 6: input the multi-exposure image pair {(I o1 ,I u1 ),(I o2 ,I u2 ),…,(I ob ,I ub ),…,(I oB ,I uB} into the trained multi-scale encoder respectively to obtain S scales of overexposed image features {Io f1 ,Io f2 , …, Io fs , …, Io fS} and underexposed image features {Iu f1 ,Iu f2 , …, Iu fs , …, Iu fS}, wherein Io fs represents the s th overexposed image feature, and Iu fs represents the s th underexposed image feature.

[0082] add the s th overexposed image feature Io fs and the s th underexposed image feature Iu fs and take the average to obtain the s th fusion feature f s , thereby obtaining a fusion feature set {f1,f2,…,f s ,…,f S} and inputting the fusion feature set into the trained decoder, thereby obtaining a fusion result {Output1,Output2,…,Output b ,…,Output B}, wherein Output b represents the fusion result of the overexposed image I ob of the b th Y channel and the underexposed image I ub of the b th Y channel, and the fusion process is shown in Figure 3 .

[0083] convert {Output1,Output2,…,Output b ,…,Output B} from the Ycbcr domain to the RGB domain, and finally obtain a color image with uniform exposure {Result1,Result2,…,Result b ,…,Result B}, wherein Result b represents the bth color image result.

[0084] In this embodiment, an electronic device includes a memory configured to store a program supporting a processor to perform the multi-exposure image fusion method, and the processor configured to execute the program stored in the memory.

[0085] In this embodiment, a computer readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, performs the steps of the multi-exposure image fusion method.

Claims

1. A multi-exposure image fusion method based on a multi-scale autoencoder, characterized in that, Comprising the following steps: Step 1: Obtain P RGB natural images and convert them to grayscale images, denoted as {I1, I2, …, Ip} and as a training set, where Ip represents the pth grayscale image. p ,…,Ip P} and as a training set, where Ip represents the pth grayscale image. p ​ Step 2, constructing a multi-scale auto-encoder network, comprising: a multi-scale encoder and a decoder; Step 2.1: the multi-scale encoder comprises W convolutional blocks A1, A2, …, A w ,…,A W W-1, X convolutional blocks N1, N2, …, N x ,…,N X X-1, and Y visual transformers Trans1, Trans2, …, Trans y ,…,Trans Y Y-1, wherein A w represents the w-th convolutional block, and the w-th convolutional block A w comprises a convolutional layer with a convolution kernel of A x A and a ReLU activation function; N x represents the x-th convolutional block; and the x-th convolutional block N x comprises a convolutional layer with a convolution kernel of N x N and a ReLU activation function; Trans y represents the y-th visual transformer; Y = W - 2; Step 2.1.1: the p-th gray image I p After inputting into the multi-scale encoder and sequentially passing through the processing of the 1st convolutional block A1 and the 1st convolutional block N1, the obtained primary shallow feature map I N1 After sequentially passing through the processing of Y visual converters Trans1, Trans2, …, Trans y ,…, Trans Y , the corresponding Y primary deep feature maps wherein, denotes the y-th primary deep feature map; Step 2.1.2: processing the input into the second convolutional block A2 to obtain a shallow feature map with C channels Input into the second convolutional block A2 for processing to obtain a shallow feature map with C channels Processing the input into the second convolutional block A2 for processing to obtain a shallow feature map with C channels Processing the input into the second convolutional block A2 for processing to obtain a shallow feature map with C channels w Processing the input into the second convolutional block A2 for processing to obtain a shallow feature map with C channels W Processing the input into the second convolutional block A2 for processing to obtain a shallow feature map with C channels wherein represents the w-2th deep feature map; the shallow feature map and the deep feature map comprise W-1 comprehensive feature maps Step 2.1.3: W-th integrated feature map After processing by the X-th convolutional block N X The (X-1)-th multi-scale feature map is obtained Step 2.1.4: Adding the W-1th integrated feature map to the W-2th up-sampled feature map After up-sampling, the W-2th up-sampled feature map is obtained The up-sampled feature map And the W-2th integrated feature map After element-wise addition operation, the W-2th intermediate feature map is obtained The W-2th intermediate feature map After the X-1th convolution block N X-1 The X-2th feature is obtained Step 2.1.5: upsampling the W-3th up-sampling feature to obtain a W-4th up-sampling feature X-2 ​​​​​​ Step 2.1.6: After processing X-1 multi-scale feature maps in turn according to the process of step 2.1.5 represents the x-1th multi-scale feature map​ Step 2.2: The decoder consists of P convolutional blocks and an output block Conv, where the P convolutional blocks are densely connected in upper triangle and are sequentially denoted as where Decoder (i,j) represents the i-th column j-th row convolutional block, and the i-th column j-th row convolutional block Decoder (i,j) includes a convolutional layer with a convolution kernel of N x N and a ReLU activation function; I = J = X - 2, The output block Conv comprises a convolution layer with a convolution kernel of AxA and a ReLU activation function; Step 2.2.1: X-1 multi-scale features outputted by the encoder are inputted into the decoder The row indices of the corresponding decoder are respectively denoted as 1, 2, …, j, …, J; then the output of the decoder is denoted as The output of the decoder is denoted as wherein, denotes the jth output multi-scale feature; Step 2.2.2: Decoder of the 1st column jth row (1,j) The input of and its up-sampled feature map Decoder of the 1st column jth row (1,j) The output of (1,j) ; Step 2.2.3: the convolution block Decoder of the rest of the columns except the first column and the jth row (i,j) The input of the convolution block Decoder is the jth plurality of scale feature maps The j+1th row and the i-1th column decoder output feature I (i-1,j+1) The up-sampling feature map of the jth row and the i-1th column decoder And the feature map I output by the jth row and the i-1th column decoder, the jth row and the i-2th column decoder to the jth row and the 1th column decoder (i-1,j) … (1,j) The spliced feature map; the convolution block Decoder (i,j) The output of the convolution block Decoder is the feature map I (i,j) So that the feature I is obtained after the P convolution blocks of the decoder The feature I is obtained after the P convolution blocks of the decoder (I,1) ; the feature I (I,1) After the processing of the output block Conv, the output result O is obtained p ; Step 3: constructing a total loss function L of the multi-scale auto-encoder network using formula (1): L = L ssim + λL pixel (1) In formula (1), λ represents a weight coefficient of pixel loss, L ssim represents a structural similarity loss function, and is obtained from formula (2), L pixel represents a pixel loss function, and is obtained from formula (3); L ssim = 1 - SSIM(I p , O p ) (2) In formula (2), SSIM represents structural similarity; Step 4: based on the training set, using a back propagation algorithm to train the multi-scale auto-encoder network, and calculating the total loss function L to adjust the network parameters until a maximum iteration number is reached, thereby obtaining a trained multi-scale auto-encoder network; Step 5: Obtain the B pair of multi-exposure images and convert to the Ycbcr color gamut to obtain a pair of images only retaining the Y channel, thereby obtaining the preprocessed B pair of multi-exposure images {(I o1 , u1 ,(I o2 , u2 ),(I ob , ub ),(I oB , uB )} wherein (I ob , ub ) represents the bth pair of multi-exposure images, I ob represents the bth Y channel overexposure image, and I ub represents the bth Y channel underexposure image. Step 6: input the multi-exposure image pairs {(I o1 , u1 ),(I o2 , u2 ),…,(I ob , ub ),…,(I oB , uB )} into the trained multi-scale encoder respectively to obtain S scales of over-exposed image features {Io f1 , Io f2 , …, Io fs , …, Io fS} and under-exposed image features {Iu f1 , Iu f2 , …, Iu fs , …, Iu fS}, wherein Io fs represents the s-th over-exposed image feature, and Iu fs represents the s-th under-exposed image feature; The s-th overexposed image feature Io fs and the s-th underexposed image feature Iu fs are added and averaged to obtain the s-th fusion feature f s , so as to obtain a fusion feature set {f1, f2, …, f s , …, f S} and input into the trained decoder, so as to obtain a fusion result {Output1, Output2, …, Output b , …, Output B}, wherein Output b represents the fusion result of the b-th Y channel overexposed image I ob and the b-th Y channel underexposed image I ub . {Output1, Output2, …, Output b ,…,Output B} are converted to the RGB domain through the Ycbcr domain, and finally the color images {Result1, Result2, …, Result b ,…,Result B} with uniform exposure are obtained, wherein Result b represents the bth color image result.

2. An electronic device comprising a memory and a processor, characterized in that The memory is used to store a program supporting the processor to execute the multi-exposure image fusion method of claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to perform the steps of the multi-exposure image fusion method of claim 1.

Images