Image and video exposure correction method based on zero-reference multi-exposure fusion network

Abstract

The present invention belongs to the field of image processing technology, and discloses an image and video exposure correction method based on a zero-reference multi-exposure fusion network, including the following steps: constructing a ZSDEC‑Net image and video exposure correction dataset; building an image enhancement network based on zero-reference multi-exposure fusion correction; performing multi-exposure fusion correction on images; performing multi-exposure fusion correction on videos; and deploying the multi-exposure fusion correction method for images and videos on a Web terminal. The present invention adopts the above-mentioned image and video exposure correction method based on a zero-reference multi-exposure fusion network, uses a deep learning network to perform exposure correction on the s-curve of the image, obtains the intermediate exposure correction result of the illumination channel and the corresponding inverted illumination map, selects the best exposure area for fusion through multi-exposure fusion technology, and obtains the global best exposure correction result, which is applicable to various exposure conditions, performs rapid and thorough exposure correction on images and videos, and improves the visual quality of images and videos.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image and video exposure correction method based on a zero-reference multi-exposure fusion network. Background Art

[0002] Due to the limitations of the shooting environment and equipment, the resulting images may be underexposed or overexposed. Under such inappropriate exposure conditions, some details in the image will be obscured by underexposure or overexposure, which seriously affects the visual experience of the image and limits the application of subsequent high-level visual tasks such as target detection, target segmentation, and autonomous driving. In addition, these images are degraded in quality, with not only exposure problems, but also poor contrast and noise. Therefore, image exposure correction is a challenging image processing problem.

[0003] There are some problems with previous image exposure correction methods. First, for traditional image exposure correction methods, they are mainly divided into histogram equalization and Retinex-based models. These methods can perform simple exposure correction on images, but they rely heavily on manual priors and have limited generalization capabilities, and cannot handle complex exposure scenes well. In recent years, learning-based image exposure correction methods have received widespread attention. At present, most learning-based exposure correction methods are mainly used to process underexposed images and do not fully consider overexposure. In addition, some methods improve the feature extraction ability of images by introducing Transformer structures. However, this structure consumes a lot of computational overhead. For supervised image exposure correction methods, good correction effects have been achieved, but paired data sets need to be prepared, which limits the application of such methods in high-level visual tasks. For unsupervised image exposure correction methods, the problems of supervised methods can be effectively avoided, and they are highly competitive. At present, the existing image exposure correction network only relies on the input information of a single image for exposure estimation. The network does not make full use of the original information of the image, making it difficult to adapt to various complex exposure conditions. At the same time, the exposure correction effect of the image is not thorough, and ultimately the global optimal exposure correction result cannot be obtained. Multi-exposure fusion can effectively cope with various exposure conditions.

[0004] Therefore, studying an image and video exposure correction method based on a zero-reference multi-exposure fusion network (Zero Shot Deep Framework for Image Exposure Correction, ZSDEC-Net) has important theoretical research significance and practical application value. Summary of the invention

[0005] The purpose of the present invention is to provide an image and video exposure correction method based on a zero-reference multi-exposure fusion network, which uses a deep learning network to perform exposure correction on the S-curve of the image, obtains the intermediate exposure correction results of the illumination channel and the corresponding inverted illumination diagram, and finally, through the multi-exposure fusion technology, selects the best exposure area from the input image and the two intermediate exposure-corrected images for fusion, thereby obtaining the global best exposure correction result; and the present invention is applicable to various exposure conditions, can perform fast and thorough exposure correction on images or videos, and improves the visual quality of images and videos.

[0006] To achieve the above object, the present invention provides an image and video exposure correction method based on a zero-reference multi-exposure fusion network, comprising the following steps:

[0007] S1. Build ZSDEC-Net image and video exposure correction dataset;

[0008] S2. Build an image enhancement network based on zero-reference multi-exposure correction;

[0009] S3, performing multi-exposure fusion correction on the image sequence;

[0010] S4, performing multi-exposure fusion correction on the video sequence;

[0011] S5. Deploy a multi-exposure fusion correction method for images and videos on the Web.

[0012] Preferably, in step S1, a real exposure-corrected image and video data set is collected, and the specific process is as follows:

[0013] S121. First, use a smart phone to take photos; wherein, turn off the smart night photography mode of the phone to avoid the phone's own algorithm from correcting the image and video;

[0014] S122, secondly, the shooting mode is to select a camera position in a dark place to shoot a bright object far away, so that the image contains both underexposed and overexposed areas;

[0015] S123, then, select the images and videos, remove the unclear images and videos in the captured images, and adjust the images to a uniform size of 512×512;

[0016] S124. Finally, the image and video exposure correction dataset is obtained.

[0017] Preferably, in step S2, the data collected in step S1 is preprocessed, and the specific process is as follows:

[0018] First, convert the input RGB image into Y, I, and Q channels; Y represents the illumination channel, and I and Q are color channels;

[0019] Secondly, the Y channel is subjected to guided filtering and denoising, and then its size is adjusted to (512, 512) and normalized to (0, 1) to obtain the denoised lighting channel Y0;

[0020] Finally, the above Y0 channel is inverted to obtain the inverted lighting channel Y1=1-Y0.

[0021] Preferably, in the network construction of step S2, the network structure based on zero-reference multi-exposure fusion correction consists of a multi-exposure correction module and a multi-exposure fusion module; wherein, for the multi-exposure correction network structure, by estimating two parameters in the S-type exposure estimation curve formula, high exposure and low exposure , to adjust the image The exposure is as follows:

[0022] ;

[0023] in, Represents the lighting image and invert lighting channels , is the high exposure of the exposure curve, is the low exposure of the exposure curve; is the increment function, , and are the default parameters, set to 5, 14 and 1.6 respectively.

[0024] Preferably, the exposure parameter estimation network consists of a downsampling module and a fully connected layer module; the downsampling module includes 5 downsampling layers, each of which consists of a convolutional layer, a BN layer, a ReLU layer and a final maximum pooling layer; the fully connected module includes 3 fully connected blocks and 2 fully connected layers, each of which includes 1 fully connected layer, a BN layer and a ReLU layer.

[0025] Preferably, in the network training of step S2, the total loss function of the exposure correction network is constructed ;in, Contains exposure control loss , noise loss , spatial consistency loss and total variation loss ;

[0026] Total loss function , as shown below:

[0027] ;

[0028] in, , , and Represents the weight parameter that balances different losses during network training;

[0029] Exposure Control Loss Constrain the pixels of the reconstructed image to 0.5 as follows:

[0030] ;

[0031] in, The default value is 0.5; for The image after exposure correction;

[0032] In order to further suppress noise, the image noise loss is established , as shown below:

[0033] ;

[0034] in, is a weighting parameter, the default ; r represents the upper, lower, left, and right regions of image pixels, and Indicates the values ​​of the upper, lower, left and right regions in the corresponding image;

[0035] Using total variation loss , so that the reconstructed image is smoothed as follows:

[0036] ;

[0037] in, and express The horizontal and vertical gradients of the image;

[0038] During image exposure correction, spatial consistency loss is used Constrain the consistency of similar regional differences as follows:

[0039] ;

[0040] in, J is the number of local regions, m ∈Ω( j ) is based on the region j The four adjacent areas are centered, namely, up, down, left, and right; and For the j The average brightness of the uncorrected image and the average brightness of the corrected image. and For the region j is the average brightness of the uncorrected image and the average brightness of the corrected image corresponding to the four different areas in the center;

[0041] By constraining the above loss, we can obtain The exposure correction result of the image ; Then, based on the intermediate exposure correction results, the ratio is calculated as follows:

[0042] ;

[0043] in, The image is corrected for exposure. Using this ratio, new Y, I, and Q channels are generated, and finally the new Y, I, and Q channels are converted into the best RGB exposure-corrected image. .

[0044] Preferably, in step S3, multi-exposure fusion correction is performed on the image sequence, and the specific process is as follows:

[0045] S31. Calculate the quality index of the image pixels, including the contrast, saturation and brightness values ​​of the intermediate exposure correction results S0 and S1 of the input images S, Y0 and Y1, as follows:

[0046] Image contrast for:

[0047] ;

[0048] in, It refers to the gradient of the RGB exposure-corrected image; the image is normalized, then Laplace filtered, and finally the absolute value is taken as the contrast;

[0049] The saturation of the image is calculated by calculating the standard deviation between the three RGB channels, as follows:

[0050] ;

[0051] ;

[0052] in, , and Represents the three RGB pixel values ​​of the exposure correction image, It is the pixel mean of the three channels of exposure correction RGB;

[0053] The relationship between the pixel values ​​of an image and its corresponding weight follows a Gaussian distribution with a mean value of 0.5, as shown below:

[0054] ;

[0055] in, , and The relationship between the exposure-corrected image pixels and their corresponding weights conforms to a Gaussian distribution with a mean of σ = 0.5;

[0056] S32, multi-exposure fusion correction;

[0057] First, after calculating the three image evaluation indicators of each image sequence, the weight calculation formula is combined to generate the weight value of each pixel. The specific weight formula is as follows:

[0058] ;

[0059] in, , and is the control coefficient, the default value is 1;

[0060] Then, according to the calculated weights, the image sequence and the weight values ​​corresponding to the image sequence are Laplace transformed respectively, and finally the Laplace transform results are upsampled and fused according to the weights. The specific fusion formula is as follows:

[0061] ;

[0062] in, m Represents images of different scales after Laplace transform; and They represent weight matrices and corrected images of different scales after Laplace transform respectively. Finally, according to the above fusion formula, the image pixels of three different sequences are fused at multiple scales to obtain the final global optimal multi-exposure fusion correction result.

[0063] Preferably, in step S5, a multi-exposure fusion correction method for images and videos is deployed on the Web side, and the specific process is as follows:

[0064] S51. Web page design suggestion: The web page includes one-click uploading of images and videos, back-end exposure correction and result saving, and creates a Docker image to achieve multi-environment deployment;

[0065] S52, Program design: First, according to the functional requirements of the front-end, encode and debug the three files of css, html, and js respectively until they meet the design requirements; then encode the back-end code, encode the app.py file, exposure correction processing file, loss and model files to meet the requirements;

[0066] S53. Result visualization: Run the web program locally, and on the web page that appears, upload images and videos with one click to obtain the final multi-exposure fusion corrected image and video results.

[0067] Therefore, the present invention adopts the above-mentioned image and video exposure correction method based on the zero-reference multi-exposure fusion network, and uses the deep learning network to perform exposure correction on the S-curve of the image to obtain the intermediate exposure correction results of the illumination channel and the corresponding inverted illumination map. Finally, through the multi-exposure fusion technology, the best exposure area is selected from the input image and the two intermediate exposure-corrected images for fusion, so as to obtain the global best exposure correction result; and the present invention is applicable to various exposure conditions, and can perform fast and thorough exposure correction on images or videos, thereby improving the visual quality of images and videos.

[0068] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a workflow diagram of the image and video exposure correction method based on the zero-reference multi-exposure fusion network of the present invention;

[0070] Figure 2 is a diagram of the exposure estimation network structure of the present invention;

[0071] Figure 3 It is a diagram of the multi-exposure fusion correction structure of the present invention;

[0072] Figure 4 It is a workflow diagram of the multi-exposure fusion correction method of the video of the present invention;

[0073] Figure 5 : is the result image of the multi-exposure fusion correction network of the present invention; wherein (a) and (b), (c) and (d) are the original image of the real scene and the result image after exposure correction respectively;

[0074] Figure 6 This is the result of the web-side image fusion correction of the present invention;

[0075] Figure 7 This is a result diagram of the web-side video fusion correction of the present invention. DETAILED DESCRIPTION

[0076] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0077] like Figure 1 As shown, the image and video exposure correction method based on the zero-reference multi-exposure fusion network of the present invention includes the following steps:

[0078] S1. Build ZSDEC-Net image and video exposure correction dataset;

[0079] S2. Build an image enhancement network based on zero-reference multi-exposure correction;

[0080] S3, fusion correction of multi-exposure image sequences;

[0081] S4, performing fusion correction on the multi-exposure video sequence;

[0082] S5. Deploy a multi-exposure fusion correction method for images and videos on the Web.

[0083] Example

[0084] S1. Build the ZSDEC-Net image and video exposure correction dataset.

[0085] S11. Sources of image and video data.

[0086] We collect 44 images with different exposures from the existing DICM dataset, 36 images with different exposures from the ExDark dataset, 16 images with different exposures from the Fusion dataset, 17 images with different exposures from the MEF dataset, and 10 images with different exposures from the LIME dataset. We also select 4 test videos provided by the ExCNet method.

[0087] S12. The process of collecting real exposure-corrected image and video datasets is as follows:

[0088] S121. First, use a Xiaomi smartphone to shoot. Turn off the smart night photography mode of the smartphone to avoid the correction of images and videos by the smartphone’s own algorithm.

[0089] S122, secondly, the shooting method is to select a camera position in a dark place to shoot a distant object with high brightness, so that the image contains both underexposed and overexposed areas;

[0090] S123, then, select the images and videos, remove the unclear images and videos in the captured images, and adjust the images to a uniform size of 512×512;

[0091] S124. Finally, the image and video exposure correction dataset contains 15 images and 2 videos, as shown in Table 1.

[0092] Table 1 The collected image datasets

[0093]

[0094] S2. Build an image enhancement network based on zero-reference multi-exposure correction.

[0095] S21, data preprocessing, the specific process is as follows:

[0096] The input RGB image is decoupled into Y, I, and Q channels, where Y represents the illumination channel and I and Q represent colors. The Y channel occupies more high-frequency information, which generally has more noise. Therefore, the Y channel is subjected to guided filtering denoising, and then its size is adjusted to (512, 512) and normalized to (0, 1) to obtain the denoised Y0 channel; then, the illumination channel Y0 is inverted to obtain the inverted illumination channel Y1=1-Y0.

[0097] S22. Network construction and training.

[0098] S221. Network construction. The specific process is as follows:

[0099] First, the network structure based on zero-reference multi-exposure fusion correction consists of a multi-exposure correction module and a multi-exposure fusion module. The software environment required for network construction and training is configured. For the exposure estimation network (DEC-Net network) structure, Figure 2 As shown; Among them, for the multi-exposure correction network structure, by estimating the two parameters in the S-shaped exposure estimation curve formula, high exposure and low exposure , to adjust the image The exposure is as follows:

[0100] ;

[0101] in, Represents the lighting image and invert lighting channels , is the high exposure of the exposure curve, is the low exposure of the exposure curve; is the increment function. To limit the increment function to the range of [0,0.5], the default parameter , and Settings are 5, 14 and 1.6. High exposure by estimating the curve and shadow amount To correct images under different exposure conditions, for underexposed pixels, increase The size of can change the slope of the curve in low grayscale areas, thereby enhancing underexposed areas more quickly. At the same time, for overexposed pixels, increasing The value of will increase the slope of the curve in the high grayscale area, resulting in a slower increase in the value of high grayscale pixels, thereby suppressing overexposure. The high exposure of the curve is estimated by the DEC-Net network and shadow amount , which consists of a downsampling module and a fully connected layer module. The downsampling module contains 5 downsampling layers. Each downsampling layer consists of a convolutional layer, a BN layer, a ReLU layer and the final maximum pooling layer, as shown in Table 2.

[0102] Table 2 Network structure of downsampling module

[0103]

[0104] Through 5 layers of downsampling, the image size is changed from the original (512, 512) to (16, 16) size, and then the tensor is flattened to (512×16×16, 512). Finally, it is passed to the fully connected module, which contains 3 layers of fully connected blocks and 2 layers of fully connected layers, each of which contains a fully connected layer, a BN layer and a ReLU layer, as shown in Table 3.

[0105] Table 3 Network structure of the fully connected layer module

[0106]

[0107] Through the above five fully connected layers, we get two important parameters of the exposure correction curve: high exposure and shadow amount Finally, with Input them together into the S-shaped exposure estimation curve of the image to obtain the initial exposure estimation graph .

[0108] S222. Network training.

[0109] First, we set the network input and output paths, as well as the hyperparameters. In order to better optimize the S-shaped exposure correction curve, we designed the total loss function of the exposure correction network (DSCE-Net): , Includes exposure control loss , noise loss , spatial consistency loss and total variation loss .

[0110] Total loss function The formula is as follows:

[0111] ;

[0112] in, , , and Represents the weight parameter for balancing different losses in network training; during training, the parameters of the loss function are 1, 1, 20, and 1600 respectively.

[0113] Exposure control loss: In order to control the exposure range of the image, an exposure control loss is proposed, the purpose is to constrain the pixels of the reconstructed image to around 0.5. , as shown below:

[0114] ;

[0115] in, The default value is 0.5; for Image after exposure correction.

[0116] Noise loss: In order to further suppress noise, a noise loss for the image is designed as follows:

[0117] ;

[0118] in, is a weighting parameter, the default =0.5; r represents the upper, lower, left, and right regions of image pixels, and Represents the values ​​of the upper, lower, left, and right regions in the corresponding image.

[0119] Total variation loss: Natural images are generally smooth. The total variation loss is used to make the reconstructed image as smooth as possible, as shown below:

[0120] ;

[0121] in, and express The horizontal and vertical gradients of the image.

[0122] Spatial consistency loss: In the process of image exposure correction, in order to make the image restoration more natural, the difference between any area of ​​the image and similar areas must be the same. Therefore, the spatial consistency loss is used To constrain the consistency of similar regional differences, as follows:

[0123] ;

[0124] in, J is the number of local regions, m ∈Ω( j ) is based on the region j Four adjacent areas (from top, bottom, left and right) as the center; and For the j The average brightness of the uncorrected image and the average brightness of the corrected image. and For the region j The average brightness of the uncorrected image and the average brightness of the corrected image corresponding to the four different areas in the center.

[0125] By constraining the above loss, we can obtain The exposure correction result of the image ; Then, based on the intermediate exposure correction results, the ratio is calculated as follows:

[0126] ;

[0127] in, The image is corrected for exposure. Using this ratio, new Y, I, and Q channels are generated, and finally the new Y, I, and Q channels are converted into the best RGB exposure-corrected image. .

[0128] S3. Perform fusion correction on the multi-exposure image sequence.

[0129] S31. Calculate quality indicators of image pixels.

[0130] The intermediate exposure correction results S0 and S1 of the input images S, Y0 and Y1 are subjected to multi-exposure fusion processing, such as Figure 3 As shown, a schematic diagram of the multi-exposure fusion correction structure of the image is shown. Specifically, first, the contrast, saturation and brightness values ​​of the S, S0 and S1 images are calculated respectively.

[0131] Among them, the image contrast is calculated as follows:

[0132] ;

[0133] Among them, the above It refers to the gradient of the RGB exposure-corrected image; the image is normalized, then Laplace filtered, and finally the absolute value is taken as the contrast.

[0134] The saturation of the image is calculated by calculating the standard deviation between the three RGB channels, as follows:

[0135] ;

[0136] ;

[0137] in, , and Represents the three RGB pixel values ​​of the exposure correction image, It is the pixel mean of the three channels of exposure correction RGB;

[0138] The calculation of brightness weights is performed by normalizing the image from 0 to 1. Pixels with values ​​close to 0.5 are considered well-exposed and are therefore assigned a larger weight. Conversely, pixels with values ​​close to 0 or 1 are classified as underexposed or overexposed, respectively, and are therefore assigned a smaller weight. The relationship between the pixel values ​​of an image and its corresponding weight follows a Gaussian distribution with a mean of 0.5, defined as follows:

[0139] ;

[0140] in, , and The relationship between the exposure-corrected image pixels and their corresponding weights conforms to a Gaussian distribution with a mean of σ = 0.5.

[0141] S32, multi-exposure fusion correction.

[0142] First, after calculating the three image evaluation indicators of each image sequence, the weight calculation formula is combined to generate the weight value of each pixel. The specific weight formula is as follows:

[0143] ;

[0144] in, , and is the control coefficient, the default value is 1;

[0145] Then, according to the calculated weights, the image sequence and the weight values ​​corresponding to the image sequence are Laplace transformed respectively, and finally the Laplace transform results are upsampled and fused according to the weights. The specific fusion formula is as follows:

[0146] ;

[0147] in, m Represents images of different scales after Laplace transform; and They represent weight matrices and corrected images of different scales after Laplace transform respectively. Finally, according to the above fusion formula, the image pixels of three different sequences are fused at multiple scales to obtain the final global optimal multi-exposure fusion correction result.

[0148] S33. Robustness experiment.

[0149] The data sets collected in step S1 are tested separately, such as Figure 5As shown in the figure, by comparison, it can be found that this method can handle various exposure conditions. In addition, by analyzing the correction results of the real data set, it can be shown that the model can effectively correct different exposures, and the correction is more thorough. The NIQE image evaluation index is used to evaluate the input image S, the intermediate exposure correction images S0 and S1, and the multi-exposure fusion image S E Through analysis, it can be found that the quality of images corrected by multi-exposure fusion is higher than that of other images that have not been corrected by exposure fusion. The results show that multi-exposure fusion correction can effectively improve the quality of images.

[0150] S4. Perform multi-exposure fusion correction on the video sequence.

[0151] like Figure 4 As shown in FIG. 1 , a flowchart of the multi-exposure fusion correction method for a video is shown. First, the frame rate and the total number of frames of the video are calculated respectively, and then each frame of the video is read in sequence, and steps S2 and S3 are repeated to perform multi-exposure fusion correction on each frame of the image, and then the multi-exposure fusion correction results of each frame are resynthesized into a video stream.

[0152] S5. Deploy a multi-exposure fusion correction method for images and videos on the Web.

[0153] S51. Recommended design of web page.

[0154] like Figure 6 and Figure 7 As shown in the figure, a schematic diagram of the deployment page structure of the multi-exposure fusion correction method for images and videos on the Web side is shown. The Web side includes one-click uploading of images and videos, back-end exposure correction and result saving, and creates a Docker image to achieve multi-environment deployment.

[0155] S52. Programming.

[0156] First, according to the functional requirements of the front-end, the three files of css, html, and js are coded and debugged respectively until they meet the design requirements;

[0157] Then, encode the backend code, mainly the app.py file, exposure correction processing file, loss and model file, so that it can meet the requirements.

[0158] S53. Result visualization.

[0159] Run the web program locally, and on the web page that appears, upload images and videos with one click to obtain the final multi-exposure fusion corrected image and video results.

[0160] Therefore, the present invention adopts the above-mentioned image and video exposure correction method based on the zero-reference multi-exposure fusion network, and uses the deep learning network to perform exposure correction on the S-curve of the image to obtain the intermediate exposure correction results of the illumination channel and the corresponding inverted illumination map. Finally, through the multi-exposure fusion technology, the best exposure area is selected from the input image and the two intermediate exposure-corrected images for fusion, so as to obtain the global best exposure correction result; and the present invention is applicable to various exposure conditions, and can perform fast and thorough exposure correction on images or videos, thereby improving the visual quality of images and videos.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. Image and video exposure correction method based on zero-reference multi-exposure fusion network, characterized in that: The following steps are involved: S1. Construct the ZSDEC-Net image and video exposure correction dataset. The specific process is as follows: S121. First, use a smart phone to take photos; wherein, turn off the smart night photography mode of the phone to avoid the phone's own algorithm from correcting the image and video; S122, secondly, the shooting mode is to select a camera position in a dark place to shoot a bright object far away, so that the image contains both underexposed and overexposed areas; S123, then, select the images and videos, remove the unclear images and videos in the captured images, and adjust the images to a uniform size of 512×512; S124. Finally, the image and video exposure correction dataset is obtained; S2. Build an image enhancement network based on zero-reference multi-exposure correction. The specific process is as follows: S21, first, preprocessing the image and video exposure correction dataset constructed in step S1; S22. According to the preprocessed image and video exposure correction dataset, an image enhancement network based on zero-reference multi-exposure correction is constructed; wherein the network structure based on zero-reference multi-exposure fusion correction consists of a multi-exposure correction module and a multi-exposure fusion module; S23. The total loss function by constructing a multi-exposure correction network Training an image enhancement network based on zero-reference multi-exposure correction; S3, perform multi-exposure fusion correction on the image sequence, the specific process is as follows: S31, calculate the quality index of the image pixels, including the contrast, saturation and brightness values ​​of the intermediate exposure correction results S0 and S1 of the input image S, the illumination image Y0 and the inverted illumination image Y1, as follows: Image contrast for: ； in, It refers to the gradient of the RGB exposure-corrected image; the image is normalized, then Laplace filtered, and finally the absolute value is taken as the contrast; The saturation of the image is calculated by calculating the standard deviation between the three RGB channels, as follows: ； ； in, , and Represents the three RGB pixel values ​​of the exposure correction image, It is the pixel mean of the three channels of exposure correction RGB; The relationship between the pixel values ​​of an image and its corresponding weight follows a Gaussian distribution with a mean of 0.5, as shown below: ； in, , and The relationship between the exposure-corrected image pixels and their corresponding weights is in accordance with the mean σ = 0.5 Gaussian distribution; S32, multi-exposure fusion correction; First, after calculating the three image evaluation indicators of each image sequence, the weight calculation formula is combined to generate the weight value of each pixel. The specific weight formula is as follows: ； in, , and is the control coefficient, the default value is 1; Then, according to the calculated weights, the image sequence and the weight values ​​corresponding to the image sequence are Laplace transformed respectively, and finally the Laplace transform results are upsampled and fused according to the weights. The specific fusion formula is as follows: ； in, m Represents images of different scales after Laplace transform; and Respectively represent the weight matrices and corrected images of different scales after Laplace transformation; Finally, according to the above fusion formula, the image pixels of three different sequences are fused at multiple scales to obtain the final global optimal multi-exposure fusion correction result; S4. Perform multi-exposure fusion correction on the video sequence. The specific process is as follows: First, the frame rate and total number of frames of the video are calculated respectively; then, each frame of the video is read in turn, and steps S2 and S3 are repeated to perform multi-exposure fusion correction on each frame; finally, the multi-exposure fusion correction results of each frame are resynthesized into a video stream; S5. Deploy the multi-exposure fusion correction method for images and videos on the Web. The specific process is as follows: S51. Web page design suggestion: The web page includes one-click uploading of images and videos, back-end exposure correction and result saving, and creates a Docker image to achieve multi-environment deployment; S52, Program design: First, according to the functional requirements of the front-end, encode and debug the three files of css, html, and js respectively until they meet the design requirements; then encode the back-end code, encode the app.py file, exposure correction processing file, loss and model files to meet the requirements; S53. Result visualization: Run the web program locally, and on the web page that appears, upload images and videos with one click to obtain the final multi-exposure fusion corrected image and video results.

2. The method for image and video exposure correction based on a zero-reference multi-exposure fusion network according to claim 1, characterized in that: In step S21, the data collected in step S1 is preprocessed, and the specific process is as follows: First, the input RGB image is converted into Y, I, and Q channels; Y represents the illumination channel, and I and Q are color channels; Secondly, the Y channel is subjected to guided filtering and denoising, and then its size is adjusted to (512, 512) and normalized to (0, 1) to obtain the denoised illumination image Y0; Finally, the above Y0 image is inverted to obtain the inverted illumination image Y1=1-Y0.

3. The method for image and video exposure correction based on a zero-reference multi-exposure fusion network according to claim 1, characterized in that: In step S22, based on the zero-reference multi-exposure fusion correction network structure, by estimating two parameters in the S-type exposure estimation curve formula, the high exposure and low exposure , to adjust the image The exposure is as follows: ； in, Represents the lighting image and invert the lighting image , is the high exposure of the exposure curve, is the low exposure of the exposure curve; is the increment function, , and are the default parameters, set to 5, 14 and 1.6 respectively.

4. The method for image and video exposure correction based on a zero-reference multi-exposure fusion network according to claim 3, characterized in that: The exposure parameter estimation network consists of a downsampling module and a fully connected layer module; the downsampling module contains 5 downsampling layers, each of which consists of a convolutional layer, a BN layer, a ReLU layer and a final maximum pooling layer; the fully connected module contains 3 fully connected blocks and 2 fully connected layers, each of which contains 1 fully connected layer, a BN layer and a ReLU layer.

Images