Multi-exposure image enhancement method and device based on adaptive weight

Abstract

The invention relates to the technical field of image processing, and provides a multi-exposure image enhancement method and device based on adaptive weights, which can enable pixel points in areas with different exposure degrees to correspond to different brightness weights by extracting the adaptive brightness weight of each image to be fused. By fusing the adaptive brightness weight, the contrast weight, the saturation weight and the visual saliency weight of each to-be-fused image, the detail and color information of each to-be-fused image can be kept and displayed optimally. The motion area is detected by calculating the local information entropy, artifacts can be better processed, and the image fusion effect is improved. By means of the pyramid fusion method, image details of different scales can be effectively reserved, meanwhile, multiple input images are smoothly fused, and the phenomena such as artifacts or false edges possibly occurring in the fusion process are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a multi-exposure image enhancement method and device based on adaptive weights. Background Art

[0002] The illuminance of objects in natural scenes varies greatly and can only be recorded within a wide dynamic range. However, the detection range of current imaging devices such as cameras is very limited, and image information is usually lost in underexposed or overexposed scenes, seriously reducing the image quality.

[0003] Multi-exposure image fusion can help solve the above problems by integrating images with different exposure levels, but the image effects obtained by fusion are often not satisfactory. It will not only cause the loss of scene details, but also may generate noise or color distortion, and lead to phenomena such as artifacts or false edges during the fusion process. Summary of the Invention

[0004] The present invention provides a multi-exposure image enhancement method and device based on adaptive weights to solve the defects existing in the prior art.

[0005] The present invention provides a multi-exposure image enhancement method based on adaptive weights, including: Obtaining a plurality of images to be fused with different exposure levels; Calculating the local information entropy of each pixel point in each image to be fused, generating an entropy map corresponding to each image to be fused, and detecting the motion area in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; Based on the pixel values of each pixel point in the reference image, applying a binary mask corresponding to the motion area in each image to be fused to correct the pixel values of each pixel point in each image to be fused, and obtaining a corrected image of each image to be fused; Extracting the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fusing the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain a fusion weight of each corrected image; Based on each corrected image, constructing a Laplacian pyramid, based on the fusion weight of each corrected image, constructing a Gaussian pyramid, and reconstructing a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

[0006] According to the multi-exposure image enhancement method based on adaptive weights provided by the present invention, the adaptive brightness weight of each corrected image is extracted based on the following steps: Determine the adaptive brightness weight of each corrected image based on the brightness image of each corrected image, the pixel mean value of the brightness image, and the pixel standard deviation. Among them, the pixel standard deviation uses a truncation function and is determined according to the brightness difference between adjacent corrected images.

[0007] According to a multi-exposure image enhancement method based on adaptive weights provided by the present invention, the pixel standard deviation is determined based on the following formula: ; Among them, is the pixel standard deviation of the brightness image of the nth corrected image, is the pixel mean value of the brightness image of the nth corrected image, is the pixel mean value of the brightness image of the (n + 1)th corrected image, is the pixel mean value of the brightness image of the (n - 1)th corrected image, N is the total number of corrected images, is the scaling factor.

[0008] According to a multi-exposure image enhancement method based on adaptive weights provided by the present invention, the contrast weight of each corrected image is extracted based on the following steps: Based on the Laplacian operator, calculate the contrast weight of each corrected image.

[0009] According to a multi-exposure image enhancement method based on adaptive weights provided by the present invention, the saturation weight of each corrected image is extracted based on the following steps: Calculate the maximum pixel value and the minimum pixel value of each pixel point in each corrected image in different color channels; Based on the maximum pixel value and the minimum pixel value, calculate the saturation weight of each corrected image.

[0010] According to a multi-exposure image enhancement method based on adaptive weights provided by the present invention, the visual saliency weight of each corrected image is extracted based on the following steps: Calculate the gradient intensity at each pixel point in the grayscale image of each corrected image; Based on the gradient intensity, calculate the visual saliency weight of each corrected image.

[0011] According to a multi-exposure image enhancement method based on adaptive weights provided by the present invention, the method of correcting the pixel values of each pixel point in each image to be fused by applying the binary mask corresponding to the motion region in each image to be fused based on the pixel values of each pixel point in the reference image to obtain the corrected image of each image to be fused includes: Based on morphological operations, perform denoising processing on the binary mask to obtain a denoising result; Based on the pixel values of each pixel point in the reference image, applying the denoising result, correct the pixel values of each pixel point in each image to be fused, and obtain the corrected image of each image to be fused.

[0012] The present invention also provides a multi-exposure image enhancement device based on adaptive weights, including: An image acquisition module, configured to acquire a plurality of images to be fused with different exposure degrees; A motion detection module, configured to calculate the local information entropy of each pixel point in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion area in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; An image correction module, configured to correct the pixel values of each pixel point in each image to be fused based on the pixel values of each pixel point in the reference image and apply the binary mask corresponding to the motion area in each image to be fused, and obtain the corrected image of each image to be fused; A weight extraction and fusion module, configured to extract the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fuse the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain the fusion weight of each corrected image; An image fusion module, configured to construct a Laplacian pyramid based on each corrected image, construct a Gaussian pyramid based on the fusion weight of each corrected image, and reconstruct a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the multi-exposure image enhancement method based on adaptive weights as described in any one of the above.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the multi-exposure image enhancement method based on adaptive weights as described in any one of the above.

[0015] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the multi-exposure image enhancement method based on adaptive weights as described in any one of the above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The multi-exposure image enhancement method and device based on adaptive weights provided by the present invention can detect moving regions by calculating local information entropy, correct the pixel values of each pixel point in each image to be fused corresponding to the binary mask of the moving region, better handle artifacts, and improve the effect of image fusion. By extracting the adaptive brightness weights of each corrected image, pixel points in regions with different exposure degrees can have different brightness weights. By extracting the visual saliency weights of each corrected image, from the perspective of visual saliency, the degree of attention of each pixel in the human visual system can be considered, thereby helping to optimize the fusion effect. By fusing the adaptive brightness weights, contrast weights, saturation weights, and visual saliency weights of each corrected image, the details and color information of each corrected image can be ensured to be best preserved and displayed. The pyramid fusion method can effectively retain image details at different scales, smoothly fuse multiple corrected images, and reduce phenomena such as artifacts or false edges that may occur during the fusion process. By processing image information separately at different scales, the local details and global structure can be better balanced, thereby generating a fusion result with better visual effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description can also be obtained by those of ordinary skill in the art without creative efforts based on these drawings.

[0018] Figure 1 is one of the flow diagrams of the multi-exposure image enhancement method based on adaptive weights provided by the present invention; Figure 2 is the second flow diagram of the multi-exposure image enhancement method based on adaptive weights provided by the present invention; Figure 3 is the structural diagram of the multi-exposure image enhancement device based on adaptive weights provided by the present invention; Figure 4 is the structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention fall within the protection scope of the present invention.

[0020] Figure 1 The flowchart of a multi-exposure image enhancement method based on adaptive weights provided in an embodiment of the present invention is shown as Figure 1 follows. The method includes: S1. Obtain multiple images to be fused with different exposure degrees; S2. Calculate the local information entropy of each pixel point in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion region in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; S3. Based on the pixel values of each pixel point in the reference image, apply the binary mask corresponding to the motion region in each image to be fused to correct the pixel values of each pixel point in each image to be fused, and obtain the corrected image of each image to be fused; S4. Extract the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fuse the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain the fusion weight of each corrected image; S5. Based on each corrected image, construct a Laplacian pyramid, based on the fusion weight of each corrected image, construct a Gaussian pyramid, and reconstruct a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

[0021] Specifically, for the multi-exposure image enhancement method based on adaptive weights provided in an embodiment of the present invention, the execution subject is a multi-exposure image enhancement device based on adaptive weights. This device can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, a tablet, etc., and no specific limitation is made here.

[0022] First, execute step S1 to obtain multiple images to be fused with different exposure degrees. Among them, the multiple images to be fused are a series of color images with different exposure degrees collected by a low dynamic range (LDR) imaging device. The images to be fused can include N in total, and N is greater than or equal to 2.

[0023] Then, execute step S2. In the practical process, when multiple images are fused into a high dynamic range image, it is often affected by camera shake or object movement, which results in a ghosting effect after fusion. Traditional image fusion methods are prone to image blurring or detail loss when dealing with moving objects. To solve this problem, the motion region is detected by calculating the local information entropy, so as to better process artifacts and improve the effect of image fusion.

[0024] By calculating the local information entropy of each pixel in the images to be fused, an entropy map corresponding to each image to be fused is generated.

[0025] Histogram equalization can be performed on each image to be fused first to redistribute the brightness values of each pixel, providing a more balanced brightness distribution for the images to be fused with different exposure levels. For color images, histogram equalization can be processed separately for each color channel to improve color saturation and visual effects.

[0026] When performing histogram equalization, the frequency of each gray level in the image to be fused can be calculated first. For each gray level , the cumulative distribution function is: ; where is the cumulative distribution function of gray level , and is the probability of gray level .

[0027] Then, the gray levels in the image to be fused are redistributed, and each gray level is mapped to a new gray level using the following formula: ; where L is the total number of gray levels, usually 256. The round function is used for rounding.

[0028] By performing histogram equalization on each image to be fused, the gray level distribution in each image to be fused can be made more uniform, eliminating the influence of pixel differences on motion detection.

[0029] Thereafter, each image to be fused is converted into a grayscale image to reduce the data dimension, eliminate the interference of color information, and simplify the subsequent calculation of entropy values. The conversion formula is as follows: ; where is the grayscale image of the nth image to be fused, and , and are the pixel values of the red, green, and blue channels of the nth image to be fused at the pixel point (x, y), respectively.

[0030] The local information entropy reflects the texture complexity and information content of an image by measuring the randomness of pixel values within an image block. The larger the entropy value, the more uncertain the pixel values within the block and the greater the information content. By calculating the local information entropy of each pixel in each grayscale image, the details and textures in the image can be effectively detected. The calculation formula for the local information entropy is as follows: ; where is the local information entropy of the nth grayscale image at the pixel point (x, y). is the probability that the pixel value i appears in the local window of the nth grayscale image. This local area can be set as needed. For example, it can be a window of 3×3, 5×5 or 7×7 pixels.

[0031] Subsequently, the local information entropy of different grayscale images at the pixel point (x, y) is normalized, and the calculated local information entropy is mapped to a standard range [0, 1] to eliminate the scale difference between different grayscale images for subsequent processing and comparison.

[0032] Thus, according to the local information entropy of the grayscale images of each image to be fused at different pixel points (x, y), the entropy map corresponding to each image to be fused can be determined.

[0033] Thereafter, the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image can be used to detect the moving region in each image to be fused.

[0034] The reference image can be the intermediate image of each image to be fused. By calculating the difference between the entropy map corresponding to other images to be fused and the entropy map corresponding to the reference image to detect the moving region, which usually shows a significant change in entropy value. The calculation formula is as follows: ; where, is the local information entropy of the pixel point (x, y) in the entropy map corresponding to the nth image to be fused, is the local information entropy of the pixel point (x, y) in the entropy map corresponding to the reference image, is the difference between the entropy map corresponding to the nth image to be fused and the entropy map corresponding to the reference image at the pixel point (x, y).

[0035] Then, the Otsu method is used for binarization to detect the moving region of each image to be fused. The moving region of each image to be fused can generate a corresponding binary mask, that is: ; where, is the value of the binary mask corresponding to the moving region of the nth image to be fused at the pixel point (x, y), Threshold is the specified threshold, which can be set as needed and is not specifically limited here.

[0036] Thereafter, step S3 is executed. Using the pixel value of each pixel point in the reference image and applying the binary mask corresponding to the moving region in each image to be fused, the pixel value of each pixel point in each image to be fused is corrected to obtain the corrected image of each image to be fused.

[0037] Among them, by using the generated binary mask and combining with the pixel values of each pixel point in the reference image to correct the pixel values of each pixel point in each image to be fused, the details of the static area can be retained, and only the moving area is adjusted. The mathematical formula is as follows: .

[0038] Among them, is the pixel value at the pixel point (x, y) in the nth image to be fused. is the pixel value at the pixel point (x, y) in the reference image.

[0039] After correction, the corrected image effectively detects and adjusts the moving area, reducing the ghosting effect caused by camera shake or object movement.

[0040] Thereafter, step S4 is executed to extract the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image. Among them, the adaptive brightness weight can be determined according to the brightness image of each corrected image. This adaptive brightness weight is the adaptive weight, indicating that the brightness weight changes with the brightness values at different pixel points in each corrected image. In this way, pixel points in areas with different exposure degrees can have different brightness weights.

[0041] The contrast weight is used to measure the severity of the change in pixel values of different pixel points in each corrected image. The saturation weight is used to measure the color purity of different pixel points in each corrected image. The higher the saturation weight, the more vivid the color. By extracting the contrast weight and saturation weight, the texture and color information of each corrected image can be better retained, enhancing the visual effect of each corrected image.

[0042] The visual saliency weight is used to measure the importance of each pixel point in different corrected images.

[0043] Thereafter, the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image are fused to obtain the fusion weight of each corrected image. The fusion process can consider the importance of the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight, and balance the contributions of different weights through parameter adjustment to ensure that the details and color information of each corrected image are best retained and displayed.

[0044] The calculation formula of the fusion weight is as follows: ; Among them, is the fusion weight at the pixel point (x, y) in the nth corrected image, is the adaptive brightness weight at the pixel point (x, y) in the nth corrected image, is the contrast weight at the pixel point (x, y) in the nth corrected image, is the saturation weight at the pixel point (x, y) in the nth corrected image, is the visual saliency weight at the pixel point (x, y) in the nth corrected image, , , , are the importance parameters of the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight at the pixel point (x, y) in the nth corrected image, respectively, which can be set according to actual needs and are not specifically limited here.

[0045] After that, the fusion weights of each corrected image can also be normalized to ensure that the fusion weights at the same pixel points in each corrected image are 1, that is: ; where is the normalization result of the fusion weight at the pixel point (x, y) in the nth corrected image.

[0046] Finally, step S5 is executed. Using the multi-resolution decomposition method, the corrected image of each corrected image is decomposed into a pyramid structure of different scales to construct a Laplacian pyramid, and the fusion weight of each corrected image is decomposed into a pyramid result of different scales to construct a Gaussian pyramid. Among them, the Gaussian pyramid represents the low-frequency information of each corrected image, and the Laplacian pyramid represents the high-frequency details of each corrected image.

[0047] Using the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image to retain the image details of different scales, a fused image is reconstructed. Here, the Laplacian pyramid and Gaussian pyramid can be fused first. At each pyramid level, the Gaussian pyramid is multiplied by the Laplacian pyramid, and then the multiplication results at the same pyramid level corresponding to each corrected image are added to obtain the fused pyramid at each pyramid level.

[0048] The formula for the fused pyramid is as follows: ; where is the fused pyramid at the l th pyramid level, is the Gaussian pyramid at the l th pyramid level corresponding to the ith corrected image, is the Laplacian pyramid at the l th pyramid level corresponding to the ith corrected image.

[0049] Using the fusion pyramid, the fused image can be reconstructed to achieve image enhancement for each image to be fused.

[0050] In the method for multi-exposure image enhancement based on adaptive weights provided in the embodiments of the present invention, the motion area is detected by calculating the local information entropy, and the binary mask corresponding to the motion area in each image to be fused is used to correct the pixel values of each pixel point in each image to be fused, which can better process artifacts and improve the effect of image fusion. By extracting the adaptive brightness weights of each corrected image, pixel points in areas with different exposure degrees can have different brightness weights. By extracting the visual saliency weights of each corrected image, from the perspective of visual saliency, the degree to which each pixel is concerned in the human visual system can be considered, thereby helping to optimize the fusion effect. By fusing the adaptive brightness weights, contrast weights, saturation weights, and visual saliency weights of each corrected image, the details and color information of each corrected image can be ensured to be optimally retained and displayed. The pyramid fusion method can effectively retain the image details at different scales, and at the same time smoothly fuse multiple corrected images, reducing phenomena such as artifacts or false edges that may occur during the fusion process. By processing the image information separately at different scales, the local details and the global structure can be better balanced, thereby generating a fusion result with better visual effects.

[0051] On the basis of the above embodiments, the adaptive brightness weights of each corrected image are obtained by the following steps: Based on the brightness image of each corrected image, the pixel mean value and the pixel standard deviation of the brightness image, the adaptive brightness weight of each corrected image is determined; Among them, the pixel standard deviation uses a truncation function and is determined according to the brightness difference between adjacent corrected images.

[0052] Specifically, the traditional brightness weight calculation formula is as follows: ; Among them, is the traditional brightness weight at the pixel point (x, y) in the nth image to be fused, is the pixel value at the pixel point (x, y) in the brightness image of the nth image to be fused, that is, the brightness value at the pixel point (x, y) in the nth image to be fused. The nth image to be fused can be normalized and then converted to the YCbCr color space, and the Y channel is extracted separately as the brightness image. However, the calculation coefficient of the Y channel is the same as the grayscale coefficient. Therefore, it can also be considered that the image after grayscale processing of the nth image to be fused is the brightness image. In the above formula, 0.5 represents the medium brightness value, and this value is used as the target brightness, so that pixels with brightness close to the middle value have higher weights and retain more details; Denotes a fixed parameter that controls the width of the Gaussian distribution. A smaller value will make the weight distribution more concentrated around 0.5, while a larger value will make the weight distribution smoother.

[0053] Traditional luminance weights cannot retain much texture and detail in the brighter areas of underexposed images or the darker areas of overexposed images. Therefore, in the embodiments of the present invention, when extracting the adaptive luminance weights of each corrected image, the luminance image of each corrected image, the pixel mean value of the luminance image, and the pixel standard deviation can be used to determine the adaptive luminance weights of each corrected image, that is: ; wherein, is the ideal exposure value, is the pixel value at the pixel point (x, y) in the luminance image of the nth corrected image, that is, the luminance value at the pixel point (x, y) in the nth corrected image, is the pixel mean value of the luminance image of the nth corrected image, that is, the luminance mean value. is the pixel standard deviation of the luminance image of the nth corrected image, that is, the luminance standard deviation.

[0054] The pixel standard deviation can be determined using a truncation function based on the luminance difference between adjacent corrected images. By using the truncation function and adjusting according to the luminance difference between adjacent corrected images, the luminance weight calculation is made adaptive to different exposure conditions. When the pixel standard deviation value is large, it indicates that the average luminance of the corrected image differs greatly from the average luminance of the adjacent corrected image, that is, the exposure difference between adjacent corrected images is large, and the distribution of the adaptive luminance weights will be wider to capture more details.

[0055] The above-mentioned adaptive luminance weights can better handle different exposure situations, improve the smoothness of the transition region, and reduce the discontinuity and artifacts caused by extreme exposure (too bright or too dark); the modified numerator adjusts the luminance center, not a single application of a fixed value of 0.5, but an adaptive luminance center value dynamically adjusted according to the actual exposure situation of the image. The numerator uses the difference between the pixel value of the luminance image of the image to be fused and the adaptive luminance mean value. The smaller the value (that is, when the two are closest on the numerator), the greater the weight, emphasizing the pixels close to the target luminance, making the adaptive luminance weights more suitable for the exposure level of the current corrected image.

[0056] Based on the above embodiments, the pixel standard deviation is determined based on the following formula: ; wherein, is the pixel mean value of the luminance image of the n + 1th corrected image, is the pixel mean of the luminance image of the (n - 1)-th corrected image, and N is the total number of corrected images, i.e., the total number of images to be fused. is the scale factor.

[0057] Based on the above embodiments, the contrast weight of each corrected image is extracted based on the following steps: Based on the Laplacian operator, calculate the contrast weight of each corrected image.

[0058] Specifically, the contrast weight of each corrected image can be calculated by the following formula: ; where is the Laplacian operator, is the pixel value at the pixel point (x, y) in the n-th corrected image.

[0059] Based on the above embodiments, the saturation weight of each corrected image is extracted based on the following steps: Calculate the maximum pixel value and the minimum pixel value of each pixel point in each corrected image in different color channels; Based on the maximum pixel value and the minimum pixel value, calculate the saturation weight of each corrected image.

[0060] Specifically, the saturation weight of each corrected image can be calculated by the following formula: ; where represents the maximum pixel value in the red channel, green channel, and blue channel at the pixel point (x, y) in the n-th corrected image, represents the minimum pixel value in the red channel, green channel, and blue channel at the pixel point (x, y) in the n-th corrected image.

[0061] Based on the above embodiments, the visual saliency weight of each corrected image is extracted based on the following steps: Calculate the gradient intensity at each pixel point in the grayscale image of each corrected image; Based on the gradient intensity, calculate the visual saliency weight of each image to be fused.

[0062] Specifically, the visual saliency weight of each corrected image can be calculated by the following formula: , ; where represents the gradient intensity at the pixel point (x, y) in the grayscale image of the n-th corrected image; and They are the grayscale image gradients along the x-axis and y-axis respectively, which can be obtained through Sobel filters.

[0063] Based on the pixel values of each pixel point in the reference image, on the basis of the above embodiments, applying the binary mask corresponding to the moving region in each image to be fused to correct the pixel values of each pixel point in each image to be fused, and obtaining the corrected image of each image to be fused, including: Based on morphological operations, denoise the binary mask to obtain a denoising result; Based on the pixel values of each pixel point in the reference image, apply the denoising result to correct the pixel values of each pixel point in each image to be fused, and obtain the corrected image of each image to be fused.

[0064] Specifically, after obtaining the binary mask, the noise of the binary mask can be removed through morphological operations. By eliminating small noise points and connecting disconnected small regions, a smoother and more accurate binary mask, that is, the denoising result, can be obtained. Among them, the morphological operations include: ; Among them, is the binary mask corresponding to the moving region of the nth image to be fused, is the binary mask after removing noise, that is, the denoising result. imopen is the opening operation (Opening Operation), which first performs an erosion operation and then a dilation operation to remove noise points and smooth the object contour. strel is the structuring element (StructuringElement), which defines the shape and size of the morphological operation. 'disk' means using a disk-shaped structuring element.

[0065] As Figure 2 shown, in the embodiments of the present invention, a multi-exposure image enhancement method based on adaptive weights is also provided. First, obtain multiple images to be fused with different exposure degrees; then perform histogram equalization on each image to be fused to redistribute the brightness values of each pixel point to obtain the equalized image of each image to be fused; thereafter, perform entropy motion detection on the equalized image to obtain a motion mask map, that is, a binary mask; thereafter, based on the pixel values of each pixel point in the reference image, apply the binary mask corresponding to the moving region in each image to be fused to correct the pixel values of each pixel point in each image to be fused, and obtain the corrected image of each image to be fused.

[0066] Thereafter, weight extraction is performed on each corrected image to obtain the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image are fused to obtain the fusion weight of each corrected image.

[0067] Based on each corrected image, a Laplacian pyramid is constructed. Based on the fusion weights of each corrected image, an initial weight map is constructed, and the initial weight map is smoothed by a filter to obtain a smoothed weight map. A Gaussian pyramid is constructed through the smoothed weight map, and a fused image is reconstructed based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

[0068] In summary, in the embodiment of the present invention, a new multi-exposure fusion model is constructed, and four weights are extracted as metrics to guide the fusion process. Based on considering the global weights among the images to be fused with different exposure degrees, a new adaptive weight metric strategy is proposed. The pixel standard deviation of the images to be fused is calculated using a truncation function, the brightness center is dynamically adjusted, and the adaptive brightness weight is extracted according to the actual exposure situation of the images, achieving the best balance between local contrast and overall consistency. In addition, entropy is introduced as a key variable for motion detection, and the dynamic region is identified and adjusted through significant changes in the entropy value, improving the problems of image blurring or detail loss easily caused by traditional methods when dealing with moving objects, providing more accurate and detailed adjustments, and being able to adapt to various complex scenarios. After combining these two methods, multi-exposure fusion can generate high-quality, detail-rich, and ghost-free images, providing a more reliable preprocessing solution for fields such as computer vision.

[0069] As Figure 3 shown, based on the above embodiment, an apparatus for enhancing multi-exposure images based on adaptive weights is provided in the embodiment of the present invention, including: An image acquisition module 31, configured to acquire a plurality of images to be fused with different exposure degrees; A motion detection module 32, configured to calculate the local information entropy of each pixel point in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion region in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; An image correction module 33, configured to correct the pixel value of each pixel point in each image to be fused based on the pixel value of each pixel point in the reference image and by applying a binary mask corresponding to the motion region in each image to be fused, to obtain a corrected image of each image to be fused; A weight extraction and fusion module 34, configured to extract the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fuse the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain the fusion weight of each corrected image; An image fusion module 35 is configured to construct a Laplacian pyramid based on each corrected image, construct a Gaussian pyramid based on the fusion weights of each corrected image, and reconstruct a fused image based on the Laplacian pyramid and the Gaussian pyramid corresponding to each corrected image.

[0070] Specifically, in the multi-exposure image enhancement device based on adaptive weights provided in the embodiments of the present invention, the functions of each module correspond one-to-one to the operation processes of each step in the embodiments of the above method, and the achieved effects are also the same. For details, please refer to the above embodiments, and the embodiments of the present invention will not be elaborated herein.

[0071] Figure 4 An example of a schematic physical structure diagram of an electronic device is shown as Figure 4 As shown, the electronic device may include: a processor (Processor) 410, a communication interface (Communications Interface) 420, a memory (Memory) 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute the multi-exposure image enhancement method based on adaptive weights provided in the above embodiments.

[0072] In addition, when the logic instructions in the above-mentioned memory 430 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (Read-Only Memory, ROM), random access memories (Random Access Memory, RAM), magnetic disks, or optical disks, etc., which can store program codes.

[0073] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-exposure image enhancement method based on adaptive weights provided in the above embodiments.

[0074] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the multi-exposure image enhancement method based on adaptive weights provided in the above embodiments.

[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0076] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-exposure image enhancement method based on adaptive weights, characterized in that Including: Obtain multiple images to be fused with different exposure levels; Calculate the local information entropy of each pixel in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion region in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; Based on the pixel values of each pixel in the reference image, apply the binary mask corresponding to the motion region in each image to be fused to correct the pixel values of each pixel in each image to be fused, and obtain the corrected image of each image to be fused; Extract the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fuse the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain the fusion weight of each corrected image; Based on each corrected image, construct a Laplacian pyramid, based on the fusion weight of each corrected image, construct a Gaussian pyramid, and reconstruct a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

2. The multi-exposure image enhancement method based on adaptive weights according to claim 1, characterized in that The adaptive brightness weight of each corrected image is extracted based on the following steps: Based on the brightness image of each corrected image, the pixel mean value, and the pixel standard deviation of the brightness image, determine the adaptive brightness weight of each corrected image; Among them, the pixel standard deviation uses a truncation function and is determined according to the brightness difference between adjacent corrected images.

3. The multi-exposure image enhancement method based on adaptive weights according to claim 2, wherein The pixel standard deviation is determined based on the following formula: ； Among them, is the pixel standard deviation of the luminance image of the nth corrected image, is the pixel mean of the luminance image of the nth corrected image, is the pixel mean of the luminance image of the (n + 1)th corrected image, is the pixel mean of the luminance image of the (n - 1)th corrected image, and N is the total number of corrected images, is the scaling factor.

4. The multi-exposure image enhancement method based on adaptive weights according to claim 1, wherein The contrast weight of each corrected image is extracted based on the following steps: Based on the Laplacian operator, calculate the contrast weight of each corrected image.

5. The multi-exposure image enhancement method based on adaptive weights according to claim 1, wherein The saturation weight of each corrected image is extracted based on the following steps: Calculate the maximum pixel value and the minimum pixel value of each pixel in each corrected image in different color channels; Based on the maximum pixel value and the minimum pixel value, calculate the saturation weight of each corrected image.

6. The multi-exposure image enhancement method based on adaptive weights according to claim 1, characterized in that The visual saliency weight of each corrected image is extracted based on the following steps: Calculate the gradient intensity at each pixel in the grayscale image of each corrected image; Based on the gradient intensity, calculate the visual saliency weight of each corrected image.

7. The multi-exposure image enhancement method based on adaptive weights according to claim 1, characterized in that, The step of based on the pixel values of each pixel in the reference image, applying the binary mask corresponding to the motion region in each image to be fused to correct the pixel values of each pixel in each image to be fused, and obtaining the corrected image of each image to be fused includes: Based on morphological operations, perform denoising processing on the binary mask to obtain a denoising result; Based on the pixel values of each pixel in the reference image, apply the denoising result to correct the pixel values of each pixel in each image to be fused, and obtain the corrected image of each image to be fused.

8. An apparatus for enhancing multi-exposure images based on adaptive weights, characterized in that, Including: An image acquisition module for obtaining multiple images to be fused with different exposure levels; A motion detection module for calculating the local information entropy of each pixel in each image to be fused, generating an entropy map corresponding to each image to be fused, and detecting the motion region in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; An image correction module, configured to correct the pixel values of each pixel point in each image to be fused based on the pixel values of each pixel point in the reference image and by applying a binary mask corresponding to the moving region in each image to be fused, so as to obtain a corrected image of each image to be fused; A weight extraction and fusion module, configured to extract the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fuse the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain a fusion weight of each corrected image; An image fusion module, configured to construct a Laplacian pyramid based on each corrected image, construct a Gaussian pyramid based on the fusion weight of each corrected image, and reconstruct a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, When the processor executes the program, it implements the multi-exposure image enhancement method based on adaptive weights as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-exposure image enhancement method based on adaptive weights as described in any one of claims 1-7.

Images