Multi-exposure image enhancement method and device based on adaptive weight

Abstract

The present invention relates to the field of image processing technology, and provides a multi-exposure image enhancement method and device based on adaptive weights. By extracting the adaptive brightness weight of each image to be fused, different brightness weights can be assigned to pixels in areas with different exposure levels. By fusing the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each image to be fused, it can be ensured that the details and color information of each image to be fused are optimally preserved and displayed. By calculating the local information entropy to detect motion areas, artifacts can be better processed and the image fusion effect can be improved. The pyramid fusion method can effectively retain image details of different scales, and at the same time smoothly fuse multiple input images, reducing artifacts or false edges that may occur during the fusion process.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a multi-exposure image enhancement method and device based on adaptive weighting. Background Art

[0002] Objects in natural scenes have vastly different illumination levels, which can only be captured within a wide dynamic range. However, current imaging devices, such as cameras, have a very limited detection range, often losing image information in underexposed or overexposed scenes, severely degrading image quality.

[0003] Multi-exposure image fusion can help solve the above problems by integrating images at different exposure levels. However, the fused image effect is often unsatisfactory. It not only leads to the loss of scene details, but also may produce noise or color distortion, and cause artifacts or false edges in the fusion process. Summary of the Invention

[0004] The present invention provides a multi-exposure image enhancement method and device based on adaptive weighting, which are used to solve the defects in the prior art.

[0005] The present invention provides a multi-exposure image enhancement method based on adaptive weights, comprising:

[0006] Acquire multiple images to be fused at different exposure levels;

[0007] Calculate the local information entropy of each pixel in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion area in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image;

[0008] Based on the pixel value of each pixel in the reference image, applying the binary mask corresponding to the motion area in each image to be fused, correcting the pixel value of each pixel in each image to be fused, to obtain a corrected image of each image to be fused;

[0009] Extracting the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fusing the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain a fusion weight of each corrected image;

[0010] Based on each corrected image, a Laplacian pyramid is constructed, based on the fusion weight of each corrected image, a Gaussian pyramid is constructed, and based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image, a fused image is reconstructed.

[0011] According to the multi-exposure image enhancement method based on adaptive weights provided by the present invention, the adaptive brightness weight of each corrected image is extracted based on the following steps:

[0012] determining an adaptive brightness weight for each corrected image based on a brightness image of each corrected image, a pixel mean value of the brightness image, and a pixel standard deviation;

[0013] The pixel standard deviation is determined by using a truncation function according to the brightness difference between adjacent corrected images.

[0014] According to a multi-exposure image enhancement method based on adaptive weights provided by the present invention, the pixel standard deviation is determined based on the following formula:

[0015] ;

[0016] in, is the pixel standard deviation of the brightness image of the nth corrected image, is the pixel mean of the brightness image of the nth corrected image, is the pixel mean of the brightness image of the n+1th corrected image, is the pixel mean of the brightness image of the n-1th corrected image, N is the total number of corrected images, is the scale factor.

[0017] According to the adaptive weighted multi-exposure image enhancement method provided by the present invention, the contrast weight of each corrected image is extracted based on the following steps:

[0018] Based on the Laplacian operator, the contrast weight of each corrected image is calculated.

[0019] According to the adaptive weighted multi-exposure image enhancement method provided by the present invention, the saturation weight of each corrected image is extracted based on the following steps:

[0020] Calculate the maximum pixel value and the minimum pixel value of each pixel in each corrected image in different color channels;

[0021] A saturation weight of each modified image is calculated based on the maximum pixel value and the minimum pixel value.

[0022] According to the adaptive weighted multi-exposure image enhancement method provided by the present invention, the visual saliency weight of each corrected image is extracted based on the following steps:

[0023] Calculate the gradient intensity at each pixel in the grayscale image of each corrected image;

[0024] Based on the gradient strength, a visual saliency weight of each corrected image is calculated.

[0025] According to the adaptive weighted multi-exposure image enhancement method provided by the present invention, based on the pixel value of each pixel in the reference image, a binary mask corresponding to the motion area in each image to be fused is applied to correct the pixel value of each pixel in each image to be fused, thereby obtaining a corrected image of each image to be fused, including:

[0026] Based on morphological operations, denoising is performed on the binary mask to obtain a denoising result;

[0027] Based on the pixel value of each pixel in the reference image, the denoising result is applied to correct the pixel value of each pixel in each image to be fused, so as to obtain a corrected image of each image to be fused.

[0028] The present invention also provides a multi-exposure image enhancement device based on adaptive weights, comprising:

[0029] An image acquisition module, used to acquire multiple images to be fused at different exposure levels;

[0030] A motion detection module is used to calculate the local information entropy of each pixel in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion area in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image;

[0031] An image correction module is configured to correct the pixel values ​​of each pixel in each image to be fused based on the pixel values ​​of each pixel in the reference image and apply a binary mask corresponding to the motion region in each image to be fused to obtain a corrected image of each image to be fused;

[0032] A weight extraction and fusion module is used to extract the adaptive brightness weight, contrast weight, saturation weight and visual saliency weight of each corrected image, and fuse the adaptive brightness weight, contrast weight, saturation weight and visual saliency weight of each corrected image to obtain the fusion weight of each corrected image;

[0033] The image fusion module is used to construct a Laplacian pyramid based on each corrected image, construct a Gaussian pyramid based on the fusion weight of each corrected image, and reconstruct a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the adaptive weight-based multi-exposure image enhancement method described above is implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described multi-exposure image enhancement methods based on adaptive weights.

[0036] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-described methods for multi-exposure image enhancement based on adaptive weights.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The adaptive weighted multi-exposure image enhancement method and device provided by the present invention detects motion regions by calculating local information entropy. Using a binary mask corresponding to the motion region in each image to be fused, the pixel values ​​of each pixel in each image to be fused are corrected, effectively handling artifacts and improving image fusion. By extracting adaptive brightness weights from each corrected image, pixels in regions with different exposure levels can be assigned different brightness weights. By extracting visual saliency weights from each corrected image, the degree of attention paid to each pixel in the human visual system is considered from a visual saliency perspective, thereby optimizing the fusion effect. By fusing the adaptive brightness, contrast, saturation, and visual saliency weights of each corrected image, the details and color information of each corrected image are optimally preserved and displayed. The pyramid fusion method effectively preserves image details at different scales while smoothly fusing multiple corrected images, reducing artifacts and false edges that may occur during the fusion process. By processing image information separately at different scales, a better balance between local details and global structure is achieved, resulting in a fusion result with enhanced visual quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings in the following description without any creative work.

[0040] Figure 1 This is one of the flow charts of the adaptive weighted multi-exposure image enhancement method provided by the present invention;

[0041] Figure 2 This is the second flow chart of the adaptive weight-based multi-exposure image enhancement method provided by the present invention;

[0042] Figure 32 is a schematic structural diagram of a multi-exposure image enhancement device based on adaptive weighting provided by the present invention;

[0043] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0045] Figure 1 FIG. 1 is a flow chart of a multi-exposure image enhancement method based on adaptive weights provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0046] S1, obtaining multiple images to be fused at different exposure levels;

[0047] S2, calculating the local information entropy of each pixel in each image to be fused, generating an entropy map corresponding to each image to be fused, and detecting the motion area in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image;

[0048] S3, based on the pixel value of each pixel in the reference image, applying the binary mask corresponding to the motion area in each image to be fused, and correcting the pixel value of each pixel in each image to be fused to obtain a corrected image of each image to be fused;

[0049] S4, extracting the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fusing the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain a fusion weight of each corrected image;

[0050] S5: constructing a Laplacian pyramid based on each corrected image, constructing a Gaussian pyramid based on the fusion weight of each corrected image, and reconstructing a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

[0051] Specifically, the adaptive weight-based multi-exposure image enhancement method provided in an embodiment of the present invention is executed by a multi-exposure image enhancement device based on adaptive weights. The device can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, a tablet, etc., which is not specifically limited here.

[0052] First, step S1 is performed to obtain multiple images to be fused at different exposure levels. The multiple images to be fused are a series of color images at different exposure levels acquired by a low dynamic range (LDR) imaging device. The total number of images to be fused may be N, where N is greater than or equal to 2.

[0053] Then, step S2 is executed. In practice, when multiple images are fused into a high dynamic range image, they are often affected by camera shake or object movement, which results in ghosting effects after fusion. Traditional image fusion methods are prone to image blur or loss of detail when processing moving objects. To address this problem, local information entropy is calculated to detect moving areas, thereby better handling artifacts and improving the image fusion effect.

[0054] By calculating the local information entropy of each pixel in each image to be fused, an entropy map corresponding to each image to be fused is generated.

[0055] Each image to be fused can be first subjected to histogram equalization to redistribute the brightness values ​​of each pixel, providing a more balanced brightness distribution for images with different exposure levels. For color images, histogram equalization can be applied to each color channel separately to improve color saturation and visual quality.

[0056] When performing histogram equalization, the frequency of each gray level in the image to be fused can be calculated first. For each gray level , the cumulative distribution function is: ;

[0057] in, Grayscale The cumulative distribution function of Is grayscale probability.

[0058] Then, redistribute the gray levels in the image to be fused, and Mapping to new grayscale The formula is as follows: ;

[0059] Where L is the total number of gray levels, usually 256. The round function is used for rounding.

[0060] By performing histogram equalization on each image to be fused, the grayscale distribution in each image to be fused can be made more uniform, eliminating the influence of pixel difference on motion detection.

[0061] After that, each image to be fused is converted into a grayscale image to reduce the data dimension, eliminate the interference of color information, and simplify the subsequent entropy calculation. The conversion formula is as follows:

[0062] ;

[0063] in, is the grayscale image of the nth image to be fused, 、 and are the pixel values ​​of the red channel, green channel, and blue channel at the pixel point (x, y) in the nth image to be fused.

[0064] Local entropy reflects the texture complexity and information content of an image by measuring the randomness of pixel values ​​within an image block. A larger entropy value indicates less certainty about the pixel values ​​within the block and greater information content. By calculating the local entropy of each pixel in a grayscale image, details and textures within the image can be effectively detected. The formula for calculating local entropy is as follows:

[0065] ;

[0066] in, is the local information entropy of the nth grayscale image at the pixel (x, y), is the probability that pixel value i in the nth grayscale image appears in the local window. The local area can be set as needed, for example, it can be a window of 3×3, 5×5 or 7×7 pixels.

[0067] Subsequently, the local information entropy of different grayscale images at the pixel point (x, y) is normalized, and the calculated local information entropy is mapped to a standard range [0, 1] to eliminate the scale differences between different grayscale images and facilitate subsequent processing and comparison.

[0068] Therefore, the entropy map corresponding to each image to be fused can be determined according to the local information entropy of the grayscale image of each image to be fused at different pixel points (x, y).

[0069] Thereafter, the motion region in each image to be fused may be detected using the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image.

[0070] The reference image can be the intermediate image of each image to be fused. The difference between the entropy map corresponding to the other images to be fused and the entropy map corresponding to the reference image is calculated to detect the motion area, which is usually manifested as a significant change in the entropy value. The calculation formula is as follows:

[0071] ;

[0072] in, is the local information entropy at the pixel point (x, y) in the entropy map corresponding to the nth image to be fused, is the local information entropy at the pixel point (x, y) in the entropy map corresponding to the reference image, is the difference between the entropy map corresponding to the nth image to be fused and the entropy map corresponding to the reference image at the pixel point (x, y).

[0073] Then, the Otsu method is used to perform binarization and detect the motion area of ​​each image to be fused. Each motion area of ​​the image to be fused can generate a corresponding binary mask, that is,

[0074] ;

[0075] in, is the value of the binary mask corresponding to the motion region of the nth image to be fused at the pixel point (x, y). Threshold is the specified threshold value, which can be set as needed and is not specifically limited here.

[0076] Thereafter, step S3 is executed to correct the pixel value of each pixel in each image to be fused using the pixel value of each pixel in the reference image and applying the binary mask corresponding to the motion area in each image to be fused to obtain a corrected image of each image to be fused.

[0077] Among them, the pixel values ​​of each pixel in each image to be fused are corrected using the generated binary mask combined with the pixel values ​​of each pixel in the reference image. This can retain the details of the static area and only adjust the moving area. The mathematical formula is as follows:

[0078] .

[0079] in, is the pixel value at the pixel point (x, y) in the nth image to be fused. is the pixel value at the pixel point (x, y) in the reference image.

[0080] The corrected image obtained after correction effectively detects and adjusts the motion area, reducing the ghosting effect caused by camera shake or object movement.

[0081] Thereafter, step S4 is executed to extract the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image. The adaptive brightness weight can be determined based on the brightness image of each corrected image. The adaptive brightness weight is an adaptive weight, meaning that the brightness weight varies with the brightness values ​​of different pixels in each corrected image. This allows pixels in areas with different exposure levels to have different brightness weights.

[0082] Contrast weighting measures the magnitude of pixel value changes across each corrected image. Saturation weighting measures the color purity across each pixel within the corrected image, with higher saturation weighting indicating more vivid colors. By extracting contrast and saturation weighting, we can better preserve the texture and color information of each corrected image, enhancing the visual quality of each corrected image.

[0083] The visual saliency weight is used to measure the importance of each pixel in different corrected images.

[0084] The adaptive brightness, contrast, saturation, and visual saliency weights of each corrected image are then fused to obtain a fused weight for each corrected image. The fusion process considers the importance of the adaptive brightness, contrast, saturation, and visual saliency weights, balancing their contributions through parameter adjustment to ensure optimal preservation and presentation of detail and color information in each corrected image.

[0085] The calculation formula of fusion weight is as follows:

[0086] ;

[0087] in, is the fusion weight at the pixel (x, y) in the nth corrected image, is the adaptive brightness weight of the pixel (x, y) in the nth corrected image, is the contrast weight of the pixel (x, y) in the nth corrected image, is the saturation weight of the pixel (x, y) in the nth corrected image, is the visual saliency weight of the pixel (x, y) in the nth corrected image, 、 、 、 are the importance parameters of the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight at the pixel point (x, y) in the nth corrected image, respectively. They can be set according to actual needs and are not specifically limited here.

[0088] Afterwards, the fusion weights of each corrected image can be normalized to ensure that the fusion weights at the same pixel points in each corrected image are 1, that is:

[0089] ;

[0090] in, is the normalized result of the fusion weight at the pixel (x, y) in the nth corrected image.

[0091] Finally, step S5 is performed. Using a multi-resolution decomposition method, the corrected image of each corrected image is decomposed into pyramid structures of different scales to construct a Laplacian pyramid. The fusion weights of each corrected image are decomposed into pyramid results of different scales to construct a Gaussian pyramid. The Gaussian pyramid represents the low-frequency information of each corrected image, while the Laplacian pyramid represents the high-frequency details of each corrected image.

[0092] The Laplacian pyramid and Gaussian pyramid corresponding to each corrected image are used to preserve image details at different scales and reconstruct a fused image. Here, the Laplacian and Gaussian pyramids can be fused first. At each pyramid level, the Gaussian pyramid is multiplied with the Laplacian pyramid. The multiplication results at the same pyramid level corresponding to each corrected image are then added together to obtain a fused pyramid at each pyramid level.

[0093] The formula for fusion pyramid is as follows:

[0094] ;

[0095] in, It is l A fusion pyramid on each pyramid level, is the first corrected image corresponding to l Gaussian pyramids on pyramid levels, is the first corrected image corresponding to l Laplacian pyramid with 2 pyramid levels.

[0096] By using the fusion pyramid, the fused image can be reconstructed to achieve image enhancement of each image to be fused.

[0097] The adaptive weighted multi-exposure image enhancement method provided in embodiments of the present invention detects motion regions by calculating local information entropy. Using a binary mask corresponding to the motion region in each image to be fused, the pixel values ​​of each pixel in each image to be fused are corrected. This method effectively handles artifacts and improves image fusion. By extracting adaptive brightness weights from each corrected image, pixels in regions with different exposure levels can be assigned different brightness weights. By extracting visual saliency weights from each corrected image, the degree of attention paid to each pixel in the human visual system is considered from a visual saliency perspective, thereby optimizing the fusion effect. By fusing the adaptive brightness, contrast, saturation, and visual saliency weights of each corrected image, the details and color information of each corrected image are optimally preserved and displayed. The pyramid fusion method effectively preserves image details at different scales while smoothly fusing multiple corrected images, reducing artifacts and false edges that may occur during the fusion process. By processing image information separately at different scales, a better balance between local details and global structure is achieved, resulting in a fusion result with enhanced visual quality.

[0098] Based on the above embodiment, the adaptive brightness weight of each corrected image is extracted based on the following steps:

[0099] determining an adaptive brightness weight for each corrected image based on a brightness image of each corrected image, a pixel mean value of the brightness image, and a pixel standard deviation;

[0100] The pixel standard deviation is determined by using a truncation function according to the brightness difference between adjacent corrected images.

[0101] Specifically, the traditional brightness weight calculation formula is as follows:

[0102] ;

[0103] in, is the brightness weight of the pixel (x, y) in the traditional nth image to be fused, is the pixel value at the pixel point (x, y) in the brightness image of the nth image to be fused, that is, the brightness value at the pixel point (x, y) in the nth image to be fused. The nth image to be fused can be normalized and converted to the YCbCr color space. Extracting the Y channel alone is the brightness image. However, the calculation coefficient of the Y channel is the same as the grayscale coefficient, so it can also be considered that the image after grayscale conversion of the nth image to be fused is the brightness image. In the above formula, 0.5 represents a medium brightness value. This value is used as the target brightness, so that pixels with brightness close to the middle value have a higher weight, retaining more details; Represents a fixed parameter that controls the width of the Gaussian distribution. The value will make the weight distribution more concentrated around 0.5, while a larger A larger value will make the weight distribution smoother.

[0104] Traditional brightness weighting cannot preserve more texture and details in brighter areas of underexposed images or darker areas of overexposed images. Therefore, in the embodiments of the present invention, when extracting the adaptive brightness weight of each corrected image, the brightness image, the pixel mean value, and the pixel standard deviation of each corrected image can be used to determine the adaptive brightness weight of each corrected image, that is,

[0105] ;

[0106] in, is the ideal exposure value, is the pixel value at the pixel point (x, y) in the brightness image of the nth corrected image, that is, the brightness value at the pixel point (x, y) in the nth corrected image, is the pixel mean of the brightness image of the nth corrected image, that is, the brightness mean. is the pixel standard deviation of the brightness image of the nth corrected image, that is, the brightness standard deviation.

[0107] The pixel standard deviation can be determined using a truncation function based on the brightness differences between adjacent corrected images. This truncation function makes the brightness weight calculation adaptive to different exposure conditions. Larger pixel standard deviations indicate a significant difference between the average brightness of the corrected image and the average brightness of adjacent corrected images, indicating a larger exposure difference between adjacent corrected images. Consequently, the distribution of adaptive brightness weights will be wider to capture more detail.

[0108] The above-mentioned adaptive brightness weight can better handle different exposure conditions, improve the smoothness of the transition area, and reduce the discontinuity and artifacts caused by extreme exposure (too bright or too dark); the modified numerator adjusts the brightness center. Instead of a single fixed value of 0.5, it dynamically adjusts the adaptive brightness center value according to the actual exposure of the image. The numerator adopts the difference between the pixel value of the brightness image of the image to be fused and the adaptive brightness mean. The smaller the value (that is, when the two are closest on the numerator), the greater the weight, emphasizing pixels close to the target brightness, so that the adaptive brightness weight is more suitable for the exposure level of the current corrected image.

[0109] Based on the above embodiment, the pixel standard deviation is determined based on the following formula:

[0110] ;

[0111] in, is the pixel mean of the brightness image of the n+1th corrected image, is the pixel mean of the brightness image of the n-1th corrected image, N is the total number of corrected images, that is, the total number of images to be fused, is the scale factor.

[0112] Based on the above embodiment, the contrast weight of each corrected image is extracted based on the following steps:

[0113] Based on the Laplacian operator, the contrast weight of each corrected image is calculated.

[0114] Specifically, the contrast weight of each corrected image can be calculated by the following formula:

[0115] ;

[0116] in, is the Laplace operator, is the pixel value at the pixel point (x, y) in the nth corrected image.

[0117] Based on the above embodiment, the saturation weight of each corrected image is extracted based on the following steps:

[0118] Calculate the maximum pixel value and the minimum pixel value of each pixel in each corrected image in different color channels;

[0119] A saturation weight of each modified image is calculated based on the maximum pixel value and the minimum pixel value.

[0120] Specifically, the saturation weight of each corrected image can be calculated by the following formula:

[0121] ;

[0122] in, Represents the maximum pixel value in the red channel, green channel, and blue channel at the pixel point (x, y) in the nth corrected image, Represents the minimum pixel value in the red channel, green channel, and blue channel at the pixel point (x, y) in the nth corrected image.

[0123] Based on the above embodiment, the visual saliency weight of each corrected image is extracted based on the following steps:

[0124] Calculate the gradient intensity at each pixel in the grayscale image of each corrected image;

[0125] Based on the gradient strength, the visual saliency weight of each image to be fused is calculated.

[0126] Specifically, the visual saliency weight of each corrected image can be calculated by the following formula:

[0127] , ;

[0128] in, Represents the gradient intensity at the pixel (x, y) in the grayscale image of the nth corrected image; and are the grayscale image gradients along the x-axis and y-axis, respectively, which can be obtained by Sobel filter.

[0129] On the basis of the above embodiment, based on the pixel value of each pixel in the reference image, the binary mask corresponding to the motion area in each image to be fused is applied to correct the pixel value of each pixel in each image to be fused, so as to obtain a corrected image of each image to be fused, including:

[0130] Based on morphological operations, denoising is performed on the binary mask to obtain a denoising result;

[0131] Based on the pixel value of each pixel in the reference image, the denoising result is applied to correct the pixel value of each pixel in each image to be fused, so as to obtain a corrected image of each image to be fused.

[0132] Specifically, after obtaining the binary mask, the noise of the binary mask can be removed through morphological operations. By eliminating small noise points and disconnected small areas, a smoother and more accurate binary mask, i.e., the denoising result, is obtained. Among them, morphological operations include:

[0133] ;

[0134] in, is the binary mask corresponding to the motion area of ​​the nth image to be fused, is the binary mask used to remove noise, i.e., the denoising result. imopen is an opening operation, which performs an erosion followed by a dilation operation to remove noise and smooth object contours. strel is a structuring element, which defines the shape and size of the morphological operation. 'disk' indicates the use of a disk-shaped structuring element.

[0135] like Figure 2As shown, an embodiment of the present invention also provides a multi-exposure image enhancement method based on adaptive weights, which first obtains multiple images to be fused with different exposure levels; then performs histogram equalization on each image to be fused, redistributes the brightness value of each pixel, and obtains an equalized image of each image to be fused; thereafter, entropy motion detection is performed on the equalized image to obtain a motion mask map, that is, a binary mask; thereafter, based on the pixel value of each pixel in the reference image, the binary mask corresponding to the motion area in each image to be fused is applied to correct the pixel value of each pixel in each image to be fused, and obtain a corrected image of each image to be fused.

[0136] Afterwards, weight extraction is performed on each corrected image to obtain the adaptive brightness weight, contrast weight, saturation weight and visual saliency weight of each corrected image, and the adaptive brightness weight, contrast weight, saturation weight and visual saliency weight of each corrected image are fused to obtain the fusion weight of each corrected image.

[0137] Based on each corrected image, a Laplacian pyramid is constructed. Based on the fusion weight of each corrected image, an initial weight map is constructed. The initial weight map is smoothed by a filter to obtain a smoothed weight map. A Gaussian pyramid is constructed through the smoothed weight map. Based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image, a fused image is reconstructed.

[0138] In summary, a new multi-exposure fusion model is constructed in the embodiment of the present invention, and four weights are extracted as metrics to guide the fusion process. On the basis of considering the global weights between the images to be fused with different exposure levels, a new adaptive weight measurement strategy is proposed, which uses a truncation function to calculate the pixel standard deviation of the image to be fused, dynamically adjusts the brightness center, and extracts adaptive brightness weights according to the actual exposure of the image, thereby achieving the best balance between local contrast and overall consistency. In addition, entropy is introduced as a key variable for motion detection, and dynamic areas are identified and adjusted through significant changes in entropy values. This improves the problem of image blur or detail loss that is easily caused by traditional methods when processing moving objects, provides more precise and detailed adjustments, and can adapt to a variety of complex scenes. The combination of these two methods enables multi-exposure fusion to generate high-quality, detail-rich and ghost-free images, providing a more reliable preprocessing solution for fields such as computer vision.

[0139] like Figure 3 As shown, based on the above embodiment, an embodiment of the present invention provides a multi-exposure image enhancement device based on adaptive weights, including:

[0140] An image acquisition module 31 is used to acquire multiple images to be fused at different exposure levels;

[0141] A motion detection module 32 is configured to calculate the local information entropy of each pixel in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect motion areas in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image;

[0142] An image correction module 33 is configured to correct the pixel values ​​of each pixel in each image to be fused based on the pixel values ​​of each pixel in the reference image and apply the binary mask corresponding to the motion region in each image to be fused to obtain a corrected image of each image to be fused;

[0143] a weight extraction and fusion module 34 for extracting the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fusing the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain a fusion weight for each corrected image;

[0144] The image fusion module 35 is configured to construct a Laplacian pyramid based on each corrected image, construct a Gaussian pyramid based on the fusion weight of each corrected image, and reconstruct a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image.

[0145] Specifically, the functions of each module in the adaptive weight-based multi-exposure image enhancement device provided in the embodiment of the present invention correspond one-to-one to the operational procedures of each step in the embodiment of the above method, and the effects achieved are also consistent. Please refer to the above embodiment for details, and no further details will be given in the embodiment of the present invention.

[0146] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call logic instructions in the memory 430 to execute the adaptive weight-based multi-exposure image enhancement method provided in the above embodiments.

[0147] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0148] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the adaptive weight-based multi-exposure image enhancement method provided in the above embodiments.

[0149] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the adaptive weight-based multi-exposure image enhancement method provided in the above embodiments.

[0150] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0151] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multi-exposure image enhancement method based on adaptive weights, characterized in that: include: Acquire multiple images to be fused at different exposure levels; Calculate the local information entropy of each pixel in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion area in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; Based on the pixel value of each pixel in the reference image, applying the binary mask corresponding to the motion area in each image to be fused, correcting the pixel value of each pixel in each image to be fused, to obtain a corrected image of each image to be fused; Extracting the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image, and fusing the adaptive brightness weight, contrast weight, saturation weight, and visual saliency weight of each corrected image to obtain a fusion weight of each corrected image; Based on each corrected image, a Laplacian pyramid is constructed, based on the fusion weight of each corrected image, a Gaussian pyramid is constructed, and based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image, a fused image is reconstructed; The calculation formula of the local information entropy is as follows: ； in, is the local information entropy of the nth grayscale image at the pixel (x, y), is the probability that pixel value i in the nth grayscale image appears in the local window; The calculation formula for the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image is as follows: ； in, is the local information entropy at the pixel point (x, y) in the entropy map corresponding to the reference image, is the difference between the entropy map corresponding to the nth image to be fused and the entropy map corresponding to the reference image at the pixel point (x, y); The calculation formula of the binary mask corresponding to the motion area of ​​each image to be fused is as follows: ； in, is the value of the binary mask corresponding to the motion area of ​​the nth image to be fused at the pixel point (x, y), and Threshold is the specified threshold; The mathematical formula for the corrected image of each image to be fused is as follows: ； in, is the pixel point (x, y) in the corrected image of the nth image to be fused, is the pixel value at the pixel point (x, y) in the nth image to be fused, is the pixel value at the pixel point (x, y) in the reference image.

2. The adaptive weighted multi-exposure image enhancement method according to claim 1, wherein: The adaptive brightness weight of each corrected image is extracted based on the following steps: determining an adaptive brightness weight for each corrected image based on a brightness image of each corrected image, a pixel mean value of the brightness image, and a pixel standard deviation; The pixel standard deviation is determined by using a truncation function according to the brightness difference between adjacent corrected images.

3. The adaptive weighted multi-exposure image enhancement method according to claim 2, wherein: The pixel standard deviation is determined based on the following formula: ； in, is the pixel standard deviation of the brightness image of the nth corrected image, is the pixel mean of the brightness image of the nth corrected image, is the pixel mean of the brightness image of the n+1th corrected image, is the pixel mean of the brightness image of the n-1th corrected image, N is the total number of corrected images, is the scale factor.

4. The method for multi-exposure image enhancement based on adaptive weighting according to claim 1, wherein: The contrast weight of each corrected image is extracted based on the following steps: Based on the Laplacian operator, the contrast weight of each corrected image is calculated.

5. The method for multi-exposure image enhancement based on adaptive weighting according to claim 1, wherein: The saturation weight of each corrected image is extracted based on the following steps: Calculate the maximum pixel value and the minimum pixel value of each pixel in each corrected image in different color channels; A saturation weight of each modified image is calculated based on the maximum pixel value and the minimum pixel value.

6. The adaptive weighted multi-exposure image enhancement method according to claim 1, wherein: The visual saliency weight of each corrected image is extracted based on the following steps: Calculate the gradient intensity at each pixel in the grayscale image of each corrected image; Based on the gradient strength, a visual saliency weight of each corrected image is calculated.

7. The adaptive weighted multi-exposure image enhancement method according to claim 1, wherein: The method comprises: applying a binary mask corresponding to a motion region in each image to be fused based on a pixel value of each pixel in the reference image to correct the pixel value of each pixel in each image to be fused to obtain a corrected image of each image to be fused, including: Based on morphological operations, denoising is performed on the binary mask to obtain a denoising result; Based on the pixel value of each pixel in the reference image, the denoising result is applied to correct the pixel value of each pixel in each image to be fused, so as to obtain a corrected image of each image to be fused.

8. A multi-exposure image enhancement device based on adaptive weighting, characterized in that: include: An image acquisition module, used to acquire multiple images to be fused at different exposure levels; A motion detection module is used to calculate the local information entropy of each pixel in each image to be fused, generate an entropy map corresponding to each image to be fused, and detect the motion area in each image to be fused based on the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image; An image correction module is configured to correct the pixel values ​​of each pixel in each image to be fused based on the pixel values ​​of each pixel in the reference image and apply a binary mask corresponding to the motion region in each image to be fused to obtain a corrected image of each image to be fused; A weight extraction and fusion module is used to extract the adaptive brightness weight, contrast weight, saturation weight and visual saliency weight of each corrected image, and fuse the adaptive brightness weight, contrast weight, saturation weight and visual saliency weight of each corrected image to obtain the fusion weight of each corrected image; An image fusion module is used to construct a Laplacian pyramid based on each corrected image, construct a Gaussian pyramid based on the fusion weight of each corrected image, and reconstruct a fused image based on the Laplacian pyramid and Gaussian pyramid corresponding to each corrected image; The calculation formula of the local information entropy is as follows: ； in, is the local information entropy of the nth grayscale image at the pixel (x, y), is the probability that pixel value i in the nth grayscale image appears in the local window; The calculation formula for the difference between the entropy map corresponding to each image to be fused and the entropy map corresponding to the reference image is as follows: ； in, is the local information entropy at the pixel point (x, y) in the entropy map corresponding to the reference image, is the difference between the entropy map corresponding to the nth image to be fused and the entropy map corresponding to the reference image at the pixel point (x, y); The calculation formula of the binary mask corresponding to the motion area of ​​each image to be fused is as follows: ； in, is the value of the binary mask corresponding to the motion area of ​​the nth image to be fused at the pixel point (x, y), and Threshold is the specified threshold; The mathematical formula for the corrected image of each image to be fused is as follows: ； in, is the pixel point (x, y) in the corrected image of the nth image to be fused, is the pixel value at the pixel point (x, y) in the nth image to be fused, and is the pixel value at the pixel point (x, y) in the reference image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the adaptive weight-based multi-exposure image enhancement method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the adaptive weight-based multi-exposure image enhancement method according to any one of claims 1 to 7 is implemented.

Images