Method for hdr image reconstruction based on motion-aware weight mask and pseudo-exposure compensation

Abstract

The application discloses a kind of based on motion perception weight mask and pseudo-exposure compensation HDR image reconstruction method, comprising: selecting reference image from multiple exposure image sequence, the block structure similarity analysis of each exposure image and reference image is carried out, and the structural similarity matrix of each exposure image relative to reference image is obtained;According to structural similarity matrix, the motion region of each exposure image is detected, and the motion detection diagram of each exposure image is generated;The motion detection diagram of each exposure image is handled, and weight mask is obtained, generates pseudo-exposure image using histogram matching method;Based on weight mask, exposure image and pseudo-exposure image are weighted and fused, and the latent image corresponding to each exposure image is generated;Each latent image is evaluated, and the quality evaluation result is handled, and the fusion weight of each latent image is obtained;According to fusion weight, using multi-resolution fusion strategy, each latent image is weighted and fused, and obtains HDR image.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, and more specifically, to a high dynamic range image reconstruction method based on motion-aware weighted masks and pseudo-exposure compensation. Background Technology

[0002] High Dynamic Range (HDR) images are typically obtained by fusing multiple images of the same scene at different exposure levels to restore rich details in both light and dark areas of the scene. Therefore, they are widely used in fields such as digital photography, video surveillance, and autonomous driving.

[0003] However, when moving targets (such as pedestrians or vehicles) are present in the scene, their positions differ across the exposed images. Direct fusion can easily produce ghosting artifacts, affecting image quality. Furthermore, underexposed areas (shadows) and overexposed areas (highlights) in the reference image are often misidentified as moving areas and removed due to a lack of effective texture information. Because of the lack of effective cross-exposure information compensation, these areas remain underexposed or overexposed after fusion, resulting in the loss of shadow and highlight details.

[0004] Therefore, how to effectively suppress motion ghosting while compensating for the lack of information in the extreme exposure areas of the reference image is a technical problem that urgently needs to be solved in the field of dynamic scene multi-exposure image fusion. Summary of the Invention

[0005] This invention addresses the challenges of eliminating motion ghosting in multi-exposure image fusion under dynamic scenes and the problem of exposure distortion remaining after fusion due to underexposed and overexposed areas of the reference image being misjudged as motion areas. It provides an HDR image reconstruction method based on motion-aware weighted masks and pseudo-exposure compensation. By combining motion-aware weighted masks based on block structure similarity with histogram matching pseudo-exposure compensation technology, high dynamic range image reconstruction is achieved.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An HDR image reconstruction method based on motion-aware weighted masks and pseudo-exposure compensation includes the following steps:

[0008] A reference image is selected from the multi-exposure image sequence, and a block structure similarity analysis is performed between each exposure image and the reference image to obtain the structure similarity matrix of each exposure image relative to the reference image;

[0009] Based on the structural similarity matrix of each exposed image relative to the reference image, the motion regions in each exposed image are detected and labeled to generate motion detection maps for each exposed image.

[0010] The motion detection maps of each exposure image are processed to obtain a weighted mask. A histogram matching method is used to generate a pseudo-exposure image with a reference image as input and the exposure image as the target.

[0011] Based on weighted masks, each exposed image and its corresponding pseudo-exposed image are weighted and fused to generate a potential image corresponding to each exposed image.

[0012] The quality of each potential image is evaluated based on multiple dimensions of indicators. The quality evaluation results of multiple dimensions are multiplied and normalized to obtain the fusion weight of each potential image.

[0013] Based on the fusion weights of each potential image, a multi-resolution fusion strategy is adopted to perform weighted fusion on each potential image to obtain an HDR image.

[0014] Further, a reference image is selected from the multi-exposure image sequence, specifically as follows:

[0015] First, sort the multiple exposure images with different exposure levels according to their brightness values ​​to ensure that the exposure images in the multi-exposure image sequence are arranged from dark to light according to their brightness values;

[0016] Then, a window convolution operation is performed on each exposed image in the multi-exposure image sequence. The local average brightness value of each pixel is calculated using a convolution window of a specified size. The total number of underexposed and overexposed pixels in each exposed image is counted. Based on the total number of underexposed and overexposed pixels, the exposure quality of the exposed image is scored.

[0017] Finally, the exposure image with the lowest exposure quality score, i.e., the exposure image with the fewest underexposed and overexposed pixels, is selected as the reference image.

[0018] Furthermore, block structure similarity analysis is performed on each exposed image and the reference image to obtain the structure similarity matrix of each exposed image relative to the reference image, specifically:

[0019] Set the block size, perform local mean filtering on the reference image and the current exposure image respectively, calculate the mean, variance and covariance within the block, and then calculate the block-level structural similarity;

[0020] By limiting the structural similarity calculation results to the range of [-1,1], the structural similarity matrix of each exposed image relative to the reference image is obtained.

[0021] Furthermore, based on the structural similarity matrix of each exposed image relative to the reference image, the moving regions in each exposed image are detected and labeled to generate motion detection maps for each exposed image, specifically:

[0022] A threshold is set to distinguish between static and moving pixels. The structural similarity value of each block in the structural similarity matrix is ​​traversed. When the structural similarity value of a block is greater than the threshold, the region corresponding to that block is determined to be a static region; otherwise, it is determined to be a moving region. Based on this, motion detection maps for each exposure image are generated.

[0023] Furthermore, before motion detection is performed on each exposed image, the extreme exposure areas of the reference image are forcibly rendered static, specifically as follows:

[0024] First, identify the underexposed and overexposed areas in the reference image, and mark the underexposed and overexposed areas as forced still areas;

[0025] In motion detection of each exposed image, for pixel positions that fall within the forced stillness area, regardless of whether there is motion between them and the reference image, the pixel value of the corresponding position in the exposed image is directly used, and motion detection judgment based on structural similarity is no longer performed.

[0026] Furthermore, the motion detection maps of each exposed image are processed to obtain a weighted mask, specifically:

[0027] After obtaining the motion detection map, a morphological dilation of the motion region is performed using a dilation kernel of a specified size to expand the boundary of the motion region and eliminate false detections and boundary artifacts. Finally, the dilated motion detection map is subjected to a Gaussian smoothing filter of a specified standard deviation to soften the hard boundaries and obtain a weighted mask map. The weighted mask map uses continuous values ​​to represent the static confidence of each pixel, where 1 represents static and 0 represents motion.

[0028] Furthermore, a histogram matching method is used, with the reference image as input and the exposed image as the target, to generate a pseudo-exposure image, specifically as follows:

[0029] Expand the original N exposure images to N images are the original exposure images. For each exposure image that is not a reference image, the reference image is used as the source image, and the histogram distribution of the exposure image is used as the target. The histogram matching algorithm is used to map the pixel intensity of the reference image to the histogram distribution of the target exposure to generate a pseudo exposure image. The pseudo exposure image retains the content features of the reference image, including object shape, detail texture and motion state, and has the same exposure level and brightness distribution as the exposure image. The generated pseudo exposure image is processed by pixel value truncation to ensure that the output value range is within [0,1].

[0030] Furthermore, based on a weighted mask, each exposed image and its corresponding pseudo-exposed image are weighted and fused to generate a potential image corresponding to each exposed image, specifically:

[0031] For a reference image, its latent image is itself; for each exposure image of a non-reference image, a latent image is synthesized pixel by pixel using a weighted mask as the fusion weight; in static regions, the latent image retains the true content of the original exposure image; in moving regions, the latent image is replaced by a pseudo-exposure image.

[0032] Furthermore, the quality of each potential image is evaluated based on multiple dimensions of indicators. The quality evaluation results of multiple dimensions are multiplied and normalized to obtain the fusion weight of each potential image, specifically:

[0033] The quality of each potential image is evaluated from three dimensions: contrast, saturation, and exposure mode. The quality evaluation results of the three dimensions are multiplied point by point and normalized to obtain the fusion weight of each potential image at each pixel position.

[0034] After converting the potential image into a grayscale image, the Laplacian operator is used to filter the grayscale image, and the absolute value of the filtered response is taken as the contrast quality evaluation value of each pixel.

[0035] For each potential image, the standard deviation of the pixel values ​​in the R, G, and B channels relative to their mean is calculated as the saturation quality assessment value.

[0036] The normalized pixel values ​​of each channel are evaluated using a Gaussian function centered on the intermediate exposure value to obtain the exposure quality assessment value.

[0037] The quality assessment values ​​of contrast, saturation, and exposure mode are multiplied point by point to obtain the comprehensive quality weight of each potential image:

[0038] Subsequently, the overall quality weights of all potential images are normalized along the pixel dimension to obtain the fusion weights of each potential image.

[0039] Furthermore, based on the fusion weights of each potential image, a multi-resolution fusion strategy is adopted to perform weighted fusion of each potential image at different scales to obtain an HDR image, specifically:

[0040] The number of pyramid layers K is adaptively determined based on the spatial resolution of the potential image:

[0041] For each potential image, a K-layer Laplacian pyramid is constructed, and the corresponding normalized fusion weight map is used to construct a K-layer Gaussian pyramid. At each layer, the fusion weights for all potential images are normalized.

[0042] At each level of the pyramid, the Laplacian coefficients of all potential images are weighted and summed:

[0043] Finally, starting from the top of the fusion pyramid, the Laplacian coefficients of the corresponding layers are upsampled and superimposed to complete the image reconstruction. The pixel values ​​are then truncated to the range of [0,1] to obtain a high dynamic range reconstruction result without artifact interference.

[0044] Compared with existing technologies, this invention adopts a block-level structural similarity motion detection mechanism to accurately identify the boundaries of moving regions, construct a motion-aware adaptive weight mask, and combine it with morphological dilation and Gaussian smoothing to effectively eliminate ghosting artifacts while fully preserving the texture details of static regions.

[0045] This invention marks underexposed and overexposed areas in the reference image as forced stationary areas during the motion detection stage to prevent them from being misidentified as moving areas. Simultaneously, through histogram matching pseudo-exposure compensation technology, pseudo-exposure images matching each exposed image are generated, enabling these extremely exposed areas to obtain effective information from other exposed images. This, in turn, restores shadow and highlight details in the fusion result, fully leveraging the wide dynamic range advantage of multi-exposure fusion.

[0046] This invention comprehensively considers the synergistic effect of image contrast, color saturation, and exposure mode on the fusion weight allocation. It combines contrast quality assessment, saturation quality assessment, exposure mode quality assessment, and a multi-resolution Laplacian pyramid weighted fusion strategy for image reconstruction, ensuring that the fusion result achieves optimal balance in terms of brightness, color, and detail.

[0047] This invention effectively eliminates ghosting artifacts introduced by moving objects in dynamic scenes, and solves the problem of local exposure distortion caused by missing information in underexposed and overexposed areas of the reference image. It accurately generates high dynamic range images with rich details and natural transitions, and has important theoretical, practical, and promotional application value. Attached Figure Description

[0048] Figure 1 This is a flowchart of an HDR image reconstruction method based on motion-aware weighted masks and pseudo-exposure compensation.

[0049] Figure 2 This is a schematic diagram of an HDR image reconstruction method based on motion-aware weighted masks and pseudo-exposure compensation.

[0050] Figure 3 This is a schematic diagram of a static scene.

[0051] Figure 4 This is a schematic diagram of a dynamic scene. Detailed Implementation

[0052] The following description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the HDR image reconstruction method based on motion-aware weighted masks and pseudo-exposure compensation of the present invention.

[0053] Please see Figure 1 and Figure 2 This invention discloses an HDR image reconstruction method based on motion-aware weighted masks and pseudo-exposure compensation, comprising the following steps:

[0054] S1: Select a reference image from the multi-exposure image sequence, perform block structure similarity analysis between each exposure image and the reference image, and obtain the structure similarity matrix of each exposure image relative to the reference image;

[0055] S2: Based on the structural similarity matrix of each exposed image relative to the reference image, the motion regions in each exposed image are detected and labeled to generate motion detection maps for each exposed image;

[0056] S3: Process the motion detection maps of each exposure image to obtain a weight mask, and use the histogram matching method to generate a pseudo-exposure image with the reference image as input and the exposure image as the target.

[0057] S4: Based on the weighted mask, each exposed image and its corresponding pseudo-exposed image are weighted and fused to generate a potential image corresponding to each exposed image;

[0058] S5: Evaluate the quality of each potential image based on multiple dimensions of indicators, multiply the quality evaluation results of multiple dimensions and normalize them to obtain the fusion weight of each potential image.

[0059] S6: Based on the fusion weights of each potential image, a multi-resolution fusion strategy is adopted to perform weighted fusion of each potential image to obtain an HDR image.

[0060] Step S1: Select a reference image from the multi-exposure image sequence, perform block structure similarity analysis between each exposure image and the reference image, and obtain the structure similarity matrix of each exposure image relative to the reference image.

[0061] Input a multi-exposure image sequence, and the system will automatically select the optimal reference image from the sequence.

[0062] Specifically, firstly, multiple exposure images with different exposure levels are sorted according to their brightness values ​​to ensure that the exposure images in the multi-exposure image sequence are arranged from darkest to brightest; then, a window convolution operation is performed on each exposure image in the multi-exposure image sequence, using a specified... A convolution window of a certain size is used to calculate the local average brightness value of each pixel. The total number of underexposed and overexposed pixels in each exposed image is counted. The average brightness of underexposed pixels is lower than the underexposure threshold. The average brightness of overexposed pixels exceeds the overexposed threshold. The exposure quality of the exposed image is scored based on the total number of underexposed and overexposed pixels. Finally, the exposed image with the lowest exposure quality score, i.e., the image with the fewest underexposed and overexposed pixels, is selected as the reference image. This reference image contains the richest detail information and the best color characteristics for subsequent motion detection.

[0063] Block structure similarity analysis is performed on each exposed image and the reference image to obtain the structure similarity matrix of each exposed image relative to the reference image.

[0064] Specifically, the block size is set to For each pixel, local mean filtering is applied to both the reference image and the currently exposed image to calculate the mean, variance, and covariance within the block, and then the block-level structural similarity is calculated.

[0065] The formula for calculating structural similarity is as follows:

[0066] ;

[0067] In the formula, Represents the local standard deviation of the reference image; This represents the local standard deviation of the currently exposed image; This represents the covariance between the reference image and the currently exposed image; To prevent small constants with denominators of zero; values ​​are taken as follows: By limiting the structural similarity calculation results to the range of [-1,1], the structural similarity matrix of each exposed image relative to the reference image is obtained.

[0068] Step S2: Based on the structural similarity matrix of each exposed image relative to the reference image, the motion regions in each exposed image are detected and labeled to generate motion detection maps for each exposed image.

[0069] Specifically, a threshold τ is set to distinguish between static and moving pixels. The structural similarity value of each block in the structural similarity matrix (i.e., the element value in the matrix) is traversed. When the structural similarity value of a block is greater than the threshold τ, the region corresponding to that block is determined to be a static region; otherwise, it is determined to be a moving region. Based on this, a binary motion detection map of each exposure image is generated, providing a basis for subsequent weight mask generation and image fusion.

[0070] Before detecting and labeling the moving regions in each exposed image, considering that the reference image lacks effective structural information in the underexposed and overexposed areas, the extreme exposure areas of the reference image are forcibly staticized.

[0071] Specifically, the first step is to identify underexposed areas (pixel values ​​below the underexposed threshold) in the reference image. ) and overexposed areas (pixel values ​​exceeding the overexposed threshold) Underexposed and overexposed areas are marked as forced stillness regions. During motion detection in each exposed image, for pixel positions falling within these forced stillness regions, regardless of whether there is motion between the pixel and the reference image, the pixel value of the corresponding position in that exposed image is directly used, without further motion detection based on structural similarity. The purpose of this is to fully utilize the effective information from other exposed images in the underexposed and overexposed areas of the reference image, bypassing the information deficiencies of the reference image and ensuring the detail integrity and color accuracy of the fusion result in these key areas.

[0072] Step S3: Process the motion detection maps of each exposure image to obtain a weighted mask. Use the histogram matching method, with the reference image as input and the exposure image as the target, to generate a pseudo-exposure image.

[0073] After obtaining the motion detection map, a morphological dilation kernel of size ω3×ω3 is used to dilate the motion region to expand its boundaries and eliminate false detections and boundary artifacts. Finally, the dilated motion detection map is smoothed by a Gaussian smoothing filter with a standard deviation of σ1 to soften hard boundaries, resulting in a weighted mask map with a corresponding filter window size of [missing information]. This weighted mask map uses continuous values ​​to represent the stationary confidence of each pixel, where 1 indicates stationary and 0 indicates motion, providing guidance for subsequent image fusion and block weighting.

[0074] To compensate for the content differences between the exposed images of the reference image and the non-reference image, a pseudo-exposure image is generated using a histogram matching method.

[0075] Specifically, the original N exposure images are expanded to The dataset consists of N images, where the first N images are the original exposure images. For each exposure image i that is not a reference image, the reference image is used as the reference image. The source image is the i-th exposed image. Using the histogram distribution of the reference image as the target, a histogram matching algorithm is employed to map the pixel intensity of the reference image to the histogram distribution of the target exposure, thereby generating a pseudo-exposure image. The pseudo-exposure image retains the content features of the reference image, including object shape, detailed texture, and motion state, while also possessing the same exposure level and brightness distribution as the i-th exposed image. Its mathematical expression is:

[0076] ;

[0077] in, The cumulative histogram function for the reference image; Let be the cumulative histogram function for the i-th exposed image. It is its inverse function. The generated pseudo-exposure image is truncated by pixel values ​​to ensure that the output value range is within [0,1].

[0078] Step S4: Based on the weight mask, each exposed image and its corresponding pseudo-exposed image are weighted and fused to generate a potential image corresponding to each exposed image.

[0079] Specifically, for a reference image, its latent image is itself; for each exposure image i of a non-reference image, the latent image is synthesized pixel by pixel using the weight mask m(x,y) as the fusion weight, according to the following formula:

[0080] ;

[0081] In the formula, Let i be the potential image corresponding to the i-th exposed image; This is the i-th original exposure image; Let be the pseudo-exposure image corresponding to the i-th original exposure image.

[0082] In the stationary region (m=1), the latent image retains the true content of the original exposed image; in the moving region (m=0), the latent image is replaced by a pseudo-exposed image.

[0083] Since the content structure of the pseudo-exposure image is consistent with that of the reference image, ghosting artifacts introduced by moving targets can be effectively eliminated while preserving the brightness characteristics of the exposure image. This ultimately yields a potential image set equal in number to the multi-exposure image sequence, providing ghost-free input for subsequent multi-exposure fusion.

[0084] Step S5: Evaluate the quality of each potential image according to multiple dimensions, multiply the quality evaluation results of multiple dimensions and normalize them to obtain the fusion weight of each potential image.

[0085] Multidimensional quality measurement and fusion weight calculation: The quality of each potential image is evaluated from three dimensions: contrast, saturation and exposure mode. The quality evaluation results of the three dimensions are multiplied point by point and normalized to obtain the fusion weight of each potential image at each pixel position, which is used for subsequent multi-resolution fusion.

[0086] Contrast quality assessment: After converting the latent image to grayscale, the grayscale image is filtered using the Laplacian operator. The absolute value of the filtered response is taken as the contrast quality assessment value for each pixel, reflecting the richness of local details in the neighborhood of that pixel. That is, for the i-th latent image, its contrast quality assessment value is:

[0087] ;

[0088] In the formula, The grayscale value of the pixel in the grayscale image corresponding to the potential image. Represents the Laplace operator;

[0089] Saturation quality assessment: For each potential image, the standard deviation of the pixel values ​​in the R, G, and B channels relative to their mean is calculated as the saturation quality assessment value.

[0090] ;

[0091] Among them, R i G is the pixel value of the red channel of the i-th potential image at that pixel; i B is the pixel value of the green channel of the i-th potential image at that pixel; i Let be the pixel value of the blue channel of the i-th potential image at that pixel; This represents the average value of the three pixel channels; the higher the saturation, the richer the color information of the pixel.

[0092] Exposure moderateness quality assessment: The normalized pixel values ​​of each channel are evaluated using a Gaussian function centered at 0.5 (the intermediate exposure value, i.e., the intermediate brightness level that is neither too dark nor too overexposed after normalization), to obtain the exposure moderateness quality assessment value:

[0093] ;

[0094] in, This represents an exponential function with the natural constant e as its base, used to map a deviation of 0.5 from the median exposure value to a weighted value in the interval (0,1]. The red channel pixel values ​​of the potential image are normalized to [0,1]. The green channel pixel values ​​of the potential image are normalized to [0,1]. The blue channel pixel values ​​of the potential image are normalized to [0,1]; σ2 is the Gaussian scaling parameter for moderate exposure, σ2=0.2. The closer the pixel value is to the median exposure value of 0.5, the higher the weight; the weight of overexposed or underexposed areas approaches zero.

[0095] Weighted fusion generation: The quality evaluation values ​​of the three dimensions of contrast, saturation, and exposure mode are multiplied point by point to obtain the comprehensive quality weight of each potential image. :

[0096] ;

[0097] Then, the overall quality weights of all potential images are normalized along the pixel dimension to ensure that the sum of the overall quality weights at each pixel is 1, thus obtaining the fusion weights of each potential image:

[0098] ;

[0099] in, To prevent small constants with a denominator of zero, the normalized fusion weights This reflects the relative reliability of each potential image at each pixel location and will serve as input for the next step of multi-resolution Laplacian pyramid fusion.

[0100] Step S6: Based on the fusion weights of each potential image, a multi-resolution fusion strategy is adopted to perform weighted fusion on each potential image to obtain an HDR image.

[0101] Multi-resolution Laplacian pyramid fusion: A multi-resolution fusion strategy is adopted to perform weighted fusion of potential images at different scales to obtain high dynamic range fusion results with rich details and natural transitions.

[0102] Adaptive determination of the number of pyramid layers: The number of pyramid layers K is adaptively determined based on the spatial resolution of the latent image.

[0103] ;

[0104] Where H is the height of the potential image; W is the width of the potential image; ensuring that the pyramid decomposition does not exceed the image size limit.

[0105] For each potential image Constructing a K-layer Laplace pyramid The corresponding normalized fusion weight graph Constructing a K-layer Gaussian pyramid And at each layer, the fusion weights for all potential images are normalized:

[0106] ;

[0107] in, This represents the normalized fusion weight of the i-th potential image at pixel position (x,y) in the k-th layer; The original fusion weight of the i-th latent image at pixel position (x,y) in layer k; N represents the total number of latent images participating in the fusion, i,j=1,2,…,N; k represents the pyramid level, k=0,1,… Ensure the sum of weights for each layer is 1 to avoid brightness deviations in the fusion result. At each layer k of the pyramid, the Laplacian coefficients of all potential images are weighted and summed:

[0108] ;

[0109] Among them, F k(x,y) represents the Laplacian coefficient at pixel position (x,y) in the k-th layer of the fused pyramid; Let represent the Laplacian coefficient of the i-th potential image at pixel position (x,y) in layer k.

[0110] Finally, starting from the top of the fusion pyramid, the Laplacian coefficients of the corresponding layers are upsampled and superimposed to complete the image reconstruction. The pixel values ​​are then truncated to the range of [0,1] to obtain a high dynamic range reconstruction result without artifact interference.

[0111] Figure 3 and Figure 4 The paper presents a comparison of the processing effects of the proposed method with three traditional mainstream multi-exposure fusion methods. Figure 3 The image shows a static church scene. The left side displays three images with different exposure levels, while the right side shows the fusion results of four methods. It can be seen that all four methods achieved good fusion results and restored the details and colors in the scene well. Figure 4 As a dynamic scene, pedestrians move within the scene during the multi-exposure shooting process.

[0112] like Figure 4 As shown in the red dashed box, T. Mertens' method produced obvious ghosting artifacts in the pedestrian area, Zhang Wei's method did not produce obvious ghosting, but the details of distant buildings were blurred and the details of clouds in the sky were severely lost, and Kede Ma's method produced white halos at strong edges.

[0113] In contrast, the method proposed in this invention not only effectively avoids ghosting artifacts in pedestrian movement areas, but also clearly preserves the details of clouds in the sky and the outline textures of distant buildings. It achieves good fusion effects in both underexposed and overexposed areas, realistically restoring the details and colors in the scene.

[0114] For the two scenarios above, the parameter settings of this invention are shown in Table 1. Experiments have shown that this set of parameter values ​​is well applicable to most scenarios.

[0115] Table 1 Experimental Parameter Settings

[0116]

[0117] In summary, this invention employs a block-level structural similarity motion detection mechanism to accurately identify the boundaries of moving regions, constructs a motion-aware adaptive weight mask, and combines morphological dilation and Gaussian smoothing to effectively eliminate ghosting artifacts while fully preserving the texture details of static regions.

[0118] This invention marks underexposed and overexposed areas in the reference image as forced stationary areas during the motion detection stage to prevent them from being misidentified as moving areas. Simultaneously, through histogram matching pseudo-exposure compensation technology, pseudo-exposure images matching each exposed image are generated, enabling these extremely exposed areas to obtain effective information from other exposed images. This, in turn, restores shadow and highlight details in the fusion result, fully leveraging the wide dynamic range advantage of multi-exposure fusion.

[0119] This invention comprehensively considers the synergistic effect of image contrast, color saturation, and exposure mode on the fusion weight allocation. It combines contrast quality assessment, saturation quality assessment, exposure mode quality assessment, and a multi-resolution Laplacian pyramid weighted fusion strategy for image reconstruction, ensuring that the fusion result achieves optimal balance in terms of brightness, color, and detail.

[0120] This invention effectively eliminates ghosting artifacts introduced by moving objects in dynamic scenes, and solves the problem of local exposure distortion caused by missing information in underexposed and overexposed areas of the reference image. It accurately generates high dynamic range images with rich details and natural transitions, and has important theoretical, practical, and promotional application value.

[0121] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit disclosed in the present invention should fall within the patent scope covered by the present invention.

Claims

1. A method for HDR image reconstruction based on motion-aware weighted masks and pseudo-exposure compensation, characterized in that, Includes the following steps: A reference image is selected from the multi-exposure image sequence, and a block structure similarity analysis is performed between each exposure image and the reference image to obtain the structure similarity matrix of each exposure image relative to the reference image. Based on the structural similarity matrix of each exposed image relative to the reference image, the motion regions in each exposed image are detected and labeled to generate motion detection maps for each exposed image. The motion detection maps of each exposure image are processed to obtain a weighted mask. A histogram matching method is used to generate a pseudo-exposure image with a reference image as input and the exposure image as the target. Based on weighted masks, each exposed image and its corresponding pseudo-exposed image are weighted and fused to generate a potential image corresponding to each exposed image. The quality of each potential image is evaluated based on multiple dimensions of indicators. The quality evaluation results of multiple dimensions are multiplied and normalized to obtain the fusion weight of each potential image. Based on the fusion weights of each potential image, a multi-resolution fusion strategy is adopted to perform weighted fusion on each potential image to obtain an HDR image.

2. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, Selecting a reference image from a multi-exposure image sequence specifically involves: First, sort the multiple exposure images with different exposure levels according to their brightness values ​​to ensure that the exposure images in the multi-exposure image sequence are arranged from dark to light according to their brightness values; Then, a window convolution operation is performed on each exposed image in the multi-exposure image sequence. The local average brightness value of each pixel is calculated using a convolution window of a specified size. The total number of underexposed and overexposed pixels in each exposed image is counted. Based on the total number of underexposed and overexposed pixels, the exposure quality of the exposed image is scored. Finally, the exposure image with the lowest exposure quality score, i.e., the exposure image with the fewest underexposed and overexposed pixels, is selected as the reference image.

3. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, Block structure similarity analysis was performed on each exposed image and the reference image to obtain the structure similarity matrix of each exposed image relative to the reference image, as follows: Set the block size, perform local mean filtering on the reference image and the current exposure image respectively, calculate the mean, variance and covariance within the block, and then calculate the block-level structural similarity; By limiting the structural similarity calculation results to the range of [-1,1], the structural similarity matrix of each exposed image relative to the reference image is obtained.

4. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, Based on the structural similarity matrix of each exposed image relative to the reference image, motion regions in each exposed image are detected and labeled to generate motion detection maps for each exposed image, specifically: A threshold is set to distinguish between static and moving pixels. The structural similarity value of each block in the structural similarity matrix is ​​traversed. When the structural similarity value of a block is greater than the threshold, the region corresponding to that block is determined to be a static region; otherwise, it is determined to be a moving region. Based on this, motion detection maps for each exposure image are generated.

5. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, Before detecting and labeling moving regions in each exposed image, the extreme exposure regions of the reference image are forced to remain still, specifically as follows: First, identify the underexposed and overexposed areas in the reference image, and mark the underexposed and overexposed areas as forced still areas; In motion detection of each exposed image, for pixel positions that fall within the forced stillness area, regardless of whether there is motion between them and the reference image, the pixel value of the corresponding position in the exposed image is directly used, and motion detection judgment based on structural similarity is no longer performed.

6. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, The motion detection maps of each exposed image are processed to obtain a weighted mask, specifically as follows: After obtaining the motion detection map, a morphological dilation of the motion region is performed using a dilation kernel of a specified size to expand the boundary of the motion region and eliminate false detections and boundary artifacts. Finally, the dilated motion detection map is subjected to a Gaussian smoothing filter of a specified standard deviation to soften the hard boundaries and obtain a weighted mask map. The weighted mask map uses continuous values ​​to represent the static confidence of each pixel, where 1 represents static and 0 represents motion.

7. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, Using a histogram matching method, with a reference image as input and the exposed image as the target, a pseudo-exposure image is generated, specifically as follows: Expand the original N exposure images to N images are the original exposure images. For each exposure image that is not a reference image, the reference image is used as the source image, and the histogram distribution of the exposure image is used as the target. The histogram matching algorithm is used to map the pixel intensity of the reference image to the histogram distribution of the target exposure to generate a pseudo exposure image. The pseudo exposure image retains the content features of the reference image, including object shape, detail texture and motion state, and has the same exposure level and brightness distribution as the exposure image. The generated pseudo exposure image is processed by pixel value truncation to ensure that the output value range is within [0,1].

8. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, Based on a weighted mask, each exposed image and its corresponding pseudo-exposed image are weighted and fused to generate a potential image corresponding to each exposed image, specifically: For a reference image, its latent image is itself; for each exposure image of a non-reference image, a latent image is synthesized pixel by pixel using a weighted mask as the fusion weight. In the static region, the latent image retains the true content of the original exposed image; In the motion region, the potential image is replaced with a false exposure image.

9. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, The quality of each potential image is evaluated based on multiple dimensions of indicators. The results of the quality evaluations from multiple dimensions are multiplied and normalized to obtain the fusion weight of each potential image, specifically: The quality of each potential image is evaluated from three dimensions: contrast, saturation, and exposure mode. The quality evaluation results of the three dimensions are multiplied point by point and normalized to obtain the fusion weight of each potential image at each pixel position. After converting the potential image into a grayscale image, the Laplacian operator is used to filter the grayscale image, and the absolute value of the filtered response is taken as the contrast quality evaluation value of each pixel. For each potential image, the standard deviation of the pixel values ​​in the R, G, and B channels relative to their mean is calculated as the saturation quality assessment value. The normalized pixel values ​​of each channel are evaluated using a Gaussian function centered on the intermediate exposure value to obtain the exposure quality assessment value. The quality assessment values ​​of contrast, saturation, and exposure mode are multiplied point by point to obtain the comprehensive quality weight of each potential image: Subsequently, the overall quality weights of all potential images are normalized along the pixel dimension to obtain the fusion weights of each potential image.

10. The HDR image reconstruction method based on motion-aware weighted mask and pseudo-exposure compensation according to claim 1, characterized in that, Based on the fusion weights of each potential image, a multi-resolution fusion strategy is adopted to perform weighted fusion of each potential image at different scales to obtain an HDR image, specifically: The number of pyramid layers K is adaptively determined based on the spatial resolution of the potential image: For each potential image, a K-layer Laplacian pyramid is constructed, and the corresponding normalized fusion weight map is used to construct a K-layer Gaussian pyramid. At each layer, the fusion weights for all potential images are normalized. At each level of the pyramid, the Laplacian coefficients of all potential images are weighted and summed: Finally, starting from the top of the fusion pyramid, the Laplacian coefficients of the corresponding layers are upsampled and superimposed to complete the image reconstruction. The pixel values ​​are then truncated to the range of [0,1] to obtain a high dynamic range reconstruction result without artifact interference.

Images