A low-contrast image joint enhancement and super-resolution reconstruction method

Abstract

The application discloses a low-contrast image joint enhancement and super-resolution reconstruction method, which comprises the following steps: firstly, performing multi-scale Gaussian filtering on the low-contrast image; secondly, expanding the imaging dynamic range of the quotient image by using an S function; thirdly, performing color correction on the expansion result to obtain a contrast-improved image; and finally, optimizing the resolution and detail information of the contrast-improved image by a blind super-resolution reconstruction model constrained by a total variation to obtain a detail information and resolution improvement result. The low-contrast image joint enhancement and super-resolution reconstruction can help to overcome the influence of image adverse factors, such as low contrast / resolution and fuzzy details, and lay a foundation for image feature extraction and image interpretation.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology and is applicable to various imaging modal data. Specifically, it relates to a method for joint enhancement and super-resolution reconstruction of low-contrast images. Background Technology

[0002] Scientific data shows that 75% of the information humans receive from the outside world comes from the human visual system, and images are the primary source of information for the visual system. However, due to various factors (such as insufficient lighting, inclement weather, time constraints, and limitations of imaging equipment performance), image quality deteriorates (e.g., low contrast / resolution, blurred details), failing to achieve the desired effect. Degraded images affect both the intuitive evaluation of the image and higher-level image processing (such as feature extraction and image interpretation). Therefore, it is necessary to process degraded images to improve their quality, including but not limited to improving image contrast / resolution, optimizing details, and removing noise.

[0003] Image enhancement aims to emphasize the overall or local characteristics of an image, highlight regions of interest, and reduce or filter out irrelevant parts, thereby enriching image information and improving image interpretation / recognition performance (S. Bettahar, et al. PDE-based enhancement of color images in RGB space. IEEE Trans. Image Processing, 2012, 21(5), 2500-2512.). Image enhancement methods are mainly divided into methods based on traditional histogram stretching / equalization, fuzzy theory, multi-scale analysis, human vision, mathematical morphology, and Retinex theory. Each of these algorithms has its own advantages and disadvantages. Relatively speaking, image enhancement based on Retinex theory has a more ideal overall effect. However, the Retinex algorithm is prone to loss of detail, color distortion, and halo phenomena when processing low-contrast images.

[0004] Furthermore, the aforementioned image enhancement algorithms struggle to improve image resolution (such as spatial resolution), while image resolution measures the richness of detail contained within an image. Compared to low-resolution images, high-resolution images possess higher pixel density, richer texture details, and higher reliability (Q. Ma, et al. Deep unfolding network for spatiospectral image super-resolution. IEEE Trans. Computational Imaging, 2022, 8, 28-40.). Super-resolution reconstruction can reconstruct small-sized low-resolution images into large-sized high-resolution images, a technique already applied in remote sensing imaging, medical imaging, and security monitoring (Q. Zhang, et al. Collaborative network for super-resolution and semantic segmentation of remote sensing images. IEEE Trans. Geoscience & Remote Sensing, 2021, DOI: 10.1109 / TGRS.2021.3099300.).

[0005] Super-resolution reconstruction techniques are mainly divided into three categories: interpolation-based methods, reconstruction-based methods, and learning-based methods. However, interpolation-based methods often lead to blurry and jagged reconstructed images; reconstruction-based methods rely on the construction of constraint terms and the accuracy of image registration, making them unsuitable for large magnification factors. Learning-based methods can solve some bottleneck problems in traditional techniques, but they rely on the assumption that the distribution of the test set and training set are consistent. When the degradation model of the training set is inconsistent with that of the test set (or when the degradation model is unknown), the reconstruction results often suffer from significant degradation. Meanwhile, current super-resolution reconstruction techniques are ineffective for low-contrast images, necessitating the development of joint enhancement and super-resolution reconstruction methods for low-contrast and low-resolution images to overcome or compensate for problems such as image blurring, low quality, and indistinct regions of interest caused by limitations in the image acquisition system or environment. Summary of the Invention

[0006] This invention addresses the aforementioned technical problems of existing image enhancement and super-resolution reconstruction methods by providing a joint enhancement and super-resolution reconstruction method for low-contrast images. This method first expands the imaging dynamic range of low-contrast images using an S-function, and then employs a fully variational constrained blind super-resolution reconstruction model to optimize resolution and detail information, thereby obtaining improved contrast, detail, and resolution.

[0007] A method for joint enhancement and super-resolution reconstruction of low-contrast images includes the following steps:

[0008] Step 1: Perform multi-scale Gaussian filtering on the low-contrast image I(x,y), where (x,y) represents the pixel coordinates;

[0009] Step 2: Calculate the pixel-by-pixel ratio between the low-contrast image and the Gaussian-filtered image after multi-scale Gaussian filtering to obtain the quotient image u. i (x,y), where i is the index of the Gaussian function for different scales of Gaussian filtering;

[0010] Step 3, for the commercial map u i First, a normalization transformation is performed on (x, y) to obtain the normalized result graph v. i (x,y), then normalize the result graph v i The S-function transformation of (x,y) is performed to further obtain the weighted imaging dynamic extension map R(x,y);

[0011] Step 4: Perform color correction on the weighted imaging dynamic spread map R(x,y) obtained in Step 3 to obtain the color-corrected map q. j (x,y), j=1,…,H, where H is the number of channels in the image.

[0012] Step 5: Based on the enhancement result Q obtained in Step 4, Q = {q} j After inputting (x,y)} into the blind super-resolution reconstruction model, the super-resolution reconstruction result Y and the fuzzy degradation kernel estimate K are output. Based on the super-resolution reconstruction result Y and the fuzzy degradation kernel estimate K, the optimized super-resolution reconstruction result X is obtained by performing total variational constraints.

[0013] As described above, the multi-scale Gaussian filtering in step 1 is based on the following formula:

[0014]

[0015] Among them, h i (x,y) represents the Gaussian filtered image corresponding to the i-th Gaussian function, g i (x,y) represents the multi-scale Gaussian filter function, (x,y) represents the pixel coordinates, and σ i This represents the standard deviation of the Gaussian function.

[0016] As mentioned above, the standard deviation σ of Gaussian functions at different scales in step 1 i Based on the following formula:

[0017] σ i =A i ×max(I(x,y))

[0018] The standard deviation σ of Gaussian functions at different scalesi different.

[0019] As described above in step 2, the commercial diagram u i (x,y) is based on the following formula:

[0020] u i (x,y)=I(x,y) / (h i (x,y)+ε)

[0021] Where ε is a positive number.

[0022] The normalized result graph v in step 2 as described above i (x,y) is based on the following formula:

[0023]

[0024] Among them, v i (x,y) is the normalized result graph, max(u i (x,y)) represents the maximum value of the pixel amplitude in the graph, min(u i (x,y)) represents the minimum value of the pixel amplitude in the business image.

[0025] As mentioned above, the S-function transformation in step 3 is based on the following formula:

[0026]

[0027] Where m and n are constants.

[0028] As described above, the weighted imaging dynamic extension map R(x,y) in step 3 is based on the following formula:

[0029]

[0030] Where, ω i is a constant that is not less than 0 and represents the weights of the Gaussian filtering results at different scales. B is the total number of Gaussian functions for Gaussian filtering at different scales.

[0031] As described above, in step 4, the color correction image q j (x,y) is based on the following formula:

[0032]

[0033] Among them, I j (x,y) is the j-th image channel of the low-contrast image I(x,y), R j (x,y) is the j-th image channel of the imaging dynamic extension map R(x,y), and F is the mapping function.

[0034] The expression for the total variational constraint, as described above, is:

[0035] min‖X‖ TV +λ‖XY‖ TV stKX=Q,

[0036] Where λ is a tradeoff parameter and is greater than or equal to 1, ||·|| TV It is a 1-norm.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. Based on Retinex theory, an improved multi-scale Retinex algorithm is proposed. It uses the S-function to replace the logarithmic function in Retinex theory and combines it with color correction to effectively eliminate color inversion. While improving image contrast, it also improves problems such as blurry details, resulting in better image visual effects.

[0039] 2. To further improve image details and spatial resolution, image enhancement and super-resolution reconstruction are combined to construct a unified framework for image enhancement and super-resolution reconstruction. On the one hand, this can solve the problem that single image enhancement algorithms are difficult to improve image spatial resolution. On the other hand, it can also overcome the shortcomings of single super-resolution reconstruction algorithms in dealing with low-contrast images. It simultaneously optimizes the contrast, resolution and detail information of the image, which is helpful for image interpretation and analysis.

[0040] 3. Based on the theory of super-resolution reconstruction, a blind super-resolution reconstruction model with total variation constraints is proposed when the degradation mechanism of high-resolution images is unknown. First, the degradation mechanism is estimated through the blind super-resolution reconstruction model. Second, the advantages of learning-based and reconstruction-based methods are integrated to establish the total variation consistency constraint of high-resolution and low-resolution image pairs. Then, the quality of the reconstructed image is further improved by minimizing the total variation constraint. Attached Figure Description

[0041] Figure 1 The flowchart of this invention includes four steps: 1. Performing multi-scale Gaussian filtering on the low-contrast image; 2. Normalizing the original image and the multi-scale filtering result, and using an S-function to expand the imaging dynamic range of the normalized result quotient; 3. Performing color correction on the output result to obtain an image with improved contrast; 4. Optimizing the resolution and detail information of the contrast-enhanced image through a blind super-resolution reconstruction model with total variation constraints to obtain improved detail information and resolution results.

[0042] Figure 2The images show the results of joint enhancement and super-resolution reconstruction of low-contrast images. A represents the original low-contrast image, B represents the enhancement result of image A, and C represents the super-resolution reconstruction result of image B. Images A and B have the same resolution, 255×510, while image C has a resolution four times that of A and B, at 1020×2040. Detailed Implementation

[0043] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0044] A method for joint enhancement and super-resolution reconstruction of low-contrast images includes the following steps:

[0045] Step 1, multi-scale Gaussian filtering, aims to overcome phenomena such as halo and unclear local details that may be caused by single-scale Gaussian filtering.

[0046] Multi-scale Gaussian filtering of a low-contrast image I(x,y) can be expressed as follows:

[0047]

[0048] Among them, h i (x, y) represents the i-th Gaussian function (with standard deviation σ). i The corresponding Gaussian filtered image, g i (x,y) represents the multi-scale Gaussian filter function, (x,y) represents the pixel coordinates, and σ i Let represent the standard deviation of the Gaussian function (also known as the scaling parameter), and ...

[0049] The standard deviation σ of Gaussian functions at different scales i Based on the following formula, the standard deviation σ of Gaussian functions at different scales... i different:

[0050] σ i =A i ×max(I(x,y))

[0051] The standard deviation σ of the Gaussian function i The size determines the effective range of the convolution kernel. When σ iWhen the value is small, image details can be enhanced well, but the colors of the output result are easily distorted; when σ is small... i When the value is large, the image color fidelity is high, but the local details of the output result are not clear, and there is obvious halo phenomenon at strong edges. Therefore, in this embodiment, three different scale parameters, large, medium, and small, are selected, and their expression can be expressed as follows:

[0052] σ1=A1×max(I(x,y)),σ2=A2×max(I(x,y)), σ3=A3×max(I(x,y)), (2) where σ1, σ2 and σ3 represent three scale parameters: large, medium and small; max(I(x,y)) represents the maximum pixel value of image I; A1, A2 and A3 represent three constants, and A1>A2>A3>0, and the value ranges of A1, A2 and A3 can be [0.7,0.9], [0.4,0.6] and [0.1,0.3], respectively.

[0053] Step 2, quotient calculation, aims to obtain the pixel-by-pixel ratio of the original image to the filtered result.

[0054] Calculate the original low-contrast image with different Gaussian functions (standard deviation σ). i The pixel-by-pixel ratio result of the Gaussian-filtered image after Gaussian filtering, i.e., the quotient image u. i (x,y), the calculation formula can be expressed as u i (x,y)=I(x,y) / (h i (x,y)+ε), where ε is a very small positive number to avoid a denominator of zero, such as ε=10. -4 ~10 -8 .

[0055] Step 3, S-function transformation, aims to expand the dynamic range of imaging.

[0056] The quotient diagram u obtained in step 2 i (x,y) is first normalized, and its expression can be represented as:

[0057]

[0058] Among them, v i (x,y) is the normalized result graph, max(u i (x,y)) represents the maximum value of the pixel amplitude in the graph, min(u i (x,y)) represents the minimum value of the pixel amplitude in the business image.

[0059] Then, the normalized result image v i The S-function transformation of (x,y) can be expressed as:

[0060]

[0061] Where m and n are constants, ensuring r i The range of values ​​for (x, y) is [0, 1]. The ranges of values ​​for m and n are [1, 3] and [-2, 2], respectively.

[0062] The weighted dynamic extension map R(x,y) is obtained, and its expression can be represented as follows:

[0063]

[0064] Where, ω i ω is a constant not less than 0, where i = 1, 2, 3, representing the weights of the Gaussian filtering results at different scales. i The choice of can be adjusted based on experimental results. B is the total number of Gaussian functions for different scale Gaussian filtering. In this embodiment, B is 3.

[0065] Step 4, color correction, aims to eliminate color inversion and other phenomena.

[0066] Color correction is performed on the weighted imaging dynamic spread map R(x,y) obtained in step 3, and its expression can be represented as follows:

[0067]

[0068] Among them, I j (x,y) is the j-th image channel of the low-contrast image I(x,y), R j (x,y) represents the j-th image channel of the imaging dynamic extension map R(x,y), q j (x,y) represents the color correction image, and F is the mapping function, which can be the S function from step 3; c j is the color correction factor, j = 1, ..., H, where H is the number of channels in the image. If the low-contrast image I is a color image, H = 3; if image I is a grayscale image, H = 1.

[0069] Step 5, total variational constraint super-resolution reconstruction, aims to improve the detail and resolution of the image.

[0070] First, the enhanced result Q = {q} obtained from steps 1-4 is... jThe input (x,y)} is fed into a blind super-resolution reconstruction model, which serves as the existing model. If the low-contrast image I is a color image, Q = {q1(x,y),q2(x,y),q3(x,y)}; if the low-contrast image I is a grayscale image, Q = {q1(x,y)}. After blind super-resolution reconstruction, the output is the super-resolution reconstruction result Y and the fuzzy degradation kernel estimate K. Then, the optimized super-resolution reconstruction result X is obtained through total variation constraints. The expression for the total variation constraints can be expressed as:

[0071] min ‖X‖ TV +λ‖XY‖ TV ,stKX=Q, (7)

[0072] Where λ≥0 is the trade-off parameter. Equation (7) can be solved using the Alternating Direction Multiplier Method (ADMM), the soft thresholding iteration method, etc., to obtain the final blind super-resolution reconstruction result with total variational constraints. TV It can be replaced by the 1-norm, i.e., ||·|| TV It is ||·||1.

[0073] Steps 1-5 involve first applying multi-scale Gaussian filtering to the low-contrast image, then using an S-function to expand the imaging dynamic range of the quotient map, followed by color correction of the expanded result, and finally optimizing the image resolution and detail information using a fully variational constrained blind super-resolution reconstruction model, as shown in the attached figure. Figure 1 As shown. This can overcome the shortcomings of single image enhancement algorithms or single super-resolution reconstruction algorithms, and achieve simultaneous optimization of image contrast, resolution and detail information, which is helpful for image feature extraction, image interpretation and analysis.

[0074] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method of low-contrast image joint enhancement and super-resolution reconstruction, characterized in that, The method comprises the following steps: Step 1, on low-contrast image performing multi-scale Gaussian filtering, denotes a pixel point coordinate; Step 2, calculate the pixel-by-pixel ratio result of the low-contrast image and the Gaussian filtered image after multi-scale Gaussian filtering to obtain a quotient image , is the serial number of the Gaussian function of different scale Gaussian filtering Step 3, transforming the business graph First, a normalization transformation is performed to obtain a normalized result graph Then, the normalized result graph is subjected to an S function transformation to further obtain a weighted imaging dynamic expansion graph ; Step 4, dynamically expanding the weighted imaging of step 3 Color correction is performed to obtain a color correction map , wherein is the number of channels of the image, Step 5, enhanced results obtained according to step 4 After inputting to the blind super-resolution reconstruction model for reconstruction, outputting a super-resolution reconstruction result Y and a blur degradation kernel estimation K, and obtaining an optimized super-resolution reconstruction result X according to the super-resolution reconstruction result Y and the blur degradation kernel estimation K under total variation constraint, The multi-scale Gaussian filtering of the step 1 is based on the following formula: ， in, Indicates the first The Gaussian filtered image corresponding to the Gaussian function. This represents a multi-scale Gaussian filter function. Represents pixel coordinates. The standard deviation of the Gaussian function is represented by... standard deviations of the different scale Gaussian functions based on the following equation: standard deviations of the different scale gaussians different, The chart in step 2 Based on the following formula: wherein is positive, The normalized result map Based on the following equation: , wherein is a normalized result image, denotes a maximum value of the magnitude of the pixels of the quotient image, denotes a minimum value of the magnitude of the pixels of the quotient image, The S function transformation in the step 3 is based on the following formula: , wherein and are constants.

2. The method of joint low-contrast image enhancement and super-resolution reconstruction according to claim 1, the weighted imaging dynamic range expansion map in step 3 based on the following equation: , , wherein, is a weight for the result of the different scale Gaussian filtering and is a constant not less than 0, is the total number of Gaussian functions of the different scale Gaussian filtering.

3. The method of claim 2, wherein the color correction map in step 4 is based on the following equation: ###00006### where I is the input image, I is the output image, and C is the color correction map. where I is the input image, I is the output image, and C is the color correction map. , wherein For low contrast images The Image channels, For imaging dynamic extension map The Image channels, It is a mapping function.

4. The low-contrast image joint enhancement and super-resolution reconstruction method according to claim 3, wherein the expression of the total variation constraint is: , wherein, is a trade-off parameter and is greater than or equal to 1, is a 1-norm.

Images