A low-light image enhancement method based on block matching three-dimensional transformation

By using a block-matching 3D transformation method, the problems of preserving texture edge information and suppressing noise in low-light image enhancement are solved, achieving efficient image quality improvement. This method is applicable to the enhancement of low-light images and low-light infrared images.

CN114298943BActive Publication Date: 2024-11-22GUILIN UNIVERSITY OF TECHNOLOGY +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202111680241.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-30
Publication Date
2024-11-22
Estimated Expiration
2041-12-30

AI Technical Summary

Technical Problem

Existing low-light image enhancement methods are not effective in preserving image texture edge information and suppressing noise, and the algorithms are slow to execute.

Method used

A block-matching 3D transformation method is adopted. The low-light image is divided into similar block groups, and two-dimensional biorthogonal wavelet and discrete cosine transform are performed. Combined with Haar transform, the illumination and reflection components are reconstructed respectively. The illumination component is enhanced by exponential-logarithmic hybrid transform, and finally the image is enhanced by dot product synthesis.

Benefits of technology

It effectively preserves image texture details, suppresses noise, improves image quality, and has a fast processing speed, objectively outperforming existing algorithms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure SMS_3
    Figure SMS_3
  • Figure QLYQS_1
    Figure QLYQS_1
Patent Text Reader

Abstract

The present invention relates to a low-light image enhancement method based on block matching three-dimensional transformation, comprising the following steps: (1) taking a similar block group in a low-light image and scanning it into a three-dimensional matrix; (2) performing a three-dimensional transformation on the three-dimensional matrix, wherein a biorthogonal wavelet transform and a discrete cosine transform are respectively used in the two-dimensional transformation to respectively reconstruct an illumination component and a reflection component after inverse transformation; (3) using an exponential logarithmic mixed transformation to enhance the illumination component; (4) performing a dot multiplication of the enhanced illumination component and the reconstructed reflection component to obtain an enhanced similar block group, and stacking and recombining all blocks in the enhanced similar block group according to their original positions in the matching process to obtain an enhanced image. In view of the fact that such images have the characteristics of low resolution, low illumination, low contrast, and low visual quality, and the subsequent practical application of the images is seriously affected, the present invention can simply and effectively enhance low-light images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of image enhancement, and in particular to a low-light image enhancement method based on block matching three-dimensional transformation. Background Art

[0002] Due to the development and widespread use of devices such as smartphones, cameras and surveillance cameras, the number of images collected in low-light environments at night has begun to expand rapidly. Due to the extremely low contrast and brightness of such low-light images, their effectiveness in subsequent practical applications has been seriously affected. In order to enable the acquisition system to capture high-quality images under low-light conditions, low-light image enhancement technology has emerged and has been widely used in fields such as autonomous driving, photography, military, target detection and surveillance.

[0003] Low-light image enhancement is a popular research field in computer vision and image processing. Its purpose is to improve the visual effects of low-light images, increase the clarity of images, and make them more suitable for specific applications than the original images, such as providing high-quality clear images for subsequent video image detection, tracking or recognition.

[0004] Most existing low-light image enhancement methods rely on the Retinex theory model, which assumes that image intensity is the product of reflectivity and illumination coefficient. Algorithms based on the Retinex theory calculate the illumination component by removing the reflection component. In recent years, deep learning methods have been introduced. First, a neural network model is trained to estimate the decomposed illumination and reflection, and then the neural network is used to test low-light images.

[0005] There is a Retinex model (STAR) that uses structure and texture perception. Due to the low contrast, low resolution and high noise characteristics of low-light images, low-light image enhancement has always been a challenging problem. This method is a Retinex model (STAR) for illumination and reflectivity decomposition of a single image. The proposed model solves the problem of low-light image enhancement to a certain extent. The STAR model mainly decomposes the image into illumination component and reflection component through local derivatives, enhances the illumination component, and synthesizes the illumination and reflection to obtain the enhanced image. The model alleviates the over-smoothing of the illumination component, thereby reducing the sharpening phenomenon of the enhanced image to a certain extent.

[0006] Another existing technology is BM3D, which is proposed for the problem of image denoising. It applies the non-local idea to the field of image denoising. Since the prior characteristics of image enhancement and image denoising are similar, we combine its block matching idea with biorthogonal wavelets and discrete cosine transform and apply it to image enhancement.

[0007] Since the above-mentioned prior art only processes the illumination component after decomposing the image into illumination and reflection components, the image enhancement result of this method cannot well preserve the texture edge information of the image, and the noise suppression in extremely low-light image enhancement is not ideal. In addition, the optimization method adopted in the later stage of the model also causes the algorithm to be slow during execution. Summary of the invention

[0008] The present invention proposes a low-light image enhancement problem based on a block matching three-dimensional transformation method. This invention aims to solve the problem that such images have the characteristics of low resolution, low illumination, low contrast and poor visual quality, which seriously affects the subsequent practical application of the images. The present invention can enhance the low-light images simply and effectively.

[0009] The technical solution of the present invention is achieved by the following methods:

[0010] A low-light image enhancement method based on block matching three-dimensional transformation comprises the following steps: (1) dividing the low-light image into N blocks, performing block matching in sequence according to a certain step length to obtain N similar block groups, and recording the positions of the blocks in the similar block groups in the image;

[0011] (2) Perform a two-dimensional biorthogonal wavelet transform on each block in the similar block group, and then perform a Haar transform between blocks. Use the transformed low-frequency coefficients to reconstruct the illumination component;

[0012] Perform a 2D discrete cosine transform on each block in the similar block group, and then perform a Haar transform between blocks. Use the transformed high-frequency coefficients to reconstruct the reflection component;

[0013] (3) Use exponential logarithmic mixed transformation to enhance the illumination component;

[0014] (4) Performing a dot product of the enhanced illumination component and the reconstructed reflection component obtains an enhanced similar block group. All blocks in the enhanced similar block group are put back into the stack according to their original positions in the matching process to obtain an enhanced image.

[0015] The step (1) of the present invention includes dividing the low-light image into N blocks of size N1×N1, extracting an image block of size N1×N1 as a reference block on the low-light image according to a specified step size Nstep, dividing a neighborhood of size Ns×Ns centered on the reference block in the low-light image, respectively calculating the Euclidean distance between the reference block and other blocks in the neighborhood, and selecting (N2-1) blocks with the smallest distance to form a similar block group together with the reference block. The matched similar block groups are scanned into three-dimensional matrices M of size (N1×N1)×N2; the obtained similar block groups are non-locally similar in the image, so the transformation enhancement operation on the block group can avoid the problem of artifacts and false signals in the processed image.

[0016] In the step (2) of the present invention, discrete cosine transform is used in the two-dimensional transform of the three-dimensional transform of the reconstructed reflection component.

[0017] The biorthogonal wavelet transform is used in the two-dimensional transform to reconstruct the three-dimensional transform of the illumination components.

[0018] The step (2) described in the present invention also includes performing a corresponding inverse transformation on the transformed coefficient matrix while retaining its low-frequency coefficients to obtain a reconstructed illumination component I; and performing a corresponding inverse transformation on the transformed coefficient matrix while retaining its high-frequency coefficients to obtain a reconstructed reflection component R.

[0019] The present invention adopts exponential logarithmic mixed transformation to enhance the illumination component, and the enhanced illumination component can reduce the exposure of the image in addition to enhancing the dark area.

[0020] .

[0021] Since the biorthogonal wavelet transform has the ability to characterize local information of the signal in both the time domain and the frequency domain, the low-frequency coefficients obtained by combining it with the three-dimensional transform on the similar block group can well retain the smooth information of the source image. Therefore, in order to obtain a better illumination component, the present invention selects the biorthogonal wavelet transform in the two-dimensional transform when reconstructing the three-dimensional transform of the illumination component. The obtained similar block group is transformed in three dimensions, wherein the two-dimensional one adopts the discrete cosine transform and the three-dimensional one adopts the Haar transform.

[0022] Since the discrete cosine transform has a periodic characteristic, its transformation can concentrate strong energy in the low-frequency part of the spectrum, while the remaining high-frequency part can better retain the texture and edge contour information of the source image. In order to obtain a better reflection component, the present invention selects the discrete cosine transform in the two-dimensional transformation when reconstructing the three-dimensional transformation of the reflection component. The obtained similar block group is transformed in three dimensions, wherein the discrete cosine transform is used in two dimensions and the Haar transform is used in three dimensions.

[0023] Compared with the existing technology, the beneficial effects of the present invention are as follows: since the present invention does not need to determine the estimated reflectivity and illumination rate globally in advance, the proposed method adopts different transformations to reconstruct the illumination component and the reflection component, which is simple in theory, fast and effective, and the enhanced result can effectively suppress noise and well preserve texture details. In addition to being able to enhance natural low-light images well, the method proposed in the present invention also has good performance in enhancing low-light infrared images. The subjective quality of the enhanced image is very natural and very consistent with the visual effect of the human eye. The image enhancement result is also far superior to the best existing image enhancement algorithm in terms of objective evaluation. DETAILED DESCRIPTION

[0024] The present invention is further described in detail below based on specific embodiments.

[0025] A low-light image enhancement method based on block matching three-dimensional transformation includes the following steps: (1) taking a similar block group in a low-light image and scanning it into a three-dimensional matrix;

[0026] (2) Perform a two-dimensional biorthogonal wavelet transform on each block in the similar block group, and then perform a Haar transform between blocks. Use the transformed low-frequency coefficients to reconstruct the illumination component;

[0027] Perform a 2D discrete cosine transform on each block in the similar block group, and then perform a Haar transform between blocks. Use the transformed high-frequency coefficients to reconstruct the reflection component;

[0028] (3) Use exponential logarithmic mixed transformation to enhance the illumination component;

[0029] (4) Performing a dot product of the enhanced illumination component and the reconstructed reflection component obtains an enhanced similar block group. All blocks in the enhanced similar block group are put back into the stack according to their original positions in the matching process to obtain an enhanced image.

[0030] The step (1) of the present invention includes dividing the low-light image into N blocks of size N1×N1, extracting an image block of size N1×N1 as a reference block on the low-light image according to a specified step size Nstep, dividing a neighborhood of size Ns×Ns centered on the reference block in the low-light image, calculating the Euclidean distance between the reference block and other blocks in the neighborhood, and selecting (N2-1) blocks with the smallest distance to form a similar block group together with the reference block. The matched similar block groups are scanned into three-dimensional matrices M of size (N1×N1)×N2; the distribution of the obtained similar block groups in the image is non-locally similar, so the transformation enhancement operation on the block group can avoid the problem of artifacts and false signals in the processed image.

[0031] In the step (2) of the present invention, discrete cosine transform is used in the two-dimensional transform of the three-dimensional transform of the reconstructed reflection component.

[0032] The biorthogonal wavelet transform is used in the two-dimensional transform to reconstruct the three-dimensional transform of the illumination components.

[0033] The step (2) described in the present invention also includes performing a corresponding inverse transformation on the transformed coefficient matrix while retaining its low-frequency coefficients to obtain a reconstructed reflection component I; and performing a corresponding inverse transformation on the transformed coefficient matrix while retaining its high-frequency coefficients to obtain a reconstructed reflection component R.

[0034] The present invention adopts exponential logarithmic mixed transformation to enhance the illumination component, and the enhanced illumination component can reduce the exposure of the image in addition to enhancing the dark area.

[0035] .

[0036] Different images were selected and data were compared using the method of the present invention and the method using the structure and texture perception Retinex model (STAR) (referred to as comparative example) as follows:

[0037] Table 1 Comparison of NIQE values ​​of low-light image enhancement results

[0038]

[0039] It can be seen from the data in the table that the NIQE values ​​of the method results of the present invention are much lower than those of the comparative method for all images.

Claims

1. A low-light image enhancement method based on block matching three-dimensional transformation, Features: The following steps are involved: (1) Divide the low-light image into N blocks, perform block matching in sequence according to a certain step size to obtain N similar block groups, and record the positions of the blocks in the similar block groups in the image; (2) Perform a two-dimensional biorthogonal wavelet transform on each block of the similar block group, then perform a Haar transform between blocks, and use the transformed low-frequency coefficients to reconstruct the illumination component; Perform a two-dimensional discrete cosine transform on each block of the similar block group, then perform a Haar transform between blocks, and use the transformed high-frequency coefficients to reconstruct the reflection component; (3) Use exponential logarithmic mixed transformation to enhance the illumination component; (4) Performing a dot product of the enhanced illumination component and the reconstructed reflection component obtains an enhanced similar block group. All blocks in the enhanced similar block group are put back into the stack according to their original positions in the matching process to obtain an enhanced image.

2. The low-light image enhancement method based on block matching three-dimensional transformation according to claim 1, Features: The step (1) comprises dividing the low-light image into N blocks of size N1×N1, extracting an image block of size N1×N1 as a reference block on the low-light image according to a specified step size Nstep, dividing a neighborhood of size Ns×Ns with the reference block as the center in the low-light image, calculating the Euclidean distance between the reference block and other blocks in the neighborhood, selecting (N2-1) blocks with the smallest distance to form a similar block group together with the reference block, and scanning the matched similar block groups into a three-dimensional matrix M of size (N1×N1)×N2.

3. The low-light image enhancement method based on block matching three-dimensional transformation according to claim 1, Features: The step (2) further includes performing a corresponding inverse transformation on the transformed coefficient matrix while retaining its low-frequency coefficients to obtain a reconstructed illumination component I; and performing a corresponding inverse transformation on the transformed coefficient matrix while retaining its high-frequency coefficients to obtain a reconstructed reflection component R.

4. The low-light image enhancement method based on block matching three-dimensional transformation according to claim 1, Features: The illumination component is enhanced by using an exponential logarithmic mixed transformation. The enhanced illumination component can reduce the exposure of the image in addition to improving the dark area. 。

Citation Information

Patent Citations

  • Non-local transform domain image enhancement method

    CN102708545A

  • Non-local Haar transform image denoising method

    CN109493295A

  • Low-illumination image enhancement and denoising method and system and computer equipment

    CN110298792A

  • Night low-brightness image enhancement system

    CN111311503A

  • Dark light image enhancement method based on Retinex and attention mechanism

    CN113052814A