A dark vision image enhancement method based on Retinex and image fusion

Abstract

The present invention belongs to the field of computer vision and image processing technology, and specifically relates to a scotopic image enhancement method based on Retinex and image fusion. The method comprises: obtaining the V component of an original scotopic image and performing an improved Retinex enhancement process to obtain a first processed image; designing an adaptive brightness compensation strategy based on the V component to obtain a second processed image; performing contrast compensation based on the second processed image to obtain a third processed image; and performing multi-scale fusion of the three processed images based on image brightness, image gradient, and exposure to obtain an enhanced V component, thereby obtaining a final enhanced image. The present invention can achieve a relatively good image enhancement effect, achieving a good balance between brightness enhancement, contrast enhancement, and naturalness preservation, so that the enhanced image is more consistent with human visual characteristics.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and particularly relates to a dark vision image enhancement method based on Retinex theory and image fusion technology. Background Art

[0002] In our daily lives and production, it is inevitable to capture digital images at night to obtain information at that time. Taking video surveillance as an example, images captured in nighttime environments have complex light sources and varying intensities. This results in images with numerous dark areas, extremely low brightness values, and a large amount of noise. This severely reduces the contrast resolution of human vision, making it difficult to observe useful information in the image. Therefore, recovering the details and structural information of images in dark vision environments is a serious challenge.

[0003] Compared with general low-illumination images, images in dark vision environments have lower grayscale values and very small grayscale differences, usually within dozens of levels. In addition, the images have both low overall grayscale values and low local grayscale values. The existing methods cannot well meet the needs of image enhancement in dark vision environments. Nowadays, many experts and scholars at home and abroad have proposed a large number of low-illumination image enhancement methods, but image enhancement algorithms suitable for dark vision environments need further design and verification, and existing methods also have many shortcomings. For example: the method based on histogram equalization can effectively improve the contrast and has a fast processing speed, but it is prone to color cast and loses detail information due to grayscale merging; the method based on Retinex theory is prone to "halo" phenomenon in areas with strong illumination changes such as the edges of the image; although the method based on the dehazing model can improve the visual quality to a certain extent, the enhanced image often does not conform to the actual scene and is prone to artifacts at the edges. Summary of the Invention

[0004] In view of this, the present invention provides a dark vision image enhancement method based on Retinex and image fusion, which can improve the brightness and contrast of images in dark vision environments, making the images look more natural. The technical solution steps of the present invention include the following:

[0005] Acquire an original scotopic image and extract a V component of the original scotopic image;

[0006] performing improved Retinex enhancement on the V component of the original scotopic image to obtain a first processed image;

[0007] performing adaptive brightness compensation on the V component of the original scotopic image to obtain a second processed image;

[0008] performing contrast compensation on the second processed image to obtain a third processed image;

[0009] The three processed images are fused at multiple scales according to the brightness weight, image gradient weight, and image exposure weight to obtain the final enhanced dark vision image V component.

[0010] Beneficial effects of the present invention:

[0011] The present invention utilizes an improved joint bilateral filter for Retinex enhancement, which can better preserve edges of sudden illumination changes in scotopic images and avoid haloing in the enhanced results caused by oversmoothing. The present invention also demonstrates great flexibility in scotopic image enhancement through adaptive adjustment of the grayscale transformation parameters of the non-complete Beta function, further enhancing image brightness. The present invention combines image brightness distribution weights, image gradient weights, and image exposure weights, applying them to existing Laplacian pyramid fusion methods to produce a fused image and ultimately an enhanced image. This method achieves enhanced image clarity, uniform overall brightness, higher contrast, and greater image detail. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only part of the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 This is a flow chart of a dark vision image enhancement method based on Retinex and multi-scale fusion of the present invention;

[0014] Figure 2 1 is a schematic diagram of the improved Retinex enhancement process of the present invention;

[0015] Figure 3 is a schematic diagram of illumination component estimation according to the present invention;

[0016] Figure 4 is an image collected by the present invention in a dark visual environment;

[0017] Figure 5 This is the effect diagram after image enhancement by the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] This application example provides a dark vision image enhancement method based on Retinex and multi-scale fusion. The specific implementation process is as follows: Figure 1 As shown, in this example, the present invention obtains an original image; then performs an improved Retinex enhancement process to obtain a first processed image; performs a designed adaptive brightness compensation strategy to obtain a second processed image; performs contrast compensation to obtain a third processed image; and finally, performs multi-scale fusion of the three processed images based on the designed image weight information to obtain the target image. Specifically, the following steps are included:

[0020] S1: Acquire an original scotopic image and extract a V component of the original scotopic image;

[0021] Among them, dark vision image refers to the use of image acquisition equipment in an environment with a brightness lower than 0.001cd / m 2 The original image is captured in an imaging environment at that time, and the average background grayscale of the original image is between 0 and 47. The present invention does not limit the specific content of the original image. For example, the original image can be an image containing a person object, an image containing a car object, or an image containing an environmental object.

[0022] There are many ways to obtain the original image, such as using a camera to capture the original image that meets the requirements by controlling the brightness of the ambient light.

[0023] In one embodiment, an existing dataset, such as the LOL dataset, may be selected to select images that meet the requirements as original images.

[0024] In an embodiment of the present invention, it is necessary to extract the V component from the HSV channel of the original image, wherein the parameters represented by HSV are: hue (H), saturation (S), and brightness (V). Hue H: measured by angle, with a value range of 0° to 360, and red, green, and blue are 120 degrees apart. Complementary colors differ by 180 degrees. In human perception of color, the most significant and important aspect can indeed be said to be hue; saturation S: indicates the degree of color purity, with a value range of 0.0 to 1.0, and when S=0, there is only grayscale; brightness V: indicates the lightness and darkness of the color, with a value range of 0.0 (black) to 1.0 (white); by extracting the V component that represents brightness, the brightness information in the dark vision image can be better obtained. S2: Perform improved Retinex enhancement on the V component of the original dark vision image to obtain a first processed image;

[0025] In some embodiments, as Figure 2 As shown, the improved Retinex enhancement can be performed using the following three steps (1), (2), and (3):

[0026] (1) Convert the original image from RGB color space to HSV color space

[0027] Specifically, the following existing formula can be used for conversion:

[0028] V=max(R,G,B)

[0029]

[0030]

[0031] Here, R represents the value of the R channel in the input RGB image, G represents the value of the G channel in the input RGB image, and B represents the value of the B channel in the input RGB image. This is the matrix of the image obtained by inputting the original image into the R, G, and B channels. The purpose of extracting the V component of the original image is to process only the brightness channel V of the image without changing the hue H and saturation S.

[0032] (2) Estimate the illumination component of the image based on the V component.

[0033] In the example of the present invention, the Gaussian filter for estimating the image illumination component in the original Retinex theory is replaced by an improved joint bilateral filter.

[0034] Specifically, the improved joint bilateral filtering includes: using the Y channel image of the original image in the YCbCr space as the guide image of the joint filtering and as the basis for calculating the range weight; introducing the image structure similarity index in the spatial domain similarity measurement of the joint bilateral filtering, using the image structure similarity index to calculate the joint bilateral filtering range weight, and calculating the joint bilateral filtering output image between the guide image and the V component of the original dark vision image through the normalization coefficient of the structural similarity parameter, which is the incident component L required in the Retinex theory.

[0035] The Y component of the original image can be obtained using the following calculation:

[0036] Y=0.257*R+0.504*G+0.098*B+16

[0037] Among them, R represents the value of the R channel in the input RGB image, G represents the value of the G channel in the input RGB image, and B represents the value of the B channel in the input RGB image, that is, the matrix of the image obtained by inputting the original image into the R, G, and B channels.

[0038] The introduced structural index SSIM can be obtained using the following transformation formula:

[0039]

[0040] in, Represented in pixels and The grayscale mean and variance of the square neighborhood centered at , Indicated in pixels and is the grayscale covariance of the neighborhood, C1 and C2 are constants to prevent the denominator from being zero.

[0041] The corresponding structural similarity parameter can be calculated by the above structural index. for:

[0042]

[0043] The structural similarity parameter is normalized, and the corresponding normalization coefficient K SM for:

[0044]

[0045] Using structural similarity parameters and normalization coefficient K SM , the improved joint bilateral filtering similarity measure function can be expressed as:

[0046]

[0047] in Represent the corresponding pixel points on the guide image With pixels Gray value, σ r Is the standard deviation of the similarity factor that controls the grayscale range. Through the above improved joint bilateral filtering similarity measure function, the expression of the improved joint bilateral filtering output image can be obtained as follows:

[0048]

[0049] Among them, Ω represents the pixel set, K p represents the normalization factor, I represents the input image, that is, the V component of the original dark vision image, Ω represents the pixel set, I q Represents the grayscale value of pixel q on the V component of the original dark vision image.

[0050] Further expansion yields:

[0051]

[0052] Among them, σ s is the standard deviation of the control spatial proximity factor, K p is the normalization factor:

[0053]

[0054] (x, y) and (u, v) are the coordinates of pixel p and pixel q respectively.

[0055] (3) Calculate the reflection component of the original V channel image according to the Retinex algorithm to obtain the enhanced image.

[0056] Specifically, the Retinex theory decomposes the original dark vision image into two different images: the reflection image and the illumination image, which can be estimated by estimating the illumination component, such as Figure 3 As shown, the reflection component of the image essence can be obtained by the following formula:

[0057]

[0058] Where R is the reflection component (reflection image), I is the V component of the original dark vision image, and L is the incident component (illumination image, which is the JBF[I] in the above step (2)). p ), δ is a minimum value to prevent the denominator from being 0.

[0059] S3: performing adaptive brightness compensation on the V component of the original scotopic image to obtain a second processed image;

[0060] Specifically, JND is used to adaptively control the values of α and β parameters in the non-complete Beta function and is used for grayscale adjustment of the image.

[0061] Specifically, the non-complete Beta function transformation is as follows:

[0062]

[0063] Among them, (α,u)∈[0,10], K is the original image pixel, and F(u) is the image after grayscale transformation.

[0064] The enhancement steps are as follows:

[0065] (1) Normalize the pixel values of the image;

[0066]

[0067] Among them, f′(x,y) represents the normalized grayscale value of the image, max(G) and min(G) represent the maximum and minimum grayscale values of the original image, respectively.

[0068] (2) Using JND adaptive control to control the values of α and β parameters in the incomplete Beta function;

[0069] The JND expression for the parameters controlling the non-complete Beta function is:

[0070]

[0071] Among them, T(x,y) represents the value of JND changing with illumination, L(x,y) represents the background brightness of the image, and the V component of the image is used.

[0072] The JND value is normalized and its expression is:

[0073]

[0074] Determine the values of control parameters α and β, which are expressed as follows:

[0075]

[0076]

[0077] Where ρ is the mean value of the image background brightness, and the expression is:

[0078]

[0079] (3) Using the improved non-complete Beta function to enhance the normalized image;

[0080] g′(x,y)=F(f′(x,y))

[0081] (4) Perform inverse transformation on the enhanced image to obtain the output image g″(x,y).

[0082] g″(x,y)={max(G′)-min(G′)}*g′(x,y)+min(G′)

[0083] Among them, max(G′)=255, min(G′)=0.

[0084] S4: performing contrast compensation on the second processed image to obtain a third processed image;

[0085] Specifically, this embodiment may adopt a contrast-limited adaptive histogram equalization (CLAHE) algorithm to obtain the third processed image.

[0086] Of course, the present invention may also adopt other contrast compensation methods, which are not listed here one by one.

[0087] S5: Perform multi-scale fusion on the three processed images according to the brightness weight, image gradient weight, and image exposure weight to obtain the final enhanced dark vision image V component.

[0088] Specifically, this embodiment performs multi-scale fusion based on the brightness weight, image gradient weight, and exposure weight of the processed image.

[0089] The three weight value expressions are:

[0090] (1) Determine the brightness weight W B,k (x,y) expression:

[0091] W B,k (x,y)=1-H k (x,y)

[0092] H k (x,y)=|V k (x,y)-m k (x,y)|

[0093]

[0094] Among them, H k (x, y) represents the absolute value of the kth fused image and the mean of the fused images, V k (x,y) represents the value of the V component of the k-th fused image, m k (x,y) represents the mean of the fused image, N is the number of fused input images, and its value is 3;

[0095] (2) Determine the image gradient WG,k (x,y) weight:

[0096]

[0097]

[0098]

[0099]

[0100] Among them, G k (x,y) is the gradient value of the kth processed image, are the gradients of the image in the x and y directions at the point (x, y), respectively.

[0101] (3) Determine the image exposure W E,k (x,y) weight:

[0102]

[0103]

[0104] Among them, I k (x,y) represents the mean of the normalized input image, and the standard deviation σ is 0.2

[0105] Determine the final weight W k (x,y):

[0106] W k (x,y)=W B,k (x,y)*W G,k (x,y)*W E,k (x,y)

[0107] In a preferred embodiment of the present invention, an existing Laplacian pyramid fusion method is used to fuse the input processed image and the weight map to determine a fused image.

[0108] The formula used for Laplace pyramid fusion is:

[0109]

[0110] W k (x,y)=W B,k (x,y)*W G,k (x,y)*W E,k (x,y)

[0111]

[0112] Among them, C represents the number of layers of pyramid decomposition, k represents the fusion input image index, that is, the processing image index, W k (x, y) represents the final weight value of the k-th processed image, W B,k (x,y) represents the brightness weight of the kth processed image, W G,k (x, y) represents the image gradient weight of the kth processed image, W E,k (x,y) represents the image exposure weight of the kth processed image, F C (x,y) represents the fused image of the Cth layer in the pyramid, G C represents the C-th level Gaussian pyramid decomposition, LP C represents the C-th layer image of the Laplace pyramid, F(x) is the final multi-scale fusion result, ↑ d Indicates that the process adopts upsampling method, where d = 2 C-1 .

[0113] In a preferred embodiment of the present invention, a dark vision image enhancement method based on Retinex and image fusion further includes step S6; specifically comprising:

[0114] S6: Convert the fused image from the HSV color space back to the RGB color space to obtain an enhanced dark vision image.

[0115] Specifically, the following existing formula can be used for conversion:

[0116] C=V*S

[0117]

[0118] m=VC

[0119]

[0120] (R,G,B)=((R ′ +m)*255,(G ′ +m)*255,(B+m)*255)

[0121] Figure 4 This is an image collected under a dark visual environment. The image has extremely low brightness and contrast, and the effective information of the image is submerged. Figure 4 As shown; Figure 5 This is the effect diagram after the image enhancement of the present invention. The enhanced image is specifically characterized by clear image details, obvious contrast, moderate exposure, and no problems such as local overexposure or over-darkening, and is more beautiful. Figure 5 shown.

[0122] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0123] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A dark vision image enhancement method based on Retinex and image fusion, characterized in that: The method steps include: Acquire an original scotopic image and extract a V component of the original scotopic image; performing improved Retinex enhancement on the V component of the original scotopic image to obtain a first processed image; The improved Retinex enhancement of the V component of the original scotopic image includes: performing illumination estimation on the V component of the original scotopic image using an improved joint bilateral filter to obtain an incident component of the original scotopic image, using the original image and the incident component as inputs of a Retinex algorithm, calculating a reflectance component of the original scotopic image, and obtaining a first processed image; The improved joint bilateral filtering includes: using the Y channel image of the original scotopic image in the YCbCr space as a guide image for joint filtering, introducing an image structure similarity index in the joint bilateral filtering spatial domain similarity metric, and obtaining a structure similarity parameter according to the structure similarity index; calculating a joint bilateral filtering output image between the guide image and the V component of the original scotopic image by using the structure similarity parameter and its normalized coefficient, which is the incident component L required in the Retinex algorithm; performing adaptive brightness compensation on the V component of the original scotopic image to obtain a second processed image; performing adaptive brightness compensation on the V component of the original scotopic image includes: using JND to adaptively control the values of α and β parameters in an incomplete Beta function, and using the JND to adjust the grayscale of the original scotopic image to obtain a second processed image; wherein JND represents a just resolvable difference that changes with image illumination; performing contrast compensation on the second processed image to obtain a third processed image; The three processed images are fused at multiple scales according to the brightness weight, image gradient weight, and image exposure weight to obtain the final enhanced dark vision image V component.

2. The dark vision image enhancement method based on Retinex and image fusion according to claim 1, characterized in that: The output image of the joint bilateral filtering is expressed as: Among them, K p represents the normalization factor, I represents the input image, that is, the V component of the original dark vision image, Ω represents the pixel set, I q Represents the grayscale value of pixel q on the V component of the original dark vision image, Represents the joint bilateral filtering weight distribution function between pixel q and pixel p on the V component of the original dark vision image, Indicates the corresponding pixel point on the guide image With pixels The improved joint bilateral filtering weight distribution function is specifically expressed as: Indicates the corresponding pixel point on the guide image With pixels The structural similarity parameter between Represent the corresponding pixel points on the guide image With pixels Gray value, σ r is the standard deviation of the grayscale range similarity factor; K SM represents the normalization coefficient of the structural similarity parameter; the structural similarity parameter is specifically expressed as: in, Represents the structural similarity index, specifically expressed as: in, Represented in pixels and The grayscale mean and variance of the square neighborhood centered at , Indicated in pixels and is the grayscale covariance of the neighborhood, C1 and C2 are constants to prevent the denominator from being zero.

3. The dark vision image enhancement method based on Retinex and image fusion according to claim 1, characterized in that: The Retinex algorithm expression of the first processed image is: L=JBF[I] p Where R is the reflection component, I is the V component of the original dark vision image, L is the incident component, and JBF[I] p Represents the output image of joint bilateral filtering; δ is the minimum value.

4. The dark vision image enhancement method based on Retinex and image fusion according to claim 3, characterized in that: The expressions of the α and β parameters in the non-complete Beta function using JND adaptive control are: Where J is the JND-normalized value of the image, and ρ is the mean background brightness of the image.

5. The dark vision image enhancement method based on Retinex and image fusion according to claim 1, characterized in that: The formula used for multi-scale fusion is expressed as: W k (x,y)=W B,k (x,y)*W G,k (x,y)*W E,k (x,y) Among them, C represents the number of layers of pyramid decomposition, k represents the fusion input image index, that is, the processing image index, W k (x, y) represents the final weight value of the kth processed image, I k (x,y) represents the mean of the normalized input image, W B,k (x,y) represents the brightness weight of the kth processed image, W G,k (x, y) represents the image gradient weight of the kth processed image, W E,k (x,y) represents the image exposure weight of the kth processed image, F C (x,y) represents the fused image of the Cth layer in the pyramid, G C represents the C-th level Gaussian pyramid decomposition, LP C represents the C-th layer image of the Laplace pyramid, F(x) is the final multi-scale fusion result, ↑ d Indicates that the process adopts upsampling method, d = 2 C-1 .

6. The dark vision image enhancement method based on Retinex and image fusion according to claim 1 or 5, characterized in that: The brightness weights, image gradient weights, and image exposure weights of the three processed images include: Brightness weight W B,k (x,y) is represented as: W B,k (x,y)=1-H k (x,y) H k (x,y)=|V k (x,y)-m k (x,y)| Among them, H k (x, y) represents the absolute value of the kth processed image and the mean of all processed images, V k (x, y) represents the value of the V component of the k-th processed image, m k (x,y) represents the mean of the kth processed image, N is the number of fused input images, that is, the number of processed images, and its value is 3; Image gradient weight W G,k (x,y) is represented as: Among them, G k (x,y) is the gradient value of the kth processed image, The gradients of the image in the x and y directions at the (x, y) point are processed respectively; Image exposure W E,k (x,y) weight: Among them, I k (x,y) represents the mean of the normalized input image, and σ represents the standard deviation.

Images